标题： 可解释性作为合规要求：受监管行业在设计成果生成中需要AI工具具备哪些功能 显示英文标题

标题： Explainability as a Compliance Requirement: What Regulated Industries Need from AI Tools for Design Artifact Generation

摘要： 人工智能（AI）工具用于自动化设计制品生成，在需求工程（RE）中被越来越多地使用，以将文本需求转换为结构化图表和模型。 尽管这些AI工具，特别是基于自然语言处理（NLP）的工具，承诺提高效率，但在监管严格的行业中，由于透明度和可追溯性至关重要，其采用仍然有限。 在本文中，我们通过与来自安全关键行业的十位从业者的半结构化访谈，研究了AI驱动的设计制品生成中的可解释性差距。 我们考察了当前基于AI的工具如何集成到工作流程中，以及由于缺乏可解释性而出现的挑战。 我们还探讨了缓解策略、它们对项目结果的影响以及改进可用性所需的功能。 我们的研究结果表明，不可解释的AI输出需要大量的手动验证，降低利益相关者的信任，难以处理领域特定的术语，破坏团队协作，并引入监管合规风险，通常会抵消预期的效率优势。 为了解决这些问题，我们确定了关键改进措施，包括源追踪、为工具生成的决策提供明确的理由、支持领域特定的适应性，并实现合规性验证。 本研究概述了在需求工程工作流中提高AI工具的透明度、可靠性和适用性的实用路线图，特别是在监管和安全关键环境中，可解释性对于采用和认证至关重要。 显示英文摘要

摘要： Artificial Intelligence (AI) tools for automating design artifact generation are increasingly used in Requirements Engineering (RE) to transform textual requirements into structured diagrams and models. While these AI tools, particularly those based on Natural Language Processing (NLP), promise to improve efficiency, their adoption remains limited in regulated industries where transparency and traceability are essential. In this paper, we investigate the explainability gap in AI-driven design artifact generation through semi-structured interviews with ten practitioners from safety-critical industries. We examine how current AI-based tools are integrated into workflows and the challenges arising from their lack of explainability. We also explore mitigation strategies, their impact on project outcomes, and features needed to improve usability. Our findings reveal that non-explainable AI outputs necessitate extensive manual validation, reduce stakeholder trust, struggle to handle domain-specific terminology, disrupt team collaboration, and introduce regulatory compliance risks, often negating the anticipated efficiency benefits. To address these issues, we identify key improvements, including source tracing, providing clear justifications for tool-generated decisions, supporting domain-specific adaptation, and enabling compliance validation. This study outlines a practical roadmap for improving the transparency, reliability, and applicability of AI tools in requirements engineering workflows, particularly in regulated and safety-critical environments where explainability is crucial for adoption and certification.