标题： 衡量重要性：一种评估现实世界大语言模型应用安全风险的框架 显示英文标题

标题： Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications

摘要： 目前，大多数针对大型语言模型（LLMs）的安全测试工作集中在评估基础模型上。 然而，随着系统提示、检索管道和防护机制等组件的引入，显著影响了LLM应用的整体安全性，因此在应用层面评估安全性变得越来越重要。 在本文中，我们介绍了一个用于评估LLM系统应用级别安全性的实用框架，并通过在我们组织内的多个用例中的实际部署进行了验证。 该框架包括两个部分：(1) 开发定制化安全风险分类的原则，以及(2) 评估LLM应用中安全风险的实践方法。 我们展示了所提出的框架如何在我们的内部试点中应用，为寻求扩展其安全测试工作的组织提供了参考点。 这项工作旨在弥合AI安全领域的理论概念与实际保护LLM应用的操作现实之间的差距，为安全且可扩展的部署提供可行的指导。 显示英文摘要

摘要： Most safety testing efforts for large language models (LLMs) today focus on evaluating foundation models. However, there is a growing need to evaluate safety at the application level, as components such as system prompts, retrieval pipelines, and guardrails introduce additional factors that significantly influence the overall safety of LLM applications. In this paper, we introduce a practical framework for evaluating application-level safety in LLM systems, validated through real-world deployment across multiple use cases within our organization. The framework consists of two parts: (1) principles for developing customized safety risk taxonomies, and (2) practices for evaluating safety risks in LLM applications. We illustrate how the proposed framework was applied in our internal pilot, providing a reference point for organizations seeking to scale their safety testing efforts. This work aims to bridge the gap between theoretical concepts in AI safety and the operational realities of safeguarding LLM applications in practice, offering actionable guidance for safe and scalable deployment.