标题： 通过对齐错误的人工智能的操纵攻击：风险分析与安全案例框架 显示英文标题

标题： Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework

摘要： 前沿人工智能系统在说服、欺骗和影响人类行为方面的能力正在迅速提升，当前模型已经在特定情境中展现出人类水平的说服力和战略欺骗能力。 人类通常是网络安全系统中最薄弱的环节，一个内部部署的对齐不当的人工智能系统可能会通过操纵员工来削弱人类监督。 尽管这一威胁日益增长，但操控攻击却很少受到关注，目前尚无系统框架用于评估和缓解这些风险。 为了解决这个问题，我们详细解释了为什么操控攻击是一个重大威胁，并可能导致灾难性后果。 此外，我们提出了一种针对操控风险的安全论证框架，该框架围绕三个核心论点展开：无能、控制和可信度。 对于每个论点，我们明确了证据要求、评估方法和实施考虑因素，以便人工智能公司直接应用。 本文提供了将操控风险纳入人工智能安全治理的第一个系统方法，为人工智能公司提供了一个具体的基础，在部署前评估和缓解这些威胁。 显示英文摘要

摘要： Frontier AI systems are rapidly advancing in their capabilities to persuade, deceive, and influence human behaviour, with current models already demonstrating human-level persuasion and strategic deception in specific contexts. Humans are often the weakest link in cybersecurity systems, and a misaligned AI system deployed internally within a frontier company may seek to undermine human oversight by manipulating employees. Despite this growing threat, manipulation attacks have received little attention, and no systematic framework exists for assessing and mitigating these risks. To address this, we provide a detailed explanation of why manipulation attacks are a significant threat and could lead to catastrophic outcomes. Additionally, we present a safety case framework for manipulation risk, structured around three core lines of argument: inability, control, and trustworthiness. For each argument, we specify evidence requirements, evaluation methodologies, and implementation considerations for direct application by AI companies. This paper provides the first systematic methodology for integrating manipulation risk into AI safety governance, offering AI companies a concrete foundation to assess and mitigate these threats before deployment.