标题： ARMOR：通过细致推理对齐安全和安全的大语言模型 显示英文标题

标题： ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning

摘要： 大型语言模型（LLMs）已展现出惊人的生成能力。 然而，它们容易被滥用，引发了重大的安全担忧。 尽管已经广泛采用训练后的安全对齐方法，LLMs 仍容易受到恶意指令的攻击，这些指令可以绕过安全约束。 最近的努力引入了推理时的安全推理（系统-2对齐），其中 LLM 在最终响应前进行推理过程以执行安全验证。 然而，我们发现这些检查是由临时推理驱动的，偏离了结构化的人类过程，在这种过程中，他们首先辨别用户的真正意图，然后根据真正的意图评估相关风险。 因此，这些防御措施仍然容易受到精心设计的越狱提示，这些提示将有害目标隐藏在看似无害的语言中。 为了构建安全且可靠的 LLM，我们提出了一种基于推理的安全对齐框架 ARMOR，该框架将临时的思维链推理过程替换为与人类对齐的结构化过程。 在推理时，ARMOR（1）检测可能的越狱策略，（2）提取用户的的核心意图并忽略欺骗性指令，（3）对净化后的请求应用基于策略的安全分析。 ARMOR 在适应性越狱攻击和多个安全基准上进行了评估，并进行了测试时的扩展以进一步提高其性能。 结果表明，ARMOR 显著增强了对最新适应性越狱攻击的鲁棒性，并在各种安全基准上优于近期基于推理对齐的模型。 显示英文摘要

摘要： Large Language Models (LLMs) have demonstrated remarkable generative capabilities. However, their susceptibility to misuse has raised significant safety concerns. While post-training safety alignment methods have been widely adopted, LLMs remain vulnerable to malicious instructions that can bypass safety constraints. Recent efforts have introduced inference-time safety reasoning (system-2 alignment), where LLMs conduct a reasoning process to perform safety verification before final response. We show, however, that these checks are driven by ad-hoc reasoning that diverges from the structured human process, where they first discern a user's true intent, then evaluate the associated risk based on the true intent. Consequently, these defenses remain vulnerable to sophisticated jailbreak prompts that cloak harmful goals in seemingly benign language. To build secure and safe LLMs, we propose a reasoning-based safety alignment framework, ARMOR, that replaces the ad-hoc chains of thought reasoning process with human-aligned, structured one. At inference, ARMOR (1) detects likely jailbreak strategies, (2) extracts the user's core intent while discarding deceptive instructions, and (3) applies a policy-grounded safety analysis to the purified request. ARMOR is evaluated on adaptive jailbreak attacks and multiple safety benchmarks, and a test-time scaling is conducted to further improve its performance. Results demonstrate that ARMOR significantly enhances the robustness against state-of-the-art adaptive jailbreak attacks and outperforms recent reasoning-based aligned models across various safety benchmarks.