标题： 为安全多模态大语言模型自动转向 显示英文标题

标题： Automating Steering for Safe Multimodal Large Language Models

摘要： 最近在多模态大语言模型（MLLMs）方面的进展已经释放出强大的跨模态推理能力，但也引发了新的安全担忧，尤其是在面对对抗性多模态输入时。 为了在推理过程中提高MLLMs的安全性，我们引入了一种模块化和自适应的推理时干预技术AutoSteer，而无需对底层模型进行微调。 AutoSteer包含三个核心组件：(1) 一种新颖的安全意识评分（SAS），能够自动识别模型内部层中最相关的安全差异；(2) 一个自适应的安全探测器，用于估计从中间表示中产生有害输出的可能性；以及(3) 一个轻量级的拒绝头，当检测到安全风险时选择性地进行干预以调节生成过程。 在LLaVA-OV和Chameleon上针对多种安全关键基准的实验表明，AutoSteer显著降低了文本、视觉和跨模态威胁的攻击成功率（ASR），同时保持了通用能力。 这些发现使AutoSteer成为多模态AI系统更安全部署的一种实用、可解释且有效的框架。 显示英文摘要

摘要： Recent progress in Multimodal Large Language Models (MLLMs) has unlocked powerful cross-modal reasoning abilities, but also raised new safety concerns, particularly when faced with adversarial multimodal inputs. To improve the safety of MLLMs during inference, we introduce a modular and adaptive inference-time intervention technology, AutoSteer, without requiring any fine-tuning of the underlying model. AutoSteer incorporates three core components: (1) a novel Safety Awareness Score (SAS) that automatically identifies the most safety-relevant distinctions among the model's internal layers; (2) an adaptive safety prober trained to estimate the likelihood of toxic outputs from intermediate representations; and (3) a lightweight Refusal Head that selectively intervenes to modulate generation when safety risks are detected. Experiments on LLaVA-OV and Chameleon across diverse safety-critical benchmarks demonstrate that AutoSteer significantly reduces the Attack Success Rate (ASR) for textual, visual, and cross-modal threats, while maintaining general abilities. These findings position AutoSteer as a practical, interpretable, and effective framework for safer deployment of multimodal AI systems.