标题： 隐藏在噪声中：通过潜在声学模式触发器揭示音频大语言模型对齐中的后门 显示英文标题

标题： Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment through Latent Acoustic Pattern Triggers

摘要： 随着音频大型语言模型（ALLMs）成为语音处理的强大工具，其安全影响亟需关注。 尽管已有大量研究探索了文本和视觉安全，但音频的独特特性带来了重大挑战。 本文首先进行如下研究： ALLM是否容易受到利用声学触发器的后门攻击？ 针对这一问题，我们引入了隐藏在噪声中的（HIN），这是一种新颖的后门攻击框架，旨在利用细微的、音频特有的特征。 HIN对原始音频波形进行声学修改，例如时间动态的改变和有策略地注入频谱定制的噪声。 这些变化会引入一致的模式，ALLM的声学特征编码器会捕捉到这些模式，在音频流中嵌入强大的触发器。 为了评估ALLM对基于音频特征的触发器的鲁棒性，我们开发了AudioSafe基准测试，评估九种不同的风险类型。 在AudioSafe和三个已建立的安全数据集上的广泛实验揭示了现有ALLMs中的关键漏洞：（I）如环境噪声和语音速率变化等音频特征平均攻击成功率超过90%。（II）ALLM在不同声学特征上表现出显著的敏感性差异，特别是对音量作为触发器的响应最小，（III）中毒样本的包含仅导致损失曲线的微小波动，突显了攻击的隐蔽性。 显示英文摘要

摘要： As Audio Large Language Models (ALLMs) emerge as powerful tools for speech processing, their safety implications demand urgent attention. While considerable research has explored textual and vision safety, audio's distinct characteristics present significant challenges. This paper first investigates: Is ALLM vulnerable to backdoor attacks exploiting acoustic triggers? In response to this issue, we introduce Hidden in the Noise (HIN), a novel backdoor attack framework designed to exploit subtle, audio-specific features. HIN applies acoustic modifications to raw audio waveforms, such as alterations to temporal dynamics and strategic injection of spectrally tailored noise. These changes introduce consistent patterns that an ALLM's acoustic feature encoder captures, embedding robust triggers within the audio stream. To evaluate ALLM robustness against audio-feature-based triggers, we develop the AudioSafe benchmark, assessing nine distinct risk types. Extensive experiments on AudioSafe and three established safety datasets reveal critical vulnerabilities in existing ALLMs: (I) audio features like environment noise and speech rate variations achieve over 90% average attack success rate. (II) ALLMs exhibit significant sensitivity differences across acoustic features, particularly showing minimal response to volume as a trigger, and (III) poisoned sample inclusion causes only marginal loss curve fluctuations, highlighting the attack's stealth.