标题： 带有三阶段训练的噪声去纠缠方法用于噪声鲁棒的语音识别 显示英文标题

标题： Noisy Disentanglement with Tri-stage Training for Noise-Robust Speech Recognition

摘要： 为了在嘈杂或低信噪比（SNR）条件下提升端到端（E2E）语音识别系统的性能，本文引入了NoisyD-CT，这是一种基于Conformer-Transducer架构的新三阶段训练框架。 NoisyD-CT的核心是一个特别设计的紧凑噪声解耦（NoisyD）模块（仅增加1.71M参数），集成在Conformer块和Transducer解码器之间，以实现深度噪声抑制并提高在挑战性声学噪声环境中的自动语音识别（ASR）鲁棒性。 为了充分利用NoisyD-CT的噪声抑制能力，我们进一步提出了一种干净表示一致性损失，以对齐从嘈杂语音中提取的高层表示与从对应干净语音中获得的表示。 结合噪声重建损失，这种一致性对齐使NoisyD模块能够在抑制噪声的同时保留干净和嘈杂条件下一致的关键声学和语言特征，从而生成更干净的内部表示，提升ASR性能。 此外，我们的三阶段训练策略旨在在整个模型训练过程中充分利用噪声解耦和语音识别模块的功能，最终在嘈杂条件下最大化性能提升。 我们的实验是在LibriSpeech和CHiME-4数据集上进行的，广泛的实验结果表明，我们提出的NoisyD-CT显著优于竞争性的Conformer-Transducer基线，在模拟和真实世界嘈杂测试集上分别实现了25.7%和10.6%的相对词错误率降低，同时在干净语音测试集上保持或甚至提高了性能。 源代码、模型检查点和数据模拟脚本将发布在https://github.com/litchimo/NoisyD-CT。 显示英文摘要

摘要： To enhance the performance of end-to-end (E2E) speech recognition systems in noisy or low signal-to-noise ratio (SNR) conditions, this paper introduces NoisyD-CT, a novel tri-stage training framework built on the Conformer-Transducer architecture. The core of NoisyD-CT is a especially designed compact noisy disentanglement (NoisyD) module (adding only 1.71M parameters), integrated between the Conformer blocks and Transducer Decoder to perform deep noise suppression and improve ASR robustness in challenging acoustic noise environments. To fully exploit the noise suppression capability of the NoisyD-CT, we further propose a clean representation consistency loss to align high-level representations derived from noisy speech with those obtained from corresponding clean speech. Together with a noisy reconstruction loss, this consistency alignment enables the NoisyD module to effectively suppress noise while preserving essential acoustic and linguistic features consistent across both clean and noisy conditions, thereby producing cleaner internal representations that enhance ASR performance. Moreover, our tri-stage training strategy is designed to fully leverage the functionalities of both the noisy disentanglement and speech recognition modules throughout the model training process, ultimately maximizing performance gains under noisy conditions. Our experiments are performed on the LibriSpeech and CHiME-4 datasets, extensive results demonstrate that our proposed NoisyD-CT significantly outperforms the competitive Conformer-Transducer baseline, achieving up to 25.7% and 10.6% relative word error rate reductions on simulated and real-world noisy test sets, respectively, while maintaining or even improving performance on clean speech test sets. The source code, model checkpoint and data simulation scripts will be available at https://github.com/litchimo/NoisyD-CT.