标题： 多语言语音识别使用离散标记的两步训练策略 显示英文标题

标题： Multilingual Speech Recognition Using Discrete Tokens with a Two-step Training Strategy

摘要： 预训练模型，尤其是自监督学习（SSL）模型，在自动语音识别（ASR）任务中表现出色。 虽然大多数SSL模型的应用集中在利用连续表示作为特征来训练下游任务，但离散单元的使用近年来由于其更低的存储需求和更广泛的应用范围而受到越来越多的关注。 在多语言ASR任务中，模型不同层的表示对不同语言的贡献不同，使得离散单元建模的统一变得复杂。 在本文中，我们提出了一种两阶段的训练策略，以提高预训练模型的离散标记性能，并缩小与连续表示性能的差距。 我们在XLS-R模型上验证了我们的方法，遵循Interspeech2024使用离散语音单元进行语音处理挑战的设置。 我们的方法在ML-SUPERB数据集上表现出显著的改进，使XLS-R模型的CER相对减少了44%。 这超过了由WavLM模型设定的先前基线，该模型在CER上实现了26%的相对减少。 此外，我们的方法在排行榜上的所有单系统结果中排名第一。 显示英文摘要

摘要： Pre-trained models, especially self-supervised learning (SSL) models, have demonstrated impressive results in automatic speech recognition (ASR) task. While most applications of SSL models focus on leveraging continuous representations as features for training downstream tasks, the utilization of discrete units has gained increasing attention in recent years owing to its lower storage requirements and broader range of applications. In multilingual ASR tasks, representations at different layers of the model contribute differently to various languages, complicating the unification of discrete unit modeling. In this paper, we propose a two-stage training strategy to improve the discrete token performance of pre-trained models and narrow the gap with continuous representation performance. We validate our method on the XLS-R model following the settings of Interspeech2024 Speech Processing Using Discrete Speech Unit Challenge. Our method demonstrates a significant improvement on the ML-SUPERB dataset, achieving a 44% relative reduction on CER for the XLS-R model. This surpasses the previous baseline set by the WavLM model, which achieves a 26% relative reduction on CER. Furthermore, our method achieves the first place among all the single-system results on the leaderboard.