标题： 自监督学习在自动语音识别中的语言偏差 显示英文标题

标题： Language Bias in Self-Supervised Learning For Automatic Speech Recognition

摘要： 自监督学习（SSL）在深度学习中用于在不需要对数据进行昂贵标注的情况下进行训练。 最近，大型自动语音识别（ASR）模型，如XLS-R，已利用SSL同时在一百多种语言上进行训练。 然而，更深入的调查表明，XLS-R的大部分训练数据来自少数几种语言。 通过SSL学到的偏见已在多个领域中得到证实，但多语言SSL ASR中的语言偏见尚未被彻底研究。 在本文中，我们利用彩票假设（LTH）来识别XLS-R中的语言特定子网络，并在各种不同语言上测试这些子网络的性能。 我们能够证明，在微调时，XLS-R绕过了传统的语言知识，仅基于对预训练数据贡献最大的语言所学到的权重。 显示英文摘要

摘要： Self-supervised learning (SSL) is used in deep learning to train on large datasets without the need for expensive labelling of the data. Recently, large Automatic Speech Recognition (ASR) models such as XLS-R have utilised SSL to train on over one hundred different languages simultaneously. However, deeper investigation shows that the bulk of the training data for XLS-R comes from a small number of languages. Biases learned through SSL have been shown to exist in multiple domains, but language bias in multilingual SSL ASR has not been thoroughly examined. In this paper, we utilise the Lottery Ticket Hypothesis (LTH) to identify language-specific subnetworks within XLS-R and test the performance of these subnetworks on a variety of different languages. We are able to show that when fine-tuning, XLS-R bypasses traditional linguistic knowledge and builds only on weights learned from the languages with the largest data contribution to the pretraining data.