Title: Seamless Language Expansion: Enhancing Multilingual Mastery in Self-Supervised Models Show Chinese title

Title: 无缝语言扩展：增强自监督模型的多语言掌握能力

Abstract: Self-supervised (SSL) models have shown great performance in various downstream tasks. However, they are typically developed for limited languages, and may encounter new languages in real-world. Developing a SSL model for each new language is costly. Thus, it is vital to figure out how to efficiently adapt existed SSL models to a new language without impairing its original abilities. We propose adaptation methods which integrate LoRA to existed SSL models to extend new language. We also develop preservation strategies which include data combination and re-clustering to retain abilities on existed languages. Applied to mHuBERT, we investigate their effectiveness on speech re-synthesis task. Experiments show that our adaptation methods enable mHuBERT to be applied to a new language (Mandarin) with MOS value increased about 1.6 and the relative value of WER reduced up to 61.72%. Also, our preservation strategies ensure that the performance on both existed and new languages remains intact. Show Chinese abstract

Abstract: 自监督（SSL）模型在各种下游任务中表现出色。 然而，它们通常仅针对有限的语言进行开发，在现实世界中可能会遇到新语言。 为每种新语言开发一个SSL模型成本很高。 因此，弄清楚如何高效地将现有的SSL模型适应到新语言而不损害其原始能力至关重要。 我们提出了集成LoRA到现有SSL模型中的适应方法，以扩展新语言。 我们还开发了保留策略，包括数据组合和重新聚类，以保留对现有语言的能力。 应用于mHuBERT，我们研究了它们在语音重合成任务中的有效性。 实验表明，我们的适应方法使mHuBERT能够应用于新语言（普通话），MOS值提高了约1.6，WER的相对值减少了高达61.72%。 此外，我们的保留策略确保了在现有语言和新语言上的性能保持不变。