Title: HARNESS: Lightweight Distilled Arabic Speech Foundation Models Show Chinese title

Title: HARNESS：轻量级蒸馏阿拉伯语语音基础模型

Abstract: Large pre-trained speech models excel in downstream tasks but their deployment is impractical for resource-limited environments. In this paper, we introduce HArnESS, the first Arabic-centric self-supervised speech model family, designed to capture Arabic speech nuances. Using iterative self-distillation, we train large bilingual HArnESS (HL) SSL models and then distill knowledge into compressed student models (HS, HST), preserving Arabic-specific representations. We use low-rank approximation to further compact the teacher's discrete supervision into shallow, thin models. We evaluate HArnESS on Arabic ASR, Speaker Emotion Recognition (SER), and Dialect Identification (DID), demonstrating effectiveness against HuBERT and XLS-R. With minimal fine-tuning, HArnESS achieves SOTA or comparable performance, making it a lightweight yet powerful alternative for real-world use. We release our distilled models and findings to support responsible research and deployment in low-resource settings. Show Chinese abstract

Abstract: 大型预训练语音模型在下游任务中表现出色，但在资源受限的环境中部署不切实际。 在本文中，我们介绍了HArnESS，第一个以阿拉伯语为中心的自监督语音模型系列，旨在捕捉阿拉伯语语音的细微差别。 通过迭代自蒸馏，我们训练大型双语HArnESS（HL）SSL模型，然后将知识蒸馏到压缩的学生模型（HS，HST）中，保留阿拉伯语特有的表示。 我们使用低秩近似进一步将教师的离散监督压缩成浅层、薄型模型。 我们在阿拉伯语自动语音识别（ASR）、说话人情感识别（SER）和方言识别（DID）上评估HArnESS，证明其在HuBERT和XLS-R中的有效性。 经过最少的微调，HArnESS实现了最先进或相当的性能，使其成为现实世界使用的轻量级但强大的替代方案。 我们发布我们的蒸馏模型和发现，以支持资源有限环境中的负责任的研究和部署。