标题： 多说话人编码器的说话人分离、分离和ASR统一 显示英文标题

标题： Unifying Diarization, Separation, and ASR with Multi-Speaker Encoder

摘要： 本文提出了一种统一的多说话人编码器（UME），这是一种新颖的架构，通过共享的语音基础编码器联合学习说话人分割（SD）、语音分离（SS）和多说话人自动语音识别（ASR）任务的表示。 我们利用UME多个层的隐藏表示作为残差加权求和编码（RWSE），以有效利用不同语义层次的信息，有助于任务之间的自下而上对齐。 这种联合训练方法捕捉了任务之间的固有相互依赖关系，提高了重叠语音数据的整体性能。 我们的评估表明，UME在LibriMix评估集上显著优于专门针对SD、SS和多说话人ASR的单任务基线。 值得注意的是，对于SD，UME优于之前的研究，在Libri2Mix和Libri3Mix评估集上分别实现了1.37%和2.29%的分割错误率。 显示英文摘要

摘要： This paper presents a unified multi-speaker encoder (UME), a novel architecture that jointly learns representations for speaker diarization (SD), speech separation (SS), and multi-speaker automatic speech recognition (ASR) tasks using a shared speech foundational encoder. We leverage the hidden representations from multiple layers of UME as a residual weighted-sum encoding (RWSE) to effectively use information from different semantic levels, contributing to bottom-up alignment between tasks. This joint training approach captures the inherent interdependencies among the tasks, enhancing overall performance on overlapping speech data. Our evaluations demonstrate that UME substantially improves over the single-task baselines dedicated to SD, SS, and multi-speaker ASR on LibriMix evaluation sets. Notably, for SD, UME outperforms the previous studies, achieving diarization error rates of 1.37% and 2.29% on Libri2Mix and Libri3Mix evaluation sets, respectively.