标题： PAIR-Net：通过预训练的音视频融合与对齐损失增强自我中心说话者检测 显示英文标题

标题： PAIR-Net: Enhancing Egocentric Speaker Detection via Pretrained Audio-Visual Fusion and Alignment Loss

摘要： 第一人称视频中的说话人检测（ASD）由于视点不稳定、运动模糊以及语音来源超出画面范围等因素，带来了独特的挑战——在这些条件下，传统的视觉为中心的方法会显著退化。 我们提出了PAIR-Net（预训练的带正则化的音频-视觉集成网络），这是一种有效的模型，它将部分冻结的Whisper音频编码器与经过微调的AV-HuBERT视觉主干相结合，以稳健地融合跨模态线索。 为了对抗模态不平衡，我们引入了一种跨模态对齐损失，该损失同步音频和视觉表示，从而在模态间实现更一致的收敛。 无需依赖多说话人上下文或理想的正面视图，PAIR-Net在Ego4D ASD基准测试中达到了76.6%的mAP，分别超过了LoCoNet和STHG 8.2%和12.9%的mAP。 我们的结果显示了预训练音频先验和基于对齐的融合在真实的第一人称条件下进行鲁棒ASD的价值。 显示英文摘要

摘要： Active speaker detection (ASD) in egocentric videos presents unique challenges due to unstable viewpoints, motion blur, and off-screen speech sources - conditions under which traditional visual-centric methods degrade significantly. We introduce PAIR-Net (Pretrained Audio-Visual Integration with Regularization Network), an effective model that integrates a partially frozen Whisper audio encoder with a fine-tuned AV-HuBERT visual backbone to robustly fuse cross-modal cues. To counteract modality imbalance, we introduce an inter-modal alignment loss that synchronizes audio and visual representations, enabling more consistent convergence across modalities. Without relying on multi-speaker context or ideal frontal views, PAIR-Net achieves state-of-the-art performance on the Ego4D ASD benchmark with 76.6% mAP, surpassing LoCoNet and STHG by 8.2% and 12.9% mAP, respectively. Our results highlight the value of pretrained audio priors and alignment-based fusion for robust ASD under real-world egocentric conditions.