标题： 基于多模态基础模型的歌唱语音深度伪造源归因研究 显示英文标题

标题： Towards Source Attribution of Singing Voice Deepfake with Multimodal Foundation Models

摘要： 在这项工作中，我们引入了歌唱语音深度伪造源归因（SVDSA）任务。我们假设多模态基础模型（MMFMs），如ImageBind、LanguageBind，在SVDSA中最为有效，因为它们通过跨模态预训练，能够更好地捕捉细微的源特定特性——例如每个歌唱语音深度伪造源的独特音色、音高操作或合成伪影。我们的实验验证了MMFMs对于SVDSA最有效的假设，包括与语音基础模型和音乐基础模型的对比实验。此外，受相关研究的启发，我们还探索了基础模型（FM）融合以改进SVDSA的方法。为此，我们提出了一种新颖的框架COFFE，它采用Chernoff距离作为融合FMs的有效损失函数。通过使用MMFMs的协奏曲，COFFE在所有单独的基础模型和基线融合方法中达到了最佳性能。 显示英文摘要

摘要： In this work, we introduce the task of singing voice deepfake source attribution (SVDSA). We hypothesize that multimodal foundation models (MMFMs) such as ImageBind, LanguageBind will be most effective for SVDSA as they are better equipped for capturing subtle source-specific characteristics-such as unique timbre, pitch manipulation, or synthesis artifacts of each singing voice deepfake source due to their cross-modality pre-training. Our experiments with MMFMs, speech foundation models and music foundation models verify the hypothesis that MMFMs are the most effective for SVDSA. Furthermore, inspired from related research, we also explore fusion of foundation models (FMs) for improved SVDSA. To this end, we propose a novel framework, COFFE which employs Chernoff Distance as novel loss function for effective fusion of FMs. Through COFFE with the symphony of MMFMs, we attain the topmost performance in comparison to all the individual FMs and baseline fusion methods.