Title: DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection Show Chinese title

Title: 深度伪造医生：诊断和治疗音频-视频伪造检测

Abstract: Generative AI advances rapidly, allowing the creation of very realistic manipulated video and audio. This progress presents a significant security and ethical threat, as malicious users can exploit DeepFake techniques to spread misinformation. Recent DeepFake detection approaches explore the multimodal (audio-video) threat scenario. In particular, there is a lack of reproducibility and critical issues with existing datasets - such as the recently uncovered silence shortcut in the widely used FakeAVCeleb dataset. Considering the importance of this topic, we aim to gain a deeper understanding of the key issues affecting benchmarking in audio-video DeepFake detection. We examine these challenges through the lens of the three core benchmarking pillars: datasets, detection methods, and evaluation protocols. To address these issues, we spotlight the recent DeepSpeak v1 dataset and are the first to propose an evaluation protocol and benchmark it using SOTA models. We introduce SImple Multimodal BAseline (SIMBA), a competitive yet minimalistic approach that enables the exploration of diverse design choices. We also deepen insights into the issue of audio shortcuts and present a promising mitigation strategy. Finally, we analyze and enhance the evaluation scheme on the widely used FakeAVCeleb dataset. Our findings offer a way forward in the complex area of audio-video DeepFake detection. Show Chinese abstract

Abstract: 生成式人工智能发展迅速，使得非常逼真的篡改视频和音频的创建成为可能。 这种进步带来了显著的安全和伦理威胁，因为恶意用户可以利用深度伪造技术传播虚假信息。 最近的深度伪造检测方法探索了多模态（音频-视频）威胁场景。 特别是，现有的数据集存在可重复性缺乏和关键问题——例如，在广泛使用的FakeAVCeleb数据集中最近发现的静音捷径。 鉴于该主题的重要性，我们旨在深入了解影响音频-视频深度伪造检测基准测试的关键问题。 我们通过三个核心基准测试支柱来审视这些挑战：数据集、检测方法和评估协议。 为了解决这些问题，我们重点关注最近的DeepSpeak v1数据集，并首次提出了一种评估协议，并使用最先进的模型对其进行基准测试。 我们引入了简单多模态基线（SIMBA），这是一种具有竞争力但极简的方法，能够探索各种设计选择。 我们还深入探讨了音频捷径的问题，并提出了一个有前景的缓解策略。 最后，我们分析并改进了广泛使用的FakeAVCeleb数据集上的评估方案。 我们的研究结果为音频-视频深度伪造检测这一复杂领域提供了前进的方向。