Title: Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization Show Chinese title

Title: 上下文感知的TFL：一种用于时间伪造定位的通用上下文感知对比学习框架

Abstract: Most research efforts in the multimedia forensics domain have focused on detecting forgery audio-visual content and reached sound achievements. However, these works only consider deepfake detection as a classification task and ignore the case where partial segments of the video are tampered with. Temporal forgery localization (TFL) of small fake audio-visual clips embedded in real videos is still challenging and more in line with realistic application scenarios. To resolve this issue, we propose a universal context-aware contrastive learning framework (UniCaCLF) for TFL. Our approach leverages supervised contrastive learning to discover and identify forged instants by means of anomaly detection, allowing for the precise localization of temporal forged segments. To this end, we propose a novel context-aware perception layer that utilizes a heterogeneous activation operation and an adaptive context updater to construct a context-aware contrastive objective, which enhances the discriminability of forged instant features by contrasting them with genuine instant features in terms of their distances to the global context. An efficient context-aware contrastive coding is introduced to further push the limit of instant feature distinguishability between genuine and forged instants in a supervised sample-by-sample manner, suppressing the cross-sample influence to improve temporal forgery localization performance. Extensive experimental results over five public datasets demonstrate that our proposed UniCaCLF significantly outperforms the state-of-the-art competing algorithms. Show Chinese abstract

Abstract: 多媒体取证领域的大多数研究努力都集中在检测伪造的音视频内容上，并取得了显著成果。然而，这些工作仅将深度伪造检测视为分类任务，而忽略了视频部分片段被篡改的情况。嵌入真实视频中的小段假音频和视频的时序伪造定位（TFL）仍然具有挑战性，并且更符合现实应用场景。 为了解决这个问题，我们提出了一种通用的上下文感知对比学习框架（UniCaCLF），用于TFL。我们的方法利用监督对比学习通过异常检测发现并识别伪造瞬间，从而实现对时序伪造片段的精确定位。为此，我们提出了一个新颖的上下文感知感知层，该层利用异构激活操作和自适应上下文更新器构建上下文感知对比目标，通过将其与真实瞬间特征在全局上下文距离上的对比来增强伪造瞬间特征的可分辨性。还引入了高效的上下文感知对比编码，以进一步在监督逐样本的方式下推动真实瞬间与伪造瞬间之间瞬时特征区分能力的极限，抑制跨样本影响以提高时序伪造定位性能。 在五个公共数据集上的广泛实验结果表明，我们提出的UniCaCLF显著优于最先进的竞争算法。