标题： DiffMark：基于扩散的对抗深度伪造的鲁棒水印 显示英文标题

标题： DiffMark: Diffusion-based Robust Watermark Against Deepfakes

摘要： 深度伪造通过恶意面部操作对安全和隐私构成重大威胁。 虽然稳健的水印技术有助于真实性验证和来源追踪，但现有方法在对抗深度伪造操作方面往往缺乏足够的鲁棒性。 扩散模型在图像生成方面表现出色，使得在生成过程中可以无缝融合水印与图像。 在本研究中，我们提出了一种基于扩散模型的新型稳健水印框架，称为DiffMark。 通过修改训练和采样方案，我们将面部图像和水印作为条件，引导扩散模型逐步去噪并生成相应的水印图像。 在构建面部条件时，我们通过一个时间步依赖的因子对面部图像进行加权，随着噪声的减少逐渐降低引导强度，从而更好地适应扩散模型的采样过程。 为了实现水印条件的融合，我们引入了一个跨信息融合（CIF）模块，该模块利用可学习的嵌入表来自适应地提取水印特征，并通过交叉注意力将其与图像特征相结合。 为了增强水印对抗深度伪造操作的鲁棒性，我们在训练阶段集成了一个冻结的自动编码器来模拟深度伪造操作。 此外，我们引入了对抗深度伪造的引导机制，该机制使用特定的深度伪造模型来对抗性地引导扩散采样过程，以生成更稳健的水印图像。 实验结果表明，所提出的DiffMark在典型深度伪造任务中是有效的。 我们的代码将发布在https://github.com/vpsg-research/DiffMark。 显示英文摘要

摘要： Deepfakes pose significant security and privacy threats through malicious facial manipulations. While robust watermarking can aid in authenticity verification and source tracking, existing methods often lack the sufficient robustness against Deepfake manipulations. Diffusion models have demonstrated remarkable performance in image generation, enabling the seamless fusion of watermark with image during generation. In this study, we propose a novel robust watermarking framework based on diffusion model, called DiffMark. By modifying the training and sampling scheme, we take the facial image and watermark as conditions to guide the diffusion model to progressively denoise and generate corresponding watermarked image. In the construction of facial condition, we weight the facial image by a timestep-dependent factor that gradually reduces the guidance intensity with the decrease of noise, thus better adapting to the sampling process of diffusion model. To achieve the fusion of watermark condition, we introduce a cross information fusion (CIF) module that leverages a learnable embedding table to adaptively extract watermark features and integrates them with image features via cross-attention. To enhance the robustness of the watermark against Deepfake manipulations, we integrate a frozen autoencoder during training phase to simulate Deepfake manipulations. Additionally, we introduce Deepfake-resistant guidance that employs specific Deepfake model to adversarially guide the diffusion sampling process to generate more robust watermarked images. Experimental results demonstrate the effectiveness of the proposed DiffMark on typical Deepfakes. Our code will be available at https://github.com/vpsg-research/DiffMark.