标题： 视频虚拟试穿与条件扩散变换器修复画笔 显示英文标题

标题： Video Virtual Try-on with Conditional Diffusion Transformer Inpainter

摘要： 视频虚拟试穿旨在自然地将服装适配到连续视频帧中的目标人物。 这是一项具有挑战性的任务，一方面，输出视频应具有良好的时空一致性，另一方面，给定服装的细节需要在所有帧中得到良好保留。 简单地逐帧使用基于图像的试穿方法会导致由于严重不一致而产生较差的结果。 最近基于扩散的视频试穿方法虽然很少，但恰好采用了一种类似的解决方案：将时间注意力插入基于图像的试穿模型以适应视频试穿任务，这些方法已显示出改进，但仍然存在不一致问题。 在本文中，我们提出了ViTI（Video Try-on Inpainter），将视频虚拟试穿建模并实现为一个条件视频修复任务，这与之前的方法不同。 这样，我们从视频生成问题开始，而不是基于图像的试穿问题，从一开始就具有更好的时空一致性。 具体来说，首先我们构建了一个基于扩散变压器的视频修复框架，具有完整的3D时空注意力，然后我们逐步将其适应于视频服装修复，采用一系列遮罩策略和多阶段训练。 经过这些步骤，模型可以根据提示以良好的时空一致性用适当的服装像素修复被遮罩的服装区域。 最后，与其他试穿方法一样，将服装条件添加到模型中，以确保修复后的服装外观和细节符合预期。 定量和定性实验结果表明，ViTI优于先前的工作。 显示英文摘要

摘要： Video virtual try-on aims to naturally fit a garment to a target person in consecutive video frames. It is a challenging task, on the one hand, the output video should be in good spatial-temporal consistency, on the other hand, the details of the given garment need to be preserved well in all the frames. Naively using image-based try-on methods frame by frame can get poor results due to severe inconsistency. Recent diffusion-based video try-on methods, though very few, happen to coincide with a similar solution: inserting temporal attention into image-based try-on model to adapt it for video try-on task, which have shown improvements but there still exist inconsistency problems. In this paper, we propose ViTI (Video Try-on Inpainter), formulate and implement video virtual try-on as a conditional video inpainting task, which is different from previous methods. In this way, we start with a video generation problem instead of an image-based try-on problem, which from the beginning has a better spatial-temporal consistency. Specifically, at first we build a video inpainting framework based on Diffusion Transformer with full 3D spatial-temporal attention, and then we progressively adapt it for video garment inpainting, with a collection of masking strategies and multi-stage training. After these steps, the model can inpaint the masked garment area with appropriate garment pixels according to the prompt with good spatial-temporal consistency. Finally, as other try-on methods, garment condition is added to the model to make sure the inpainted garment appearance and details are as expected. Both quantitative and qualitative experimental results show that ViTI is superior to previous works.