Title: HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models Show Chinese title

Title: HandDiffuse：基于扩散模型的双手交互生成控制器

Abstract: Existing hands datasets are largely short-range and the interaction is weak due to the self-occlusion and self-similarity of hands, which can not yet fit the need for interacting hands motion generation. To rescue the data scarcity, we propose HandDiffuse12.5M, a novel dataset that consists of temporal sequences with strong two-hand interactions. HandDiffuse12.5M has the largest scale and richest interactions among the existing two-hand datasets. We further present a strong baseline method HandDiffuse for the controllable motion generation of interacting hands using various controllers. Specifically, we apply the diffusion model as the backbone and design two motion representations for different controllers. To reduce artifacts, we also propose Interaction Loss which explicitly quantifies the dynamic interaction process. Our HandDiffuse enables various applications with vivid two-hand interactions, i.e., motion in-betweening and trajectory control. Experiments show that our method outperforms the state-of-the-art techniques in motion generation and can also contribute to data augmentation for other datasets. Our dataset, corresponding codes, and pre-trained models will be disseminated to the community for future research towards two-hand interaction modeling. Show Chinese abstract

Abstract: 现有的手部数据集大多局限于短距离且交互较弱，这是由于手的自遮挡和自相似性造成的，目前还不能满足生成交互手部运动的需求。为了缓解数据匮乏的问题，我们提出了HandDiffuse12.5M，这是一个新的数据集，包含具有强烈双手交互的时间序列。HandDiffuse12.5M在现有双手数据集中规模最大，交互最丰富。我们进一步提出了一种强大的基线方法HandDiffuse，用于使用各种控制器进行可控的交互手部运动生成。具体来说，我们将扩散模型作为主干，并设计了两种不同的控制器使用的运动表示方法。为了减少伪影，我们还提出了交互损失，它明确量化了动态交互过程。我们的HandDiffuse能够实现各种具有生动双手交互的应用，即运动插值和轨迹控制。实验表明，我们的方法在运动生成方面优于最先进的技术，并且还可以为其他数据集的数据增强做出贡献。我们的数据集、相应的代码和预训练模型将分发给社区，以促进未来关于双手交互建模的研究。