标题： SG2VID：场景图实现视频合成的精细控制 显示英文标题

标题： SG2VID: Scene Graphs Enable Fine-Grained Control for Video Synthesis

摘要： 外科模拟在培训新手外科医生方面起着至关重要的作用，可以加快他们的学习曲线并减少术中错误。然而，传统的模拟工具在提供必要的照片真实感和人体解剖学的变异性方面存在不足。为此，当前的方法正转向基于生成模型的模拟器。然而，这些方法主要集中在使用越来越复杂的条件来实现精确合成，而忽略了精细的人类控制方面。为了解决这一差距，我们介绍了SG2VID，这是首个利用场景图进行精确视频合成和精细人类控制的扩散模型。我们在三个公开数据集上展示了SG2VID的能力，这些数据集涵盖了白内障和胆囊切除手术。尽管SG2VID在定性和定量上都优于先前的方法，但它还能够实现精确的合成，对工具和解剖结构的大小和运动、新工具的引入以及整体场景布局提供准确的控制。我们还通过定性的方式展示了SG2VID如何用于生成增强，并通过一个实验展示了当训练集扩展为我们的合成视频时，它在下游阶段检测任务中的能力提升。最后，为了展示SG2VID保留人类控制的能力，我们与场景图交互以生成新的视频样本，展示重大但罕见的术中异常情况。 显示英文摘要

摘要： Surgical simulation plays a pivotal role in training novice surgeons, accelerating their learning curve and reducing intra-operative errors. However, conventional simulation tools fall short in providing the necessary photorealism and the variability of human anatomy. In response, current methods are shifting towards generative model-based simulators. Yet, these approaches primarily focus on using increasingly complex conditioning for precise synthesis while neglecting the fine-grained human control aspect. To address this gap, we introduce SG2VID, the first diffusion-based video model that leverages Scene Graphs for both precise video synthesis and fine-grained human control. We demonstrate SG2VID's capabilities across three public datasets featuring cataract and cholecystectomy surgery. While SG2VID outperforms previous methods both qualitatively and quantitatively, it also enables precise synthesis, providing accurate control over tool and anatomy's size and movement, entrance of new tools, as well as the overall scene layout. We qualitatively motivate how SG2VID can be used for generative augmentation and present an experiment demonstrating its ability to improve a downstream phase detection task when the training set is extended with our synthetic videos. Finally, to showcase SG2VID's ability to retain human control, we interact with the Scene Graphs to generate new video samples depicting major yet rare intra-operative irregularities.