标题： 通过模仿生成的视频进行机器人操作而无需物理演示 显示英文标题

标题： Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations

摘要： 这项工作介绍了机器人模仿生成视频（RIGVid），这是一个系统，使机器人能够仅通过模仿AI生成的视频来执行复杂的操作任务——如倒水、擦拭和混合——而无需任何物理演示或机器人特定的训练。 给定一个语言命令和初始场景图像，视频扩散模型会生成潜在的示范视频，视觉语言模型（VLM）会自动过滤不符合命令的结果。 然后，6D姿态跟踪器从视频中提取物体轨迹，并以与具体实现无关的方式将轨迹重新映射到机器人上。 通过广泛的现实世界评估，我们表明经过筛选的生成视频与真实演示一样有效，并且性能随着生成质量的提高而提升。 我们还表明，依赖生成视频优于更紧凑的替代方案，例如使用VLM的关键点预测，并且强大的6D姿态跟踪优于其他轨迹提取方法，例如密集特征点跟踪。 这些发现表明，由最先进的现成模型生成的视频可以为机器人操作提供有效的监督来源。 显示英文摘要

摘要： This work introduces Robots Imitating Generated Videos (RIGVid), a system that enables robots to perform complex manipulation tasks--such as pouring, wiping, and mixing--purely by imitating AI-generated videos, without requiring any physical demonstrations or robot-specific training. Given a language command and an initial scene image, a video diffusion model generates potential demonstration videos, and a vision-language model (VLM) automatically filters out results that do not follow the command. A 6D pose tracker then extracts object trajectories from the video, and the trajectories are retargeted to the robot in an embodiment-agnostic fashion. Through extensive real-world evaluations, we show that filtered generated videos are as effective as real demonstrations, and that performance improves with generation quality. We also show that relying on generated videos outperforms more compact alternatives such as keypoint prediction using VLMs, and that strong 6D pose tracking outperforms other ways to extract trajectories, such as dense feature point tracking. These findings suggest that videos produced by a state-of-the-art off-the-shelf model can offer an effective source of supervision for robotic manipulation.