Title: Video-Enhanced Offline Reinforcement Learning: A Model-Based Approach Show Chinese title

Title: 视频增强的离线强化学习：一种基于模型的方法

Abstract: Offline reinforcement learning (RL) enables policy optimization using static datasets, avoiding the risks and costs of extensive real-world exploration. However, it struggles with suboptimal offline behaviors and inaccurate value estimation due to the lack of environmental interaction. We present Video-Enhanced Offline RL (VeoRL), a model-based method that constructs an interactive world model from diverse, unlabeled video data readily available online. Leveraging model-based behavior guidance, our approach transfers commonsense knowledge of control policy and physical dynamics from natural videos to the RL agent within the target domain. VeoRL achieves substantial performance gains (over 100% in some cases) across visual control tasks in robotic manipulation, autonomous driving, and open-world video games. Show Chinese abstract

Abstract: 离线强化学习（RL）允许使用静态数据集进行策略优化，避免了大规模现实世界探索的风险和成本。 然而，由于缺乏环境交互，它在次优离线行为和价值估计不准确方面存在困难。 我们提出了 视频增强的离线 RL（VeoRL），一种基于模型的方法，该方法从广泛可用的未标记视频数据中构建交互式世界模型。 利用基于模型的行为指导，我们的方法将自然视频中的控制策略和物理动力学的常识知识转移到目标领域的 RL 代理中。 VeoRL 在机器人操作、自动驾驶和开放世界视频游戏的视觉控制任务中实现了显著的性能提升（某些情况下超过 100%）。