标题： 带有大型语言模型提示的目标实现世界模型 显示英文标题

标题： World Models with Hints of Large Language Models for Goal Achieving

摘要： 强化学习在面对长时域任务和稀疏目标时存在困难，这是由于手动奖励规范的难度。 虽然现有方法通过添加内在奖励来解决这个问题，但在具有大状态和动作空间的长时域决策任务中，它们可能无法提供有意义的指导，缺乏有目的的探索。 受人类认知的启发，我们提出了一种新的多模态基于模型的强化学习方法，名为使用大型语言模型的梦想（DLLM）。 DLLM将从语言模型中提出的提示性子目标整合到模型滚动中，以鼓励在挑战性任务中发现和达到目标。 通过在模型滚动过程中将较高的内在奖励分配给与语言模型概述的提示相一致的样本，DLLM引导智能体朝着有意义且高效的探索方向发展。 大量实验表明，在HomeGrid、Crafter和Minecraft等各种具有挑战性和稀疏奖励的环境中，DLLM分别优于最近的方法27.7%、21.1%和9.9%。 显示英文摘要

摘要： Reinforcement learning struggles in the face of long-horizon tasks and sparse goals due to the difficulty in manual reward specification. While existing methods address this by adding intrinsic rewards, they may fail to provide meaningful guidance in long-horizon decision-making tasks with large state and action spaces, lacking purposeful exploration. Inspired by human cognition, we propose a new multi-modal model-based RL approach named Dreaming with Large Language Models (DLLM). DLLM integrates the proposed hinting subgoals from the LLMs into the model rollouts to encourage goal discovery and reaching in challenging tasks. By assigning higher intrinsic rewards to samples that align with the hints outlined by the language model during model rollouts, DLLM guides the agent toward meaningful and efficient exploration. Extensive experiments demonstrate that the DLLM outperforms recent methods in various challenging, sparse-reward environments such as HomeGrid, Crafter, and Minecraft by 27.7\%, 21.1\%, and 9.9\%, respectively.