标题： CueLearner：从相对反馈中进行自举和局部策略适应 显示英文标题

标题： CueLearner: Bootstrapping and local policy adaptation from relative feedback

摘要： 人类指导已成为增强强化学习（RL）的强大工具。 然而，传统的指导形式，如示范或二进制标量反馈，可能难以收集或信息含量低，这促使探索其他形式的人类输入。 其中，相对反馈（即对如何改进一个动作的反馈，例如“更向左”）在易用性和信息丰富性之间提供了良好的平衡。 先前的研究表明，相对反馈可以用于增强策略搜索方法。 然而，这些努力仅限于特定的策略类别，并且未能有效地利用反馈。 在本工作中，我们引入了一种从相对反馈中学习的新方法，并将其与离策略强化学习相结合。 通过在两个稀疏奖励任务上的评估，我们证明了我们的方法可以通过引导其探索过程来提高强化学习的样本效率。 此外，我们展示了它能够适应环境变化或用户偏好的变化。 最后，我们通过在稀疏奖励设置中使用我们的方法学习导航策略，证明了该方法的实际应用价值。 显示英文摘要

摘要： Human guidance has emerged as a powerful tool for enhancing reinforcement learning (RL). However, conventional forms of guidance such as demonstrations or binary scalar feedback can be challenging to collect or have low information content, motivating the exploration of other forms of human input. Among these, relative feedback (i.e., feedback on how to improve an action, such as "more to the left") offers a good balance between usability and information richness. Previous research has shown that relative feedback can be used to enhance policy search methods. However, these efforts have been limited to specific policy classes and use feedback inefficiently. In this work, we introduce a novel method to learn from relative feedback and combine it with off-policy reinforcement learning. Through evaluations on two sparse-reward tasks, we demonstrate our method can be used to improve the sample efficiency of reinforcement learning by guiding its exploration process. Additionally, we show it can adapt a policy to changes in the environment or the user's preferences. Finally, we demonstrate real-world applicability by employing our approach to learn a navigation policy in a sparse reward setting.