标题： 变分OOD状态校正用于离线强化学习 显示英文标题

标题： Variational OOD State Correction for Offline Reinforcement Learning

摘要： 离线强化学习的性能显著受到状态分布偏移问题的影响，而分布外（OOD）状态校正是一种流行的方法来解决这个问题。 在本文中，我们提出了一种名为密度感知安全感知（DASP）的新方法用于OOD状态校正。 具体来说，我们的方法鼓励智能体优先选择导致更高数据密度结果的动作，从而促进其在分布内（安全）区域内的操作或返回。 为了实现这一点，我们在变分框架内优化目标，同时考虑决策的潜在结果及其密度，从而为安全决策提供关键的上下文信息。 最后，我们通过在离线MuJoCo和AntMaze套件上的广泛实验评估验证了我们所提出方法的有效性和可行性。 显示英文摘要

摘要： The performance of Offline reinforcement learning is significantly impacted by the issue of state distributional shift, and out-of-distribution (OOD) state correction is a popular approach to address this problem. In this paper, we propose a novel method named Density-Aware Safety Perception (DASP) for OOD state correction. Specifically, our method encourages the agent to prioritize actions that lead to outcomes with higher data density, thereby promoting its operation within or the return to in-distribution (safe) regions. To achieve this, we optimize the objective within a variational framework that concurrently considers both the potential outcomes of decision-making and their density, thus providing crucial contextual information for safe decision-making. Finally, we validate the effectiveness and feasibility of our proposed method through extensive experimental evaluations on the offline MuJoCo and AntMaze suites.