标题： 变分OOD状态校正用于离线强化学习 显示英文标题

标题： Variational OOD State Correction for Offline Reinforcement Learning

摘要： 离线强化学习的性能受到状态分布偏移问题的显著影响，而状态分布外（OOD）的状态校正是一种流行的解决此问题的方法。 在本文中，我们提出了一种名为密度感知安全感知（DASP）的新方法用于状态分布外校正。 具体而言，我们的方法鼓励智能体优先选择导致更高数据密度结果的动作，从而促使其操作在分布内（安全）区域之内或返回该区域。 为了实现这一点，我们在一个变分框架下优化目标，同时考虑决策可能带来的结果及其密度，从而为安全决策提供关键上下文信息。 最后，我们通过在离线MuJoCo和AntMaze套件上的大量实验评估验证了所提出方法的有效性和可行性。 显示英文摘要

摘要： The performance of Offline reinforcement learning is significantly impacted by the issue of state distributional shift, and out-of-distribution (OOD) state correction is a popular approach to address this problem. In this paper, we propose a novel method named Density-Aware Safety Perception (DASP) for OOD state correction. Specifically, our method encourages the agent to prioritize actions that lead to outcomes with higher data density, thereby promoting its operation within or the return to in-distribution (safe) regions. To achieve this, we optimize the objective within a variational framework that concurrently considers both the potential outcomes of decision-making and their density, thus providing crucial contextual information for safe decision-making. Finally, we validate the effectiveness and feasibility of our proposed method through extensive experimental evaluations on the offline MuJoCo and AntMaze suites.