标题： 回归基础：通过可达-规避安全滤波器实现无需人工干预的学习 显示英文标题

标题： Back to Base: Towards Hands-Off Learning via Safe Resets with Reach-Avoid Safety Filters

摘要： 设计能够在保证安全约束的同时完成任务的控制器仍然是一个重大挑战。 我们通常希望智能体在典型任务（如环境探索）中表现出色，同时确保它能够避免进入不安全状态，并在特定时间内返回到期望的目标。 特别是，我们受到现实世界中强化学习安全、高效无干预训练设置的启发。 通过使机器人能够在无人干预的情况下安全自主地重置到期望区域（例如充电站），我们可以提高效率并促进训练。 基于控制屏障函数的安全过滤器等方法将安全性与典型控制目标解耦，并严格保证安全性。 尽管取得了成功，但对于具有控制约束和系统不确定性的通用非线性系统的这些函数构造仍是一个开放问题。 本文介绍了一种从可达避障问题相关的价值函数获得的安全过滤器。 所提出的过滤器在最小程度上修改典型控制器的同时，避免了不安全区域，并引导系统回到期望的目标集。 通过在允许安全重置的同时保持策略性能，我们实现了高效的无干预强化学习，并推进了真实世界机器人安全训练的可行性。 我们使用软演员评论家的改进版本来演示我们的方法，以安全地训练一个摆动任务在修改后的倒立摆稳定问题上。 显示英文摘要

摘要： Designing controllers that accomplish tasks while guaranteeing safety constraints remains a significant challenge. We often want an agent to perform well in a nominal task, such as environment exploration, while ensuring it can avoid unsafe states and return to a desired target by a specific time. In particular we are motivated by the setting of safe, efficient, hands-off training for reinforcement learning in the real world. By enabling a robot to safely and autonomously reset to a desired region (e.g., charging stations) without human intervention, we can enhance efficiency and facilitate training. Safety filters, such as those based on control barrier functions, decouple safety from nominal control objectives and rigorously guarantee safety. Despite their success, constructing these functions for general nonlinear systems with control constraints and system uncertainties remains an open problem. This paper introduces a safety filter obtained from the value function associated with the reach-avoid problem. The proposed safety filter minimally modifies the nominal controller while avoiding unsafe regions and guiding the system back to the desired target set. By preserving policy performance while allowing safe resetting, we enable efficient hands-off reinforcement learning and advance the feasibility of safe training for real world robots. We demonstrate our approach using a modified version of soft actor-critic to safely train a swing-up task on a modified cartpole stabilization problem.