标题： 强化学习中的单次策略迁移 显示英文标题

标题： Single Episode Policy Transfer in Reinforcement Learning

摘要： 在强化学习（RL）中，迁移到新的未知环境动态是一项关键挑战。一个更大的挑战是在测试时单次尝试中近乎最优地执行，这可能没有密集奖励的访问权限，而当前需要多次经验回放以实现适应的方法无法解决这一问题。为了在一个具有相关动态的环境家族中实现单次试用迁移，我们提出了一种通用算法，该算法优化了一个探测器和一个推理模型，以便快速估计测试动态下的潜在变量，然后这些潜在变量立即作为通用控制策略的输入。这种模块化方法能够集成最先进的变分推理或强化学习算法。此外，我们的方法不需要在测试时访问奖励，使其能够在现有自适应方法无法工作的环境中运行。在具有单次试用测试约束的不同实验领域中，我们的方法显著优于现有的自适应方法，并且在鲁棒迁移方面表现出有利的性能。 显示英文摘要

摘要： Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience rollouts for adaptation. To achieve single episode transfer in a family of environments with related dynamics, we propose a general algorithm that optimizes a probe and an inference model to rapidly estimate underlying latent variables of test dynamics, which are then immediately used as input to a universal control policy. This modular approach enables integration of state-of-the-art algorithms for variational inference or RL. Moreover, our approach does not require access to rewards at test time, allowing it to perform in settings where existing adaptive approaches cannot. In diverse experimental domains with a single episode test constraint, our method significantly outperforms existing adaptive approaches and shows favorable performance against baselines for robust transfer.