标题： 用于分布外多智能体强化学习的意外性通信 显示英文标题

标题： Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning

摘要： 将多智能体强化学习方法应用于现实场景具有挑战性，因为它可能需要智能体快速适应训练中很少或从未遇到的意外情况。 针对此类分布外设置的泛化方法仅限于更具体、受限的分布偏移实例。 为了解决分布偏移的适应问题，我们提出了Unexpected Encoding Scheme，这是一种新颖的去中心化多智能体强化学习算法，其中智能体通信“意外性”，即环境中的令人惊讶的方面。 除了由原始奖励驱动通信产生的信息外，每个智能体还会根据先前经验预测下一个观察结果，测量预测与实际遇到的观察结果之间的差异，并将此差异作为信息进行编码。 在多机器人仓库环境中的实验表明，我们提出的方法能够稳健地适应动态变化的训练环境以及分布外环境。 显示英文摘要

摘要： Applying multi-agent reinforcement learning methods to realistic settings is challenging as it may require the agents to quickly adapt to unexpected situations that are rarely or never encountered in training. Recent methods for generalization to such out-of-distribution settings are limited to more specific, restricted instances of distribution shifts. To tackle adaptation to distribution shifts, we propose Unexpected Encoding Scheme, a novel decentralized multi-agent reinforcement learning algorithm where agents communicate "unexpectedness," the aspects of the environment that are surprising. In addition to a message yielded by the original reward-driven communication, each agent predicts the next observation based on previous experience, measures the discrepancy between the prediction and the actually encountered observation, and encodes this discrepancy as a message. Experiments on multi-robot warehouse environment support that our proposed method adapts robustly to dynamically changing training environments as well as out-of-distribution environment.