标题： MAVEN：多智能体变分探索 显示英文标题

标题： MAVEN: Multi-Agent Variational Exploration

摘要： 集中式训练与去中心化执行是合作深度多智能体强化学习中的一个重要设置，这是由于执行期间的通信限制和训练中的计算可行性。 本文分析了在复杂环境中表现优异的价值函数方法[43]。我们特别关注QMIX[40]，这是该领域的当前最先进技术。我们证明，QMIX及其类似方法对联合动作值施加的表示约束会导致探索效果明显变差且产生次优解。 此外，我们提出了一种名为MAVEN的新方法，通过引入潜在空间实现值函数方法和策略方法的混合，以实现分层控制。基于值函数的代理根据分层策略控制的共享潜在变量调整自身行为。这使得MAVEN能够实现承诺性的时间扩展探索，这对于解决复杂的多智能体任务至关重要。我们的实验结果表明，MAVEN在具有挑战性的SMAC领域[43]取得了显著的性能提升。 显示英文摘要

摘要： Centralised training with decentralised execution is an important setting for cooperative deep multi-agent reinforcement learning due to communication constraints during execution and computational tractability in training. In this paper, we analyse value-based methods that are known to have superior performance in complex environments [43]. We specifically focus on QMIX [40], the current state-of-the-art in this domain. We show that the representational constraints on the joint action-values introduced by QMIX and similar methods lead to provably poor exploration and suboptimality. Furthermore, we propose a novel approach called MAVEN that hybridises value and policy-based methods by introducing a latent space for hierarchical control. The value-based agents condition their behaviour on the shared latent variable controlled by a hierarchical policy. This allows MAVEN to achieve committed, temporally extended exploration, which is key to solving complex multi-agent tasks. Our experimental results show that MAVEN achieves significant performance improvements on the challenging SMAC domain [43].