标题： 在线模型不确定性下的鲁棒多智能体强化学习 显示英文标题

标题： Online Robust Multi-Agent Reinforcement Learning under Model Uncertainties

摘要： 经过良好训练的多智能体系统在部署到现实环境时可能会失败，这是由于训练和部署环境之间的模型不匹配，这种不匹配是由环境不确定性（包括噪声或对抗性攻击）引起的。分布鲁棒马尔可夫博弈（DRMGs）通过在定义的一组环境不确定性上优化最坏情况性能来增强系统弹性。然而，当前方法受到对模拟器或大型离线数据集的依赖限制，而这些数据通常不可用。本文开创性地研究了DRMGs中的在线学习，其中智能体直接从环境交互中学习，而无需先验数据。我们引入了{\it 鲁棒乐观纳什值迭代（RONAVI）}算法，并为此设置提供了第一个可证明的保证。我们的理论分析表明，该算法实现了低遗憾，并有效地找到了由总变化散度和Kullback-Leibler散度测量的不确定性集的最优鲁棒策略。这些结果为开发真正稳健的多智能体系统建立了一条新的、实用的路径。 显示英文摘要

摘要： Well-trained multi-agent systems can fail when deployed in real-world environments due to model mismatches between the training and deployment environments, caused by environment uncertainties including noise or adversarial attacks. Distributionally Robust Markov Games (DRMGs) enhance system resilience by optimizing for worst-case performance over a defined set of environmental uncertainties. However, current methods are limited by their dependence on simulators or large offline datasets, which are often unavailable. This paper pioneers the study of online learning in DRMGs, where agents learn directly from environmental interactions without prior data. We introduce the {\it Robust Optimistic Nash Value Iteration (RONAVI)} algorithm and provide the first provable guarantees for this setting. Our theoretical analysis demonstrates that the algorithm achieves low regret and efficiently finds the optimal robust policy for uncertainty sets measured by Total Variation divergence and Kullback-Leibler divergence. These results establish a new, practical path toward developing truly robust multi-agent systems.