标题： 无线网络的可扩展多智能体强化学习算法 显示英文标题

标题： Scalable Multi-agent Reinforcement Learning Algorithm for Wireless Networks

摘要： 可扩展性是合作智能算法在大规模网络中应用的关键障碍。强化学习（RL）作为一种无模型且高效的智能算法，在通信问题中被证明是有用的，并已在通信网络中得到验证。然而，当涉及到具有有限中心化的大型网络时，不可能使用一个集中实体来对整个网络进行联合实时决策。这引入了可扩展性的挑战，而多智能体强化学习展示了应对这些挑战的机会，并将智能算法扩展到合作的大规模网络。 在本文中，我们介绍了联邦均场多智能体强化学习结构，以捕捉大规模多智能体通信场景中的问题，其中代理共享参数以形成一致性。我们展示了架构的理论基础，并通过信息多智能体模型展示了联邦频率的影响。然后，我们在需要相邻接入点之间握手以实现协作增益的协调多点环境中测试了架构的性能。我们的结果显示，该学习结构可以有效地解决大规模网络中的协作问题，并具有良好的可扩展性。我们还展示了联邦算法的有效性，并强调了在每个接入点维持个性的重要性。 显示英文摘要

摘要： Scalability is the key roadstone towards the application of cooperative intelligent algorithms in large-scale networks. Reinforcement learning (RL) is known as model-free and high efficient intelligent algorithm for communication problems and proved useful in the communication network. However, when coming to large-scale networks with limited centralization, it is not possible to employ a centralized entity to perform joint real-time decision making for entire network. This introduces the scalability challenges, while multi-agent reinforcement shows the opportunity to cope this challenges and extend the intelligent algorithm to cooperative large-scale network. In this paper, we introduce the federated mean-field multi-agent reinforcement learning structure to capture the problem in large scale multi-agent communication scenarios, where agents share parameters to form consistency. We present the theoretical basis of our architecture and show the influence of federated frequency with an informational multi-agent model. We then exam the performance of our architecture with a coordinated multi-point environment which requires handshakes between neighbour access-points to realise the cooperation gain. Our result shows that the learning structure can effectively solve the cooperation problem in a large scale network with decent scalability. We also show the effectiveness of federated algorithms and highlight the importance of maintaining personality in each access-point.