标题： 基于二阶方法的联邦学习快速训练 显示英文标题

标题： Accelerated Training of Federated Learning via Second-Order Methods

摘要： 本文探讨了联邦学习（FL）中的二阶优化方法，解决了全局模型达到最优性能所需的收敛速度慢和通信轮次数过多的关键挑战。尽管现有的联邦学习综述主要关注与统计异构性、设备标签异构性以及一阶联邦学习方法中的隐私和安全问题相关的挑战，但对模型训练缓慢的问题关注较少。这种缓慢的训练通常会导致需要过多的通信轮次或增加通信成本，特别是在客户端数据高度异构的情况下。本文研究了利用二阶优化加速训练过程的各种联邦学习方法。我们对最先进的二阶联邦学习方法进行了全面分类，并基于全局模型的收敛速度、计算成本、内存使用、传输开销和泛化能力比较了它们的性能。我们的研究结果表明，在联邦学习中通过二阶优化引入Hessian曲率的潜力，并强调了高效利用Hessian及其逆矩阵在联邦学习中的关键挑战。这项工作为未来旨在开发可扩展且高效的联邦优化方法以改进联邦学习中全局模型训练的研究奠定了基础。 显示英文摘要

摘要： This paper explores second-order optimization methods in Federated Learning (FL), addressing the critical challenges of slow convergence and the excessive communication rounds required to achieve optimal performance from the global model. While existing surveys in FL primarily focus on challenges related to statistical and device label heterogeneity, as well as privacy and security concerns in first-order FL methods, less attention has been given to the issue of slow model training. This slow training often leads to the need for excessive communication rounds or increased communication costs, particularly when data across clients are highly heterogeneous. In this paper, we examine various FL methods that leverage second-order optimization to accelerate the training process. We provide a comprehensive categorization of state-of-the-art second-order FL methods and compare their performance based on convergence speed, computational cost, memory usage, transmission overhead, and generalization of the global model. Our findings show the potential of incorporating Hessian curvature through second-order optimization into FL and highlight key challenges, such as the efficient utilization of Hessian and its inverse in FL. This work lays the groundwork for future research aimed at developing scalable and efficient federated optimization methods for improving the training of the global model in FL.