标题： 联邦梯度匹配追踪 显示英文标题

标题： Federated Gradient Matching Pursuit

摘要： 传统机器学习技术需要将所有训练数据集中到一个服务器或数据枢纽上。 由于通信技术的发展以及许多客户端上存在大量分散的数据，协作机器学习在提供隐私保护框架的同时已成为主要研究兴趣。 特别是，联邦学习（FL）提供了一种解决方案，在保持训练数据位于本地客户端的同时学习一个共享模型。 另一方面，在广泛的机器学习和信号处理应用中，期望的解自然具有某种结构，可以相对于某个字典表述为稀疏性。 这个问题可以被表述为带有稀疏性约束的优化问题，高效求解它一直是传统集中式环境中的主要研究课题之一。 在本文中，我们提出了一种新颖的算法框架，联邦梯度匹配追踪（FedGradMP），以解决联邦学习环境下的稀疏性约束最小化问题。 我们还将算法推广以适应各种实际的联邦学习场景，当每轮仅有一部分客户端参与时，当客户端的局部模型估计可能不准确时，或者当模型参数相对于一般字典是稀疏的时候。 我们的理论分析表明了所提出算法的线性收敛性。 进行了多种数值实验，以展示所提出的框架的巨大潜力——在许多重要场景中，无需复杂的参数调整，即可在通信轮次和计算时间上实现快速收敛。 显示英文摘要

摘要： Traditional machine learning techniques require centralizing all training data on one server or data hub. Due to the development of communication technologies and a huge amount of decentralized data on many clients, collaborative machine learning has become the main interest while providing privacy-preserving frameworks. In particular, federated learning (FL) provides such a solution to learn a shared model while keeping training data at local clients. On the other hand, in a wide range of machine learning and signal processing applications, the desired solution naturally has a certain structure that can be framed as sparsity with respect to a certain dictionary. This problem can be formulated as an optimization problem with sparsity constraints and solving it efficiently has been one of the primary research topics in the traditional centralized setting. In this paper, we propose a novel algorithmic framework, federated gradient matching pursuit (FedGradMP), to solve the sparsity constrained minimization problem in the FL setting. We also generalize our algorithms to accommodate various practical FL scenarios when only a subset of clients participate per round, when the local model estimation at clients could be inexact, or when the model parameters are sparse with respect to general dictionaries. Our theoretical analysis shows the linear convergence of the proposed algorithms. A variety of numerical experiments are conducted to demonstrate the great potential of the proposed framework -- fast convergence both in communication rounds and computation time for many important scenarios without sophisticated parameter tuning.