标题： 分布混合下的联邦多任务学习 显示英文标题

标题： Federated Multi-Task Learning under a Mixture of Distributions

摘要： 随着智能手机和物联网设备生成的数据量不断增加，推动了联邦学习（FL）的发展，这是一种用于设备端机器学习模型协作训练的框架。 FL 的早期努力集中在学习一个在客户之间具有良好平均性能的全局模型，但由于本地数据分布的固有异质性，全局模型对于给定客户可能非常糟糕。 联邦多任务学习（MTL）方法可以通过制定适当的惩罚优化问题来学习个性化模型。 惩罚项可以捕捉个性化模型之间的复杂关系，但避免了对本地数据分布的明确统计假设。 在本工作中，我们提出在每个本地数据分布是未知基础分布混合的灵活假设下研究联邦 MTL。 这个假设涵盖了大多数现有的个性化 FL 方法，并为客户端-服务器和完全去中心化设置提供了联邦 EM 类似算法。 此外，它提供了一种合理的方法，在训练时未见过的客户端上提供个性化模型。 通过一种新颖的联邦代理优化框架分析了算法的收敛性，该框架可能具有普遍兴趣。 在 FL 基准测试中的实验结果表明，我们的方法提供的模型比最先进方法具有更高的准确性和公平性。 显示英文摘要

摘要： The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models. First efforts in FL focused on learning a single global model with good average performance across clients, but the global model may be arbitrarily bad for a given client, due to the inherent heterogeneity of local data distributions. Federated multi-task learning (MTL) approaches can learn personalized models by formulating an opportune penalized optimization problem. The penalization term can capture complex relations among personalized models, but eschews clear statistical assumptions about local data distributions. In this work, we propose to study federated MTL under the flexible assumption that each local data distribution is a mixture of unknown underlying distributions. This assumption encompasses most of the existing personalized FL approaches and leads to federated EM-like algorithms for both client-server and fully decentralized settings. Moreover, it provides a principled way to serve personalized models to clients not seen at training time. The algorithms' convergence is analyzed through a novel federated surrogate optimization framework, which can be of general interest. Experimental results on FL benchmarks show that our approach provides models with higher accuracy and fairness than state-of-the-art methods.