标题： 三元压缩用于高效通信的联邦学习 显示英文标题

标题： Ternary Compression for Communication-Efficient Federated Learning

摘要： 在不同位置存储的大量数据上进行学习在许多现实应用中是必不可少的。 然而，由于智能移动设备和物联网设备的日益普及，对隐私和安全的需求不断增加，共享数据充满了挑战。 联邦学习通过在不将分布式在多个设备上的数据上传到中央服务器的情况下联合训练全局模型，为隐私保护和安全机器学习提供了一个潜在的解决方案。 然而，现有的大多数联邦学习工作采用的是全精度权重的机器学习模型，而且几乎所有这些模型都包含大量不需要传输到服务器的冗余参数，导致通信成本过高。 为了解决这个问题，我们提出了一种联邦训练的三值量化（FTTQ）算法，该算法通过客户端上的自学习量化因子优化量化网络。 给出了量化因子收敛性的理论证明、FTTQ的无偏性以及减少的权重发散性。 基于FTTQ，我们提出了一种三值联邦平均协议（T-FedAvg），以减少联邦学习系统的上行和下行通信。 进行了实证实验，在公开的数据集上训练广泛使用的深度学习模型，我们的结果表明，所提出的T-FedAvg在减少通信成本方面是有效的，并且与传统的联邦学习算法相比，在非独立同分布数据上甚至可以实现略微更好的性能。 显示英文摘要

摘要： Learning over massive data stored in different locations is essential in many real-world applications. However, sharing data is full of challenges due to the increasing demands of privacy and security with the growing use of smart mobile devices and IoT devices. Federated learning provides a potential solution to privacy-preserving and secure machine learning, by means of jointly training a global model without uploading data distributed on multiple devices to a central server. However, most existing work on federated learning adopts machine learning models with full-precision weights, and almost all these models contain a large number of redundant parameters that do not need to be transmitted to the server, consuming an excessive amount of communication costs. To address this issue, we propose a federated trained ternary quantization (FTTQ) algorithm, which optimizes the quantized networks on the clients through a self-learning quantization factor. Theoretical proofs of the convergence of quantization factors, unbiasedness of FTTQ, as well as a reduced weight divergence are given. On the basis of FTTQ, we propose a ternary federated averaging protocol (T-FedAvg) to reduce the upstream and downstream communication of federated learning systems. Empirical experiments are conducted to train widely used deep learning models on publicly available datasets, and our results demonstrate that the proposed T-FedAvg is effective in reducing communication costs and can even achieve slightly better performance on non-IID data in contrast to the canonical federated learning algorithms.