标题： 高效通信的方差减少随机梯度下降 显示英文标题

标题： Communication-efficient Variance-reduced Stochastic Gradient Descent

摘要： 我们考虑通信高效的分布式优化问题，其中多个节点在每次迭代中交换重要的算法信息以解决大规模问题。 特别是，我们关注随机方差减少梯度，并提出一种新方法使其通信高效。 也就是说，在保持原始未压缩算法的线性收敛速率的同时，将通信的信息压缩为少量比特。 对真实数据集的全面理论和数值分析表明，我们的算法可以显著减少通信复杂度，最多可达95%，而几乎没有明显的代价。 此外，与目前最先进的用于解决分布式优化问题的算法相比，它在量化方面（在保持真实最小值和收敛速率方面）要更加稳健。 我们的结果对于在物联网和移动网络上使用机器学习具有重要的意义。 显示英文摘要

摘要： We consider the problem of communication efficient distributed optimization where multiple nodes exchange important algorithm information in every iteration to solve large problems. In particular, we focus on the stochastic variance-reduced gradient and propose a novel approach to make it communication-efficient. That is, we compress the communicated information to a few bits while preserving the linear convergence rate of the original uncompressed algorithm. Comprehensive theoretical and numerical analyses on real datasets reveal that our algorithm can significantly reduce the communication complexity, by as much as 95\%, with almost no noticeable penalty. Moreover, it is much more robust to quantization (in terms of maintaining the true minimizer and the convergence rate) than the state-of-the-art algorithms for solving distributed optimization problems. Our results have important implications for using machine learning over internet-of-things and mobile networks.