标题： 分层多任务学习的编码分布式计算 显示英文标题

标题： Coded Distributed Computing for Hierarchical Multi-task Learning

摘要： 在本文中，我们考虑了一个分层的分布式多任务学习（MTL）系统，其中分布式用户希望在多个中继层的帮助下，通过中央服务器协调学习不同的模型。由于用户需要在下行传输中下载不同的学习模型，与单任务学习系统相比，分布式MTL受到通信瓶颈的影响更加严重。为了解决这个问题，我们提出了一种编码的分层MTL方案，该方案利用了连接拓扑并引入了编码技术以减少通信负载。结果表明，所提出的方案可以显著减少中继和服务器之间上行和下行传输中的通信负载。此外，我们提供了最优上行和下行通信负载的信息论下界，并证明了可实现的上界与下界之间的差距在所有中继中的最小连接用户数之内。特别地，当网络连接拓扑可以精心设计时，所提出的方案可以达到信息论最优的通信负载。在真实数据集上的实验表明，与传统的非编码方案相比，我们提出的方案可以将整体训练时间减少17% $\sim$ 26%。 显示英文摘要

摘要： In this paper, we consider a hierarchical distributed multi-task learning (MTL) system where distributed users wish to jointly learn different models orchestrated by a central server with the help of a layer of multiple relays. Since the users need to download different learning models in the downlink transmission, the distributed MTL suffers more severely from the communication bottleneck compared to the single-task learning system. To address this issue, we propose a coded hierarchical MTL scheme that exploits the connection topology and introduces coding techniques to reduce communication loads. It is shown that the proposed scheme can significantly reduce the communication loads both in the uplink and downlink transmissions between relays and the server. Moreover, we provide information-theoretic lower bounds on the optimal uplink and downlink communication loads, and prove that the gaps between achievable upper bounds and lower bounds are within the minimum number of connected users among all relays. In particular, when the network connection topology can be delicately designed, the proposed scheme can achieve the information-theoretic optimal communication loads. Experiments on real datasets show that our proposed scheme can reduce the overall training time by 17% $\sim$ 26% compared to the conventional uncoded scheme.