标题： MP-SL：多跳并行拆分学习 显示英文标题

标题： MP-SL: Multihop Parallel Split Learning

摘要： 联邦学习（FL）作为一种广泛采用的协议，因其在保持数据去中心化的同时促进机器学习（ML）模型的训练而脱颖而出。 然而，当处理参与设备异构性时会出现挑战，这会导致训练过程中的延迟，尤其是在资源有限的设备上。 此外，使用大量参数训练ML模型需要超出小型设备（如移动设备和物联网（IoT）设备）能力的计算和内存资源。 为了解决这些问题，引入了类似并行分割学习（SL）的技术，使多个资源受限的设备能够在资源丰富的计算节点的帮助下积极参与协同训练过程。 然而，并行SL的一个缺点是计算节点需要大量的内存分配，例如使用100个参与者训练VGG-19需要80 GB。 在本文中，我们介绍了多跳并行SL（MP-SL），这是一种模块化且可扩展的机器学习即服务（MLaaS）框架，旨在促进资源受限设备参与协同和分布式机器学习模型训练。 值得注意的是，为了减轻每个计算节点的内存需求，MP-SL支持多跳并行SL训练。 这涉及将模型分成多个部分，并以流水线方式利用多个计算节点。 大量实验验证了MP-SL处理系统异构性的能力，结果表明，多跳配置比水平扩展的一跳并行SL设置更高效，特别是在涉及更具成本效益的计算节点的场景中。 显示英文摘要

摘要： Federated Learning (FL) stands out as a widely adopted protocol facilitating the training of Machine Learning (ML) models while maintaining decentralized data. However, challenges arise when dealing with a heterogeneous set of participating devices, causing delays in the training process, particularly among devices with limited resources. Moreover, the task of training ML models with a vast number of parameters demands computing and memory resources beyond the capabilities of small devices, such as mobile and Internet of Things (IoT) devices. To address these issues, techniques like Parallel Split Learning (SL) have been introduced, allowing multiple resource-constrained devices to actively participate in collaborative training processes with assistance from resourceful compute nodes. Nonetheless, a drawback of Parallel SL is the substantial memory allocation required at the compute nodes, for instance training VGG-19 with 100 participants needs 80 GB. In this paper, we introduce Multihop Parallel SL (MP-SL), a modular and extensible ML as a Service (MLaaS) framework designed to facilitate the involvement of resource-constrained devices in collaborative and distributed ML model training. Notably, to alleviate memory demands per compute node, MP-SL supports multihop Parallel SL-based training. This involves splitting the model into multiple parts and utilizing multiple compute nodes in a pipelined manner. Extensive experimentation validates MP-SL's capability to handle system heterogeneity, demonstrating that the multihop configuration proves more efficient than horizontally scaled one-hop Parallel SL setups, especially in scenarios involving more cost-effective compute nodes.