标题： Intra-DP：面向移动边缘计算的高性能协作推理系统 显示英文标题

标题： Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing

摘要： 在资源受限的移动设备上部署深度神经网络（DNN）面临重大挑战，尤其是在实现实时性能的同时应对有限的计算资源和电池寿命。 虽然移动边缘计算（MEC）通过与GPU服务器协作推理提供了一个有前途的解决方案，但现有方法主要依赖于逐层模型分割，并且由于DNN操作的顺序执行而经历显著的传输瓶颈。 为了解决这一挑战，我们提出了Intra-DP，这是一个针对MEC上DNN推理优化的高性能协作推理系统。 Intra DP采用了一种基于本地操作符的新并行计算技术（即，其最小单位输入不是整个输入张量的操作符，如卷积核）。 通过将它们的计算（操作）分解为几个独立的子操作，并通过并行执行重叠不同子操作的计算和传输，Intra-DP缓解了MEC中的传输瓶颈，实现了快速且节能的推理。 评估结果表明，与最先进的基线相比，Intra-DP将每次推理的延迟降低了高达50%，能耗降低了高达75%，而不会牺牲准确性。 显示英文摘要

摘要： Deploying deep neural networks (DNNs) on resource-constrained mobile devices presents significant challenges, particularly in achieving real-time performance while simultaneously coping with limited computational resources and battery life. While Mobile Edge Computing (MEC) offers collaborative inference with GPU servers as a promising solution, existing approaches primarily rely on layer-wise model partitioning and undergo significant transmission bottlenecks caused by the sequential execution of DNN operations. To address this challenge, we present Intra-DP, a high-performance collaborative inference system optimized for DNN inference on MEC. Intra DP employs a novel parallel computing technique based on local operators (i.e., operators whose minimum unit input is not the entire input tensor, such as the convolution kernel). By decomposing their computations (operations) into several independent sub-operations and overlapping the computation and transmission of different sub-operations through parallel execution, Intra-DP mitigates transmission bottlenecks in MEC, achieving fast and energy-efficient inference. The evaluation demonstrates that Intra-DP reduces per-inference latency by up to 50% and energy consumption by up to 75% compared to state-of-the-art baselines, without sacrificing accuracy.