标题： PRISM：边缘计算中基础模型的分布式推理 显示英文标题

标题： PRISM: Distributed Inference for Foundation Models at Edge

摘要： 基础模型（FMs）在从图像分类到自然语言处理的广泛应用中取得了显著成功，但在边缘设备上的部署面临重大挑战。 这引发了对开发实用且高效的策略以将基础模型带到边缘环境的兴趣。 在本工作中，我们提出了PRISM，这是一种通信高效且计算感知的分布式Transformer推理策略，适用于边缘设备。 我们的方法利用段均值表示来近似中间输出特征，大幅减少了设备间的通信。 此外，我们重新构建了自注意力机制，以消除由于位置划分中的每个设备键/值计算引起的冗余计算，并设计了一种针对自回归模型的分区感知因果掩码方案。 我们在ViT、BERT和GPT-2上对多样化的数据集进行了评估，包括CIFAR-10、CIFAR-100、ImageNet-1k、GLUE和CBT。 我们的结果表明，通信开销显著减少（在压缩率CR=128时BERT的减少高达99.2%），每个设备的计算量也有所减少（在相同设置下BERT减少了51.24%），仅造成轻微的精度下降。 这种方法为在分布式资源受限环境中部署基础模型提供了一个可扩展且实用的解决方案。 显示英文摘要

摘要： Foundation models (FMs) have achieved remarkable success across a wide range of applications, from image classification to natural langurage processing, but pose significant challenges for deployment at edge. This has sparked growing interest in developing practical and efficient strategies for bringing foundation models to edge environments. In this work, we propose PRISM, a communication-efficient and compute-aware strategy for distributed Transformer inference on edge devices. Our method leverages a Segment Means representation to approximate intermediate output features, drastically reducing inter-device communication. Additionally, we restructure the self-attention mechanism to eliminate redundant computations caused by per-device Key/Value calculation in position-wise partitioning and design a partition-aware causal masking scheme tailored for autoregressive models. We evaluate PRISM on ViT, BERT, and GPT-2 across diverse datasets, namely CIFAR-10, CIFAR-100, ImageNet-1k, GLUE, and CBT. Our results demonstrate substantial reductions in communication overhead (up to 99.2% for BERT at compression rate CR = 128) and per-device computation (51.24% for BERT at the same setting), with only minor accuracy degradation. This method offers a scalable and practical solution for deploying foundation models in distributed resource-constrained environments.