标题： AdaSkip：用于加速长上下文大语言模型推理的自适应子层跳过方法 显示英文标题

标题： AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference

摘要： 长上下文大型语言模型（LLMs）推理变得越来越关键，这促使许多研究致力于减轻这种场景下的存储和计算成本。逐层跳过方法是一种有前景的优化方法，但在长上下文推理中很少被探索。我们观察到，现有的逐层跳过策略在应用于长上下文推理时存在一些局限性，包括无法适应模型和上下文的变化，忽视子层的重要性，以及不适用于预填充阶段。本文提出了\sysname ，一种专为长上下文推理设计的自适应子层跳过方法。\sysname 通过利用实时相似性信息自适应地识别不重要的层，实现子层级别的跳过，并加速预填充和解码阶段。通过在各种长上下文基准和模型上的广泛实验，证明了\sysname 的有效性，展示了其在现有基线上的优越推理性能。 显示英文摘要

摘要： Long-context large language models (LLMs) inference is increasingly critical, motivating a number of studies devoted to alleviating the substantial storage and computational costs in such scenarios. Layer-wise skipping methods are promising optimizations but rarely explored in long-context inference. We observe that existing layer-wise skipping strategies have several limitations when applied in long-context inference, including the inability to adapt to model and context variability, disregard for sublayer significance, and inapplicability for the prefilling phase. This paper proposes \sysname, an adaptive sublayer skipping method specifically designed for long-context inference. \sysname adaptively identifies less important layers by leveraging on-the-fly similarity information, enables sublayer-wise skipping, and accelerates both the prefilling and decoding phases. The effectiveness of \sysname is demonstrated through extensive experiments on various long-context benchmarks and models, showcasing its superior inference performance over existing baselines.