标题： GRASP：使用自适应奇异参数替换冗余层以实现高效的模型压缩 显示英文标题

标题： GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression

摘要： 近期研究表明，大型语言模型（LLMs）中的许多层是功能冗余的，通过移除这些层可以实现模型压缩以减少推理成本。尽管此类方法可以提高效率，但无差别地剪枝层往往会导致显著的性能下降。本文提出了一种名为GRASP（基于梯度的自适应奇异值保留）的新颖压缩框架，通过保留敏感性感知的奇异值来缓解此问题。与直接剪枝层不同，GRASP利用小规模校准数据集上的基于梯度的归因方法自适应地识别和保留关键的奇异分量。通过用最小数量的参数替换冗余层，GRASP实现了高效压缩，同时保持了强大的性能且开销极小。在多个LLMs上的实验表明，GRASP始终优于现有的压缩方法，在20%的压缩比下达到了原始模型90%的性能。 显示英文摘要

摘要： Recent studies have demonstrated that many layers are functionally redundant in large language models (LLMs), enabling model compression by removing these layers to reduce inference cost. While such approaches can improve efficiency, indiscriminate layer pruning often results in significant performance degradation. In this paper, we propose GRASP (Gradient-based Retention of Adaptive Singular Parameters), a novel compression framework that mitigates this issue by preserving sensitivity-aware singular values. Unlike direct layer pruning, GRASP leverages gradient-based attribution on a small calibration dataset to adaptively identify and retain critical singular components. By replacing redundant layers with only a minimal set of parameters, GRASP achieves efficient compression while maintaining strong performance with minimal overhead. Experiments across multiple LLMs show that GRASP consistently outperforms existing compression methods, achieving 90% of the original model's performance under a 20% compression ratio.