标题： 重新思考层移除：一种结合层移除和奇异值选择的混合剪枝框架，用于高效的大型语言模型压缩 显示英文标题

标题： Rethinking Layer Removal: A Hybrid Pruning Framework Combining Layer Removal and Singular Value Selection for Efficient LLM Compression

摘要： 层移除是一种通过减少冗余和提高推理效率来压缩大型语言模型（LLMs）的有效技术。 然而，随意的剪枝会破坏表示稳定性，导致性能下降。 我们提出了GRASP（基于梯度的自适应奇异值保留），该方法保留表示关键的奇异值以缓解这些影响。 与直接的层移除不同，GRASP利用语法和语义丰富的数据集上的基于梯度的归因来指导表示关键奇异值的选择。 通过有选择地对受影响的层应用奇异值分解（SVD），GRASP在保持表示稳定性的同时实现高效的压缩，且开销最小。 在多个LLM上的实验表明，GRASP在困惑度和下游任务性能方面始终优于现有的压缩方法。 显示英文摘要

摘要： Layer removal is an effective technique for compressing large language models (LLMs) by reducing redundancy and improving inference efficiency. However, indiscriminate pruning disrupts representation stability, leading to performance degradation. We propose GRASP (Gradient-based Retention of Adaptive Singular Parameters), which preserves representation-critical singular values to mitigate these effects. Unlike direct layer removal, GRASP leverages gradient-based attribution on a syntax- and semantics-rich dataset to guide the selection of representation-critical singular values. By selectively applying singular value decomposition (SVD) to affected layers, GRASP achieves efficient compression while maintaining representation stability with minimal overhead. Experiments across multiple LLMs show that GRASP consistently outperforms existing compression methods in perplexity and downstream task performance.