标题： CL-LoRA：无重放类增量学习的连续低秩适应 显示英文标题

标题： CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning

摘要： 类增量学习（CIL）旨在按顺序学习新类别的同时保留之前已学类别知识。最近，结合预训练模型（PTMs）和参数高效微调（PEFT）的方法在无需回放的CIL中表现出显著性能，且不需要来自先前任务的示例。然而，现有的基于适配器的方法通过在PTMs中引入轻量级可学习模块来处理CIL任务，为每个新任务创建新的适配器，导致了参数冗余，并未能有效利用跨任务共享的知识。在这项工作中，我们提出了连续低秩适配（CL-LoRA），它引入了一种新颖的双适配器架构，其中\textbf{任务共享的适配器}用于学习跨任务知识，\textbf{任务特定的适配器}用于捕获每个新任务的独特特征。具体来说，共享适配器利用随机正交矩阵并通过梯度重分配的知识蒸馏来保存关键的共享知识。此外，我们为任务特定适配器引入了可学习的块状权重，以减轻任务间干扰，同时保持模型的可塑性。我们证明了CL-LoRA在多个基准测试中始终能够实现令人满意的表现，并减少了训练和推理计算，为使用预训练模型进行持续学习建立了一个更高效且可扩展的范式。 显示英文摘要

摘要： Class-Incremental Learning (CIL) aims to learn new classes sequentially while retaining the knowledge of previously learned classes. Recently, pre-trained models (PTMs) combined with parameter-efficient fine-tuning (PEFT) have shown remarkable performance in rehearsal-free CIL without requiring exemplars from previous tasks. However, existing adapter-based methods, which incorporate lightweight learnable modules into PTMs for CIL, create new adapters for each new task, leading to both parameter redundancy and failure to leverage shared knowledge across tasks. In this work, we propose ContinuaL Low-Rank Adaptation (CL-LoRA), which introduces a novel dual-adapter architecture combining \textbf{task-shared adapters} to learn cross-task knowledge and \textbf{task-specific adapters} to capture unique features of each new task. Specifically, the shared adapters utilize random orthogonal matrices and leverage knowledge distillation with gradient reassignment to preserve essential shared knowledge. In addition, we introduce learnable block-wise weights for task-specific adapters, which mitigate inter-task interference while maintaining the model's plasticity. We demonstrate CL-LoRA consistently achieves promising performance under multiple benchmarks with reduced training and inference computation, establishing a more efficient and scalable paradigm for continual learning with pre-trained models.