标题： 持续学习中重播的强大作用：理论视角 显示英文标题

标题： Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective

摘要： 基于重放的方法通过在当前任务的新数据和过去数据的一个子集上同时存储和训练，展示了在连续学习（CL）中解决灾难性遗忘的优越性能。尽管这种并发重放策略被广泛使用，但这种方法是否总是最优的仍不清楚。受人类学习的启发，其中顺序重温任务有助于减轻遗忘，我们探索了与标准并发重放相比，顺序重放是否能为CL带来更大的好处。为了解决这个问题，我们在过参数化线性模型中对基于重放的CL进行了理论分析，比较了两种策略：1）并发重放，即将过去的数据和新数据一起训练；2）顺序重放，即先训练新数据，然后顺序重温过去的数据。通过明确刻画遗忘和泛化误差，我们证明了当任务相似度较低时，顺序重放的表现更好。这些见解进一步激发了一种新的混合重放方法，该方法同时训练相似任务并顺序重温不相似任务。我们对其遗忘和泛化性能进行了表征，并且我们的深度神经网络实验进一步证实了混合方法优于标准并发重放。这项工作提供了基于重放的CL的第一个全面的理论分析。 显示英文摘要

摘要： Rehearsal-based methods have shown superior performance in addressing catastrophic forgetting in continual learning (CL) by storing and training on a subset of past data alongside new data in current task. While such a concurrent rehearsal strategy is widely used, it remains unclear if this approach is always optimal. Inspired by human learning, where sequentially revisiting tasks helps mitigate forgetting, we explore whether sequential rehearsal can offer greater benefits for CL compared to standard concurrent rehearsal. To address this question, we conduct a theoretical analysis of rehearsal-based CL in overparameterized linear models, comparing two strategies: 1) Concurrent Rehearsal, where past and new data are trained together, and 2) Sequential Rehearsal, where new data is trained first, followed by revisiting past data sequentially. By explicitly characterizing forgetting and generalization error, we show that sequential rehearsal performs better when tasks are less similar. These insights further motivate a novel Hybrid Rehearsal method, which trains similar tasks concurrently and revisits dissimilar tasks sequentially. We characterize its forgetting and generalization performance, and our experiments with deep neural networks further confirm that the hybrid approach outperforms standard concurrent rehearsal. This work provides the first comprehensive theoretical analysis of rehearsal-based CL.