标题： 遥感中的持续自监督学习与掩码自编码器 显示英文标题

标题： Continual Self-Supervised Learning with Masked Autoencoders in Remote Sensing

摘要： 持续学习（CL）方法的发展，旨在从连续获取的训练数据中按顺序学习新任务，已在遥感（RS）领域引起了广泛关注。 现有的RS中的CL方法在学习新任务时，增强了对灾难性遗忘的鲁棒性。 这是通过使用大量标记的训练样本实现的，这在RS中成本高昂且并不总是可行。 为了解决这个问题，我们提出了一种新颖的持续自监督学习方法，基于掩码自编码器（称为CoSMAE）。 所提出的CoSMAE由两个组件组成：i) 数据混合；以及ii) 模型混合知识蒸馏。 数据混合与通过将当前任务的图像与之前任务的图像进行插值得到保留先前数据分布的信息有关。 模型混合知识蒸馏与通过插值其模型权重以形成知识蒸馏的教师模型，同时从过去模型和当前模型中蒸馏知识有关。 这两个组件相互补充，以在数据和模型层面正则化MAE，以促进跨任务的更好泛化并降低灾难性遗忘的风险。 实验结果表明， CoSMAE在应用于MAE的最先进CL方法上实现了高达4.94%的显著改进。 我们的代码可在以下位置公开获得：https://git.tu-berlin.de/rsim/CoSMAE。 显示英文摘要

摘要： The development of continual learning (CL) methods, which aim to learn new tasks in a sequential manner from the training data acquired continuously, has gained great attention in remote sensing (RS). The existing CL methods in RS, while learning new tasks, enhance robustness towards catastrophic forgetting. This is achieved by using a large number of labeled training samples, which is costly and not always feasible to gather in RS. To address this problem, we propose a novel continual self-supervised learning method in the context of masked autoencoders (denoted as CoSMAE). The proposed CoSMAE consists of two components: i) data mixup; and ii) model mixup knowledge distillation. Data mixup is associated with retaining information on previous data distributions by interpolating images from the current task with those from the previous tasks. Model mixup knowledge distillation is associated with distilling knowledge from past models and the current model simultaneously by interpolating their model weights to form a teacher for the knowledge distillation. The two components complement each other to regularize the MAE at the data and model levels to facilitate better generalization across tasks and reduce the risk of catastrophic forgetting. Experimental results show that CoSMAE achieves significant improvements of up to 4.94% over state-of-the-art CL methods applied to MAE. Our code is publicly available at: https://git.tu-berlin.de/rsim/CoSMAE.