标题： Bisecle：持续学习中视频语言理解的绑定与分离 显示英文标题

标题： Bisecle: Binding and Separation in Continual Learning for Video Language Understanding

摘要： 前沿视觉-语言模型（VLMs）在视频理解任务中取得了显著进步。 然而，现实世界的视频通常作为连续演化的数据流存在（例如，由可穿戴眼镜捕获的动态场景），这需要模型不断适应变化的数据分布和新场景。 考虑到在新任务上微调模型的计算成本高昂，通常只更新一小部分参数，而大部分模型保持冻结。 这给大型多模态基础模型背景下的现有持续学习框架带来了新的挑战，即灾难性遗忘和更新冲突。 尽管基础模型在参数高效的持续学习方面存在困难，人类大脑中的海马体却进化出了高效的记忆形成和巩固机制。 受海马体中快速绑定和模式分离机制的启发，本文我们提出了Bisecle用于视频-语言持续学习，其中使用多方向监督模块来捕捉更多的跨模态关系，并设计了一种对比提示学习方案，以隔离任务特定知识，从而促进高效记忆存储。 绑定和分离过程进一步增强了VLMs保留复杂经验的能力，使得在视频理解任务中能够实现稳健且高效的持续学习。 我们对提出的Bisecle进行了全面评估，证明了其在多个VideoQA基准测试中减轻遗忘和增强跨任务泛化的能力。 显示英文摘要

摘要： Frontier vision-language models (VLMs) have made remarkable improvements in video understanding tasks. However, real-world videos typically exist as continuously evolving data streams (e.g., dynamic scenes captured by wearable glasses), necessitating models to continually adapt to shifting data distributions and novel scenarios. Considering the prohibitive computational costs of fine-tuning models on new tasks, usually, a small subset of parameters is updated while the bulk of the model remains frozen. This poses new challenges to existing continual learning frameworks in the context of large multimodal foundation models, i.e., catastrophic forgetting and update conflict. While the foundation models struggle with parameter-efficient continual learning, the hippocampus in the human brain has evolved highly efficient mechanisms for memory formation and consolidation. Inspired by the rapid Binding and pattern separation mechanisms in the hippocampus, in this work, we propose Bisecle for video-language continual learning, where a multi-directional supervision module is used to capture more cross-modal relationships and a contrastive prompt learning scheme is designed to isolate task-specific knowledge to facilitate efficient memory storage. Binding and separation processes further strengthen the ability of VLMs to retain complex experiences, enabling robust and efficient continual learning in video understanding tasks. We perform a thorough evaluation of the proposed Bisecle, demonstrating its ability to mitigate forgetting and enhance cross-task generalization on several VideoQA benchmarks.