标题： 内存耗尽：基于多模态稀疏滤波网络学习主特征用于半监督视频异常检测 显示英文标题

标题： MemoryOut: Learning Principal Features via Multimodal Sparse Filtering Network for Semi-supervised Video Anomaly Detection

摘要： 基于重构或预测的视频异常检测（VAD）方法面临两个关键挑战：（1）强泛化能力往往导致异常事件的准确重构或预测，这使得区分正常模式和异常模式变得困难；（2）仅依赖低级外观和运动线索限制了它们识别复杂场景中异常事件高级语义的能力。 为了解决这些局限性，我们提出了一种新的VAD框架，具有两项关键创新。 首先，为了抑制过度泛化，我们引入了稀疏特征过滤模块（SFFM），该模块利用瓶颈滤波器动态且自适应地从特征中移除异常信息。 与传统的记忆模块不同，它不需要在整个训练数据集中记住正常的原型。 此外，我们为SFFM设计了专家混合（MoE）架构。 每个专家在运行时负责提取专门的主要特征，并选择性地激活不同的专家以确保学习到的主要特征的多样性。 其次，为了克服现有方法对语义的忽视，我们将视觉-语言模型（VLM）集成进来，为视频片段生成文本描述，从而实现语义、外观和运动线索的全面联合建模。 此外，我们通过语义相似性约束和运动帧差异对比损失来强制模态一致性。 在多个公共数据集上的大量实验验证了我们多模态联合建模框架和稀疏特征过滤范式的有效性。 项目页面见 https://qzfm.github.io/sfn_vad_project_page/. 显示英文摘要

摘要： Video Anomaly Detection (VAD) methods based on reconstruction or prediction face two critical challenges: (1) strong generalization capability often results in accurate reconstruction or prediction of abnormal events, making it difficult to distinguish normal from abnormal patterns; (2) reliance only on low-level appearance and motion cues limits their ability to identify high-level semantic in abnormal events from complex scenes. To address these limitations, we propose a novel VAD framework with two key innovations. First, to suppress excessive generalization, we introduce the Sparse Feature Filtering Module (SFFM) that employs bottleneck filters to dynamically and adaptively remove abnormal information from features. Unlike traditional memory modules, it does not need to memorize the normal prototypes across the training dataset. Further, we design the Mixture of Experts (MoE) architecture for SFFM. Each expert is responsible for extracting specialized principal features during running time, and different experts are selectively activated to ensure the diversity of the learned principal features. Second, to overcome the neglect of semantics in existing methods, we integrate a Vision-Language Model (VLM) to generate textual descriptions for video clips, enabling comprehensive joint modeling of semantic, appearance, and motion cues. Additionally, we enforce modality consistency through semantic similarity constraints and motion frame-difference contrastive loss. Extensive experiments on multiple public datasets validate the effectiveness of our multimodal joint modeling framework and sparse feature filtering paradigm. Project page at https://qzfm.github.io/sfn_vad_project_page/.