Title: AD-DINOv3: Enhancing DINOv3 for Zero-Shot Anomaly Detection with Anomaly-Aware Calibration Show Chinese title

Title: AD-DINOv3：通过异常感知校准增强DINOv3的零样本异常检测

Abstract: Zero-Shot Anomaly Detection (ZSAD) seeks to identify anomalies from arbitrary novel categories, offering a scalable and annotation-efficient solution. Traditionally, most ZSAD works have been based on the CLIP model, which performs anomaly detection by calculating the similarity between visual and text embeddings. Recently, vision foundation models such as DINOv3 have demonstrated strong transferable representation capabilities. In this work, we are the first to adapt DINOv3 for ZSAD. However, this adaptation presents two key challenges: (i) the domain bias between large-scale pretraining data and anomaly detection tasks leads to feature misalignment; and (ii) the inherent bias toward global semantics in pretrained representations often leads to subtle anomalies being misinterpreted as part of the normal foreground objects, rather than being distinguished as abnormal regions. To overcome these challenges, we introduce AD-DINOv3, a novel vision-language multimodal framework designed for ZSAD. Specifically, we formulate anomaly detection as a multimodal contrastive learning problem, where DINOv3 is employed as the visual backbone to extract patch tokens and a CLS token, and the CLIP text encoder provides embeddings for both normal and abnormal prompts. To bridge the domain gap, lightweight adapters are introduced in both modalities, enabling their representations to be recalibrated for the anomaly detection task. Beyond this baseline alignment, we further design an Anomaly-Aware Calibration Module (AACM), which explicitly guides the CLS token to attend to anomalous regions rather than generic foreground semantics, thereby enhancing discriminability. Extensive experiments on eight industrial and medical benchmarks demonstrate that AD-DINOv3 consistently matches or surpasses state-of-the-art methods.The code will be available at https://github.com/Kaisor-Yuan/AD-DINOv3. Show Chinese abstract

Abstract: 零样本异常检测（ZSAD）旨在从任意新类别中识别异常，提供一种可扩展且注释高效的解决方案。 传统上，大多数ZSAD工作都基于CLIP模型，该模型通过计算视觉和文本嵌入之间的相似性来进行异常检测。 最近，像DINOv3这样的视觉基础模型展示了强大的可迁移表示能力。 在本工作中，我们首次将DINOv3适应于ZSAD。 然而，这种适应带来了两个关键挑战：（i）大规模预训练数据与异常检测任务之间的领域偏差导致特征不对齐；以及（ii）预训练表示中对全局语义的固有偏差通常会导致细微的异常被误认为是正常前景对象的一部分，而不是被识别为异常区域。 为了克服这些挑战，我们引入了AD-DINOv3，这是一种专为ZSAD设计的新颖视觉-语言多模态框架。 具体而言，我们将异常检测形式化为一个多模态对比学习问题，其中DINOv3作为视觉主干来提取块标记和CLS标记，而CLIP文本编码器为正常和异常提示提供嵌入。 为了弥合领域差距，在两种模态中引入了轻量级适配器，使其表示能够重新校准以适应异常检测任务。 除了这一基线对齐之外，我们进一步设计了一个异常感知校准模块（AACM），该模块明确引导CLS标记关注异常区域而非通用前景语义，从而增强区分能力。 在八个工业和医学基准上的广泛实验表明，AD-DINOv3始终与最先进的方法相当或超越它们。代码将在https://github.com/Kaisor-Yuan/AD-DINOv3上提供。