标题： 视觉变压器表示是否具有语义意义？ 医学影像中的案例研究 显示英文标题

标题： Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging

摘要： 视觉变压器（ViTs）由于其相比传统深度学习模型的优越准确性，在医学影像任务如疾病分类、分割和检测中迅速获得了关注。 然而，由于其规模和通过自注意机制的复杂交互，它们并不容易理解。 特别是，尚不清楚此类模型产生的表示是否具有语义意义。 在本文中，我们使用基于投影梯度的算法表明，它们的表示并不具有语义意义，并且对小的变化本质上是脆弱的。 具有难以察觉差异的图像可能有非常不同的表示；另一方面，应该属于不同语义类的图像可能有几乎相同的表示。 这种脆弱性可能导致不可靠的分类结果；例如，不易察觉的变化会导致分类准确率下降超过60%。 据我们所知，这是首次系统地展示ViT表示在医学图像分类中的基本语义无意义性，揭示了其在安全关键系统中部署时的一个关键挑战。 显示英文摘要

摘要： Vision transformers (ViTs) have rapidly gained prominence in medical imaging tasks such as disease classification, segmentation, and detection due to their superior accuracy compared to conventional deep learning models. However, due to their size and complex interactions via the self-attention mechanism, they are not well understood. In particular, it is unclear whether the representations produced by such models are semantically meaningful. In this paper, using a projected gradient-based algorithm, we show that their representations are not semantically meaningful and they are inherently vulnerable to small changes. Images with imperceptible differences can have very different representations; on the other hand, images that should belong to different semantic classes can have nearly identical representations. Such vulnerability can lead to unreliable classification results; for example, unnoticeable changes cause the classification accuracy to be reduced by over 60\%. %. To the best of our knowledge, this is the first work to systematically demonstrate this fundamental lack of semantic meaningfulness in ViT representations for medical image classification, revealing a critical challenge for their deployment in safety-critical systems.