标题： 关于误导性报告的风险：诊断多模态临床AI中的文本偏差 显示英文标题

标题： On the Risk of Misleading Reports: Diagnosing Textual Biases in Multimodal Clinical AI

摘要： 临床决策依赖于医学图像和相关临床报告的综合分析。 尽管视觉-语言模型（VLMs）可以为此类任务提供一个统一的框架，但它们可能对某一模态表现出强烈的偏见，经常忽视关键的视觉线索而偏向文本信息。 在本工作中，我们引入了选择性模态转移（SMS），这是一种基于扰动的方法，用于量化模型在二分类任务中对每个模态的依赖程度。 通过系统地在具有相反标签的样本之间交换图像或文本，我们揭示了模态特定的偏见。 我们在两个具有不同模态的医学影像数据集上评估了六种开源VLMs——四种通用模型和两种针对医学数据微调的模型：MIMIC-CXR（胸部X光）和FairVLMed（扫描激光眼底照相术）。 通过评估模型在未扰动和扰动设置下的性能以及每个模型的校准情况，我们发现对文本输入有显著依赖，即使存在互补的视觉信息也是如此。 我们还进行了一项基于注意力的定性分析，进一步确认了图像内容常常被文本细节所掩盖。 我们的研究结果突显了设计和评估真正整合视觉和文本线索的多模态医学模型的重要性，而不是依赖单一模态信号。 显示英文摘要

摘要： Clinical decision-making relies on the integrated analysis of medical images and the associated clinical reports. While Vision-Language Models (VLMs) can offer a unified framework for such tasks, they can exhibit strong biases toward one modality, frequently overlooking critical visual cues in favor of textual information. In this work, we introduce Selective Modality Shifting (SMS), a perturbation-based approach to quantify a model's reliance on each modality in binary classification tasks. By systematically swapping images or text between samples with opposing labels, we expose modality-specific biases. We assess six open-source VLMs-four generalist models and two fine-tuned for medical data-on two medical imaging datasets with distinct modalities: MIMIC-CXR (chest X-ray) and FairVLMed (scanning laser ophthalmoscopy). By assessing model performance and the calibration of every model in both unperturbed and perturbed settings, we reveal a marked dependency on text input, which persists despite the presence of complementary visual information. We also perform a qualitative attention-based analysis which further confirms that image content is often overshadowed by text details. Our findings highlight the importance of designing and evaluating multimodal medical models that genuinely integrate visual and textual cues, rather than relying on single-modality signals.