标题： 普通视觉语言模型能否匹敌医学视觉语言模型？ 评估与战略见解 显示英文标题

标题： Can Common VLMs Rival Medical VLMs? Evaluation and Strategic Insights

摘要： 医学视觉语言模型（VLMs）利用大规模预训练进行多种成像任务，但需要大量的计算和数据资源。 同时，常见的或通用的VLMs（例如CLIP、LLaVA），尽管未针对医学用途进行训练，但通过微调显示出潜力。 这引发了一个关键问题：经过高效微调的通用VLMs能否在解决特定医学成像任务时与通用医学VLMs相媲美？ 本研究系统地评估了常见VLMs和医学VLMs在疾病诊断和视觉问答（VQA）方面的表现。 使用基于CLIP和LLaVA的模型，我们检查了（1）在领域内（ID）设置中的现成性能差距，（2）微调是否能缩小这些差距，以及（3）在未见过的医学模态上的领域外（OOD）任务的泛化能力。 虽然医学专用预训练在ID设置中具有优势，但在轻量级微调后，通用VLMs的表现可以与医学专用模型相当甚至超越，其中基于LoRA的适应方法在不同任务中表现出高度有效性。 在OOD任务中，通用VLMs在某些任务中表现出强大的适应能力，挑战了医学专用预训练是必要的这一假设。 这些发现表明，利用经过微调的通用VLMs为开发大规模医学VLMs提供了一种可扩展且成本效益高的替代方案，为医学成像领域的未来研究提供了重要见解。 显示英文摘要

摘要： Medical vision-language models (VLMs) leverage large-scale pretraining for diverse imaging tasks but require substantial computational and data resources. Meanwhile, common or general-purpose VLMs (e.g., CLIP, LLaVA), though not trained for medical use, show promise with fine-tuning. This raises a key question: Can efficient fine-tuned common VLMs rival generalist medical VLMs for solving specific medical imaging tasks? This study systematically evaluates common and medical VLMs across disease diagnosis and visual question answering (VQA). Using CLIP-based and LLaVA-based models, we examine (1) off-the-shelf performance gaps in in-domain (ID) settings, (2) whether fine-tuning bridges these gaps, and (3) generalization to out-of-domain (OOD) tasks on unseen medical modalities. While medical-specific pretraining provides advantages in ID settings, common VLMs match or surpass medical-specific models after lightweight fine-tuning, with LoRA-based adaptation proving highly effective among different tasks. In OOD tasks, common VLMs demonstrate strong adaptability in some tasks, challenging the assumption that medical-specific pre-training is essential. These findings suggest that leveraging common VLMs with fine-tuning offers a scalable and cost-effective alternative to developing large-scale medical VLMs, providing crucial insights for future research in the medical imaging field.