标题： Brain-Adapter：利用适配器微调多模态大型语言模型增强神经系统疾病分析 显示英文标题

标题： Brain-Adapter: Enhancing Neurological Disorder Analysis with Adapter-Tuning Multimodal Large Language Models

摘要： 理解脑部疾病对于准确的临床诊断和治疗至关重要。多模态大型语言模型（MLLMs）的最新进展提供了一种利用文本描述解释医学图像的有前景的方法。然而，以往的研究主要集中在二维医学图像上，而三维图像中更丰富的空间信息尚未被充分探索，单模态方法由于忽略了其他模态中包含的关键临床信息而受到限制。为了解决这一问题，本文提出了一种名为Brain-Adapter的新方法，该方法通过增加一个额外的瓶颈层来学习新知识并将其实现到原始预训练的知识中。主要思想是引入一个轻量级的瓶颈层，在捕获关键信息的同时训练较少的参数，并利用对比语言-图像预训练（CLIP）策略在统一表示空间内对齐多模态数据。广泛的实验验证了我们方法在整合多模态数据以显著提高诊断准确性方面的有效性，同时没有带来高计算成本，突显了增强现实世界诊断工作流程的潜力。 显示英文摘要

摘要： Understanding brain disorders is crucial for accurate clinical diagnosis and treatment. Recent advances in Multimodal Large Language Models (MLLMs) offer a promising approach to interpreting medical images with the support of text descriptions. However, previous research has primarily focused on 2D medical images, leaving richer spatial information of 3D images under-explored, and single-modality-based methods are limited by overlooking the critical clinical information contained in other modalities. To address this issue, this paper proposes Brain-Adapter, a novel approach that incorporates an extra bottleneck layer to learn new knowledge and instill it into the original pre-trained knowledge. The major idea is to incorporate a lightweight bottleneck layer to train fewer parameters while capturing essential information and utilize a Contrastive Language-Image Pre-training (CLIP) strategy to align multimodal data within a unified representation space. Extensive experiments demonstrated the effectiveness of our approach in integrating multimodal data to significantly improve the diagnosis accuracy without high computational costs, highlighting the potential to enhance real-world diagnostic workflows.