标题： MedTVT-R1：一种增强医学推理和诊断的多模态大语言模型 显示英文标题

标题： MedTVT-R1: A Multimodal LLM Empowering Medical Reasoning and Diagnosis

摘要： 准确且可解释的多疾病诊断仍然是医学研究中的一个关键挑战，尤其是在利用异构多模态医学数据时。 当前的方法通常依赖于单模态数据，限制了其全面理解复杂疾病的能力。 为了解决这个问题，我们提出了MedTVT-R1，一种新颖的多模态大语言模型（MLLM）框架，旨在整合临床多模态数据进行推理和诊断多种疾病。 我们构建了MedTVT-QA，一个经过筛选的指令数据集，通过证据链方法提供生理水平的解释和疾病水平的诊断的问题-答案对。 MedTVT-R1包含一个模态感知层，以捕捉模态间的依赖关系并自适应地加权模态贡献。 此外，我们采用基于组相对策略优化（GRPO）的强化学习微调，并使用Jaccard奖励函数来增强诊断推理。 实验结果表明，MedTVT-R1在多模态特征利用和多疾病诊断方面表现出优越性，为临床应用（如诊断报告生成和共病推理）提供了重要潜力。 数据集和代码可在https://github.com/keke-nice/MedTVT-R1获取。 显示英文摘要

摘要： Accurate and interpretable multi-disease diagnosis remains a critical challenge in medical research, particularly when leveraging heterogeneous multimodal medical data. Current approaches often rely on single-modal data, limiting their ability to comprehensively understand complex diseases. To address this, we propose MedTVT-R1, a novel Multimodal Large Language Model (MLLM) framework designed to integrate clinical multimodal data for reasoning and diagnosing multiple diseases. We construct MedTVT-QA, a curated instruction dataset that provides question-answer pairs for physiological-level interpretations and disease-level diagnoses with a Chain of Evidence approach. MedTVT-R1 incorporates a modality perception layer to capture inter-modal dependencies and adaptively weight modality contributions. Additionally, we employ Group Relative Policy Optimization (GRPO)-based Reinforcement Fine-Tuning with a Jaccard Reward function to enhance diagnostic reasoning. Experimental results demonstrate MedTVT-R1's superiority in multimodal feature utilization and multi-disease diagnosis, offering significant potential for clinical applications such as diagnostic report generation and comorbidity reasoning. The dataset and code are available at https://github.com/keke-nice/MedTVT-R1.