标题： 视觉-语言模型用于自动胸部X光片解读：利用ViT和GPT-2 显示英文标题

标题： Vision-Language Models for Automated Chest X-ray Interpretation: Leveraging ViT and GPT-2

摘要： 放射学在现代医学中起着关键作用，这是由于其非侵入性的诊断能力。 然而，手动生成非结构化医学报告既耗时又容易出错。 这在临床工作流程中形成了一个显著的瓶颈。 尽管人工智能生成的放射学报告取得了进展，但在实现详细和准确的报告生成方面仍存在挑战。 在本研究中，我们评估了不同组合的多模态模型，这些模型结合了计算机视觉和自然语言处理，以生成全面的放射学报告。 我们使用了预训练的视觉变换器（ViT-B16）和SWIN变换器作为图像编码器。 BART和GPT-2模型作为文本解码器。 我们使用IU-Xray数据集中的胸部X光图像和报告来评估SWIN变换器-BART、SWIN变换器-GPT-2、ViT-B16-BART和ViT-B16-GPT-2模型在报告生成中的可用性。 我们的目标是找到这些模型中的最佳组合。 SWIN-BART模型在四个模型中表现最佳，在ROUGE、BLEU和BERTScore等几乎所有评估指标中都取得了显著的结果。 显示英文摘要

摘要： Radiology plays a pivotal role in modern medicine due to its non-invasive diagnostic capabilities. However, the manual generation of unstructured medical reports is time consuming and prone to errors. It creates a significant bottleneck in clinical workflows. Despite advancements in AI-generated radiology reports, challenges remain in achieving detailed and accurate report generation. In this study we have evaluated different combinations of multimodal models that integrate Computer Vision and Natural Language Processing to generate comprehensive radiology reports. We employed a pretrained Vision Transformer (ViT-B16) and a SWIN Transformer as the image encoders. The BART and GPT-2 models serve as the textual decoders. We used Chest X-ray images and reports from the IU-Xray dataset to evaluate the usability of the SWIN Transformer-BART, SWIN Transformer-GPT-2, ViT-B16-BART and ViT-B16-GPT-2 models for report generation. We aimed at finding the best combination among the models. The SWIN-BART model performs as the best-performing model among the four models achieving remarkable results in almost all the evaluation metrics like ROUGE, BLEU and BERTScore.