标题： 跨眼底变压器用于白内障的多模态糖尿病视网膜病变分级 显示英文标题

标题： Cross-Fundus Transformer for Multi-modal Diabetic Retinopathy Grading with Cataract

摘要： 糖尿病视网膜病变（DR）是全球失明的主要原因之一，也是糖尿病的常见并发症。 作为两种不同的DR分级成像工具，彩色眼底摄影（CFP）和红外眼底摄影（IFP）在临床应用中高度相关且互补。 据我们所知，这是首次研究探索一种新的多模态深度学习框架，以融合CFP和IFP的信息，从而实现更准确的DR分级。 具体来说，我们构建了一个双流架构——跨眼底Transformer（CFT），用于融合基于ViT的两种眼底图像模态特征。 特别是，引入了一个精心设计的跨眼底注意（CFA）模块，以捕捉CFP和IFP图像之间的对应关系。 此外，我们采用单模态和多模态监督，以最大化DR分级的整体性能。 在由1,713对多模态眼底图像组成的临床数据集上的广泛实验表明了我们提出方法的优势。 我们的代码将公开发布。 显示英文摘要

摘要： Diabetic retinopathy (DR) is a leading cause of blindness worldwide and a common complication of diabetes. As two different imaging tools for DR grading, color fundus photography (CFP) and infrared fundus photography (IFP) are highly-correlated and complementary in clinical applications. To the best of our knowledge, this is the first study that explores a novel multi-modal deep learning framework to fuse the information from CFP and IFP towards more accurate DR grading. Specifically, we construct a dual-stream architecture Cross-Fundus Transformer (CFT) to fuse the ViT-based features of two fundus image modalities. In particular, a meticulously engineered Cross-Fundus Attention (CFA) module is introduced to capture the correspondence between CFP and IFP images. Moreover, we adopt both the single-modality and multi-modality supervisions to maximize the overall performance for DR grading. Extensive experiments on a clinical dataset consisting of 1,713 pairs of multi-modal fundus images demonstrate the superiority of our proposed method. Our code will be released for public access.