Title: VocSegMRI: Multimodal Learning for Precise Vocal Tract Segmentation in Real-time MRI Show Chinese title

Title: VocSegMRI：实时MRI中精确声道分割的多模态学习

Abstract: Accurately segmenting articulatory structures in real-time magnetic resonance imaging (rtMRI) remains challenging, as most existing methods rely almost entirely on visual cues. Yet synchronized acoustic and phonological signals provide complementary context that can enrich visual information and improve precision. In this paper, we introduce VocSegMRI, a multimodal framework that integrates video, audio, and phonological inputs through cross-attention fusion for dynamic feature alignment. To further enhance cross-modal representation, we incorporate a contrastive learning objective that improves segmentation performance even when the audio modality is unavailable at inference. Evaluated on a sub-set of USC-75 rtMRI dataset, our approach achieves state-of-the-art performance, with a Dice score of 0.95 and a 95th percentile Hausdorff Distance (HD_95) of 4.20 mm, outperforming both unimodal and multimodal baselines. Ablation studies confirm the contributions of cross-attention and contrastive learning to segmentation precision and robustness. These results highlight the value of integrative multimodal modeling for accurate vocal tract analysis. Show Chinese abstract

Abstract: 准确地在实时磁共振成像(rtMRI)中分割发音结构仍然具有挑战性，因为大多数现有方法几乎完全依赖视觉线索。 然而同步的声学和语音信号提供了补充上下文，可以丰富视觉信息并提高精度。 在本文中，我们引入了VocSegMRI，这是一种多模态框架，通过交叉注意力融合整合视频、音频和语音输入以实现动态特征对齐。 为了进一步增强跨模态表示，我们引入了一个对比学习目标，在推理时音频模态不可用的情况下也能提高分割性能。 在USC-75 rtMRI数据集的一个子集上进行评估，我们的方法实现了最先进的性能，Dice分数为0.95，95百分位数Hausdorff距离(HD_95)为4.20毫米，优于单模态和多模态基线。 消融研究证实了交叉注意力和对比学习对分割精度和鲁棒性的贡献。 这些结果突显了集成多模态建模在准确声腔分析中的价值。