标题： MMFformer：用于抑郁检测的多模态融合Transformer网络 显示英文标题

标题： MMFformer: Multimodal Fusion Transformer Network for Depression Detection

摘要： 抑郁症是一种严重影响个体幸福感和生活质量的严重心理健康疾病，因此早期检测对于适当的护理和治疗至关重要。 检测抑郁症通常很困难，因为主要基于临床访谈中的主观评估。 因此，借助社交媒体内容的抑郁症早期诊断已成为一个突出的研究领域。 用户生成信息的广泛性和多样性带来了重大挑战，限制了相关时间信息的准确提取以及多模态数据的有效融合。 本文介绍了MMFformer，这是一种多模态抑郁症检测网络，旨在从多模态社交媒体信息中检索抑郁的时空高层次模式。 具有残差连接的Transformer网络从视频中捕捉空间特征，而Transformer编码器被用于设计音频中的重要时间动态。 此外，融合架构通过晚期和中期融合策略融合提取的特征，以找出它们之间的最相关跨模态相关性。 最后，所提出的网络在两个大规模抑郁症检测数据集上进行了评估，结果明确表明它优于现有的最先进方法，在D-Vlog数据集上的F1分数提高了13.92%，在LMVD数据集上提高了7.74%。 代码可在https://github.com/rezwanh001/Large-Scale-Multimodal-Depression-Detection公开获取。 显示英文摘要

摘要： Depression is a serious mental health illness that significantly affects an individual's well-being and quality of life, making early detection crucial for adequate care and treatment. Detecting depression is often difficult, as it is based primarily on subjective evaluations during clinical interviews. Hence, the early diagnosis of depression, thanks to the content of social networks, has become a prominent research area. The extensive and diverse nature of user-generated information poses a significant challenge, limiting the accurate extraction of relevant temporal information and the effective fusion of data across multiple modalities. This paper introduces MMFformer, a multimodal depression detection network designed to retrieve depressive spatio-temporal high-level patterns from multimodal social media information. The transformer network with residual connections captures spatial features from videos, and a transformer encoder is exploited to design important temporal dynamics in audio. Moreover, the fusion architecture fused the extracted features through late and intermediate fusion strategies to find out the most relevant intermodal correlations among them. Finally, the proposed network is assessed on two large-scale depression detection datasets, and the results clearly reveal that it surpasses existing state-of-the-art approaches, improving the F1-Score by 13.92% for D-Vlog dataset and 7.74% for LMVD dataset. The code is made available publicly at https://github.com/rezwanh001/Large-Scale-Multimodal-Depression-Detection.