标题： 基于语义引导的多模态情感解码与对抗性时序不变学习 显示英文标题

标题： Semantic-Guided Multimodal Sentiment Decoding with Adversarial Temporal-Invariant Learning

摘要： 多模态情感分析旨在从不同模态中学习表示以识别人类情感。然而，现有工作通常忽略了连续时间序列中存在的帧级冗余，导致模态表示不完整且存在噪声。为了解决这一问题，我们首次提出了时序不变学习，该方法通过约束时间步长上的分布变化来有效捕获长期时序动态，从而提高表示质量和模型的鲁棒性。为了充分利用文本知识中的丰富语义信息，我们提出了一种语义引导的融合模块。通过评估不同模态之间的相关性，该模块促进了由模态不变表示控制的跨模态交互。此外，我们引入了一个模态判别器以分离模态不变子空间和模态特定子空间。在两个公开数据集上的实验结果证明了我们模型的优越性。我们的代码可在 https://github.com/X-G-Y/SATI 获取。 显示英文摘要

摘要： Multimodal sentiment analysis aims to learn representations from different modalities to identify human emotions. However, existing works often neglect the frame-level redundancy inherent in continuous time series, resulting in incomplete modality representations with noise. To address this issue, we propose temporal-invariant learning for the first time, which constrains the distributional variations over time steps to effectively capture long-term temporal dynamics, thus enhancing the quality of the representations and the robustness of the model. To fully exploit the rich semantic information in textual knowledge, we propose a semantic-guided fusion module. By evaluating the correlations between different modalities, this module facilitates cross-modal interactions gated by modality-invariant representations. Furthermore, we introduce a modality discriminator to disentangle modality-invariant and modality-specific subspaces. Experimental results on two public datasets demonstrate the superiority of our model. Our code is available at https://github.com/X-G-Y/SATI.