标题： 平衡多模态学习：单向动态交互视角 显示英文标题

标题： Balanced Multimodal Learning: An Unidirectional Dynamic Interaction Perspective

摘要： 多模态学习通常利用多模态联合损失来整合不同模态并提升模型性能。 然而，这种联合学习策略可能导致模态不平衡，其中强模态会压倒较弱的模态，并限制从每个模态中单独信息以及模态间交互信息的利用。 现有的策略如动态损失加权、辅助目标和梯度调制基于联合损失来缓解模态不平衡。 这些方法本质上是被动的，在不平衡发生后检测并纠正，而未改变联合损失的竞争本质。 这一局限性促使我们探索一种不依赖于联合损失的新多模态不平衡学习策略，从而实现模态之间更有效的交互以及对单个模态及其交互信息的更好利用。 在本文中，我们引入了 单向动态交互（UDI），一种新的策略，该策略摒弃了传统的联合损失，转而采用主动的顺序训练方案。 UDI首先将锚定模态训练到收敛，然后通过无监督损失使用其学习到的表示来引导其他模态。 此外，模态交互的动态调整使模型能够适应当前任务，确保每个模态都能最优地贡献。 通过解耦模态优化并实现定向的信息流，UDI防止任何单一模态的主导，并促进有效的跨模态特征学习。 我们的实验结果表明，UDI在处理模态不平衡方面优于现有方法，从而在多模态学习任务中实现了性能提升。 显示英文摘要

摘要： Multimodal learning typically utilizes multimodal joint loss to integrate different modalities and enhance model performance. However, this joint learning strategy can induce modality imbalance, where strong modalities overwhelm weaker ones and limit exploitation of individual information from each modality and the inter-modality interaction information. Existing strategies such as dynamic loss weighting, auxiliary objectives and gradient modulation mitigate modality imbalance based on joint loss. These methods remain fundamentally reactive, detecting and correcting imbalance after it arises, while leaving the competitive nature of the joint loss untouched. This limitation drives us to explore a new strategy for multimodal imbalance learning that does not rely on the joint loss, enabling more effective interactions between modalities and better utilization of information from individual modalities and their interactions. In this paper, we introduce Unidirectional Dynamic Interaction (UDI), a novel strategy that abandons the conventional joint loss in favor of a proactive, sequential training scheme. UDI first trains the anchor modality to convergence, then uses its learned representations to guide the other modality via unsupervised loss. Furthermore, the dynamic adjustment of modality interactions allows the model to adapt to the task at hand, ensuring that each modality contributes optimally. By decoupling modality optimization and enabling directed information flow, UDI prevents domination by any single modality and fosters effective cross-modal feature learning. Our experimental results demonstrate that UDI outperforms existing methods in handling modality imbalance, leading to performance improvement in multimodal learning tasks.