标题： MoDA：用于细粒度视觉接地的指令调优大模型的调制适配器 显示英文标题

标题： MoDA: Modulation Adapter for Fine-Grained Visual Grounding in Instructional MLLMs

摘要： 近期，多模态大型语言模型（MLLMs）通过集成预训练的视觉编码器与大型语言模型（LLMs），在指令跟随任务上展示了令人印象深刻的性能。然而，现有方法通常难以在复杂场景中定位细粒度的视觉概念。 本文提出了一种轻量且有效的模块 MoDA（调制适配器），旨在通过指令引导的调制来优化预对齐的视觉特征。我们的方法遵循标准的 LLaVA 训练协议，包括两个阶段：(1) 使用冻结的视觉编码器和适配层将图像特征对齐到 LLMs 输入空间；(2) 在指令微调阶段使用 MoDA 适配器优化这些特征。MoDA 使用基于 Transformer 的交叉注意力机制，在对齐的视觉标记上生成调制掩码，从而根据语言指令强调语义相关的嵌入维度。调制后的特征随后传递给 LLM 进行自回归语言生成。 实验评估表明，MoDA 提升了视觉定位能力，并生成了更符合上下文的响应，证明了它作为基于图像的 MLLMs 通用增强模块的有效性。 显示英文摘要

摘要： Recently, Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on instruction-following tasks by integrating pretrained visual encoders with large language models (LLMs). However, existing approaches often struggle to ground fine-grained visual concepts in complex scenes. In this paper, we propose MoDA (Modulation Adapter), a lightweight yet effective module designed to refine pre-aligned visual features through instruction-guided modulation. Our approach follows the standard LLaVA training protocol, consisting of a two-stage process: (1) aligning image features to the LLMs input space via a frozen vision encoder and adapter layers, and (2) refining those features using the MoDA adapter during the instructional tuning stage. MoDA employs a Transformer-based cross-attention mechanism to generate a modulation mask over the aligned visual tokens, thereby emphasizing semantically relevant embedding dimensions based on the language instruction. The modulated features are then passed to the LLM for autoregressive language generation. Our experimental evaluation shows that MoDA improves visual grounding and generates more contextually appropriate responses, demonstrating its effectiveness as a general-purpose enhancement for image-based MLLMs.