标题： LLaVA-Octopus：解锁视频理解的指令驱动自适应投影融合 显示英文标题

标题： LLaVA-Octopus: Unlocking Instruction-Driven Adaptive Projector Fusion for Video Understanding

摘要： 在本文中，我们介绍了LLaVA-Octopus，一种新颖的视频多模态大语言模型。 LLaVA-Octopus根据用户指令自适应地加权来自不同视觉投影器的特征，使我们能够利用每个投影器的互补优势。 我们观察到，不同的视觉投影器在处理特定任务时表现出不同的特性。 例如，一些投影器在捕捉静态细节方面表现优异，而另一些则在处理时间信息方面更有效，还有一些更适合需要时间一致性的任务。 通过根据用户指令动态调整特征权重，LLaVA-Octopus动态选择并组合最合适的特征，显著提升了模型在多模态任务中的性能。 实验结果表明， LLaVA-Octopus在多个基准测试中表现出色，特别是在视频问答、长视频理解以及综合多项选择基准任务中，突显了其广泛的应用潜力。 显示英文摘要

摘要： In this paper, we introduce LLaVA-Octopus, a novel video multimodal large language model. LLaVA-Octopus adaptively weights features from different visual projectors based on user instructions, enabling us to leverage the complementary strengths of each projector. We observe that different visual projectors exhibit distinct characteristics when handling specific tasks. For instance, some projectors excel at capturing static details, while others are more effective at processing temporal information, and some are better suited for tasks requiring temporal coherence. By dynamically adjusting feature weights according to user instructions, LLaVA-Octopus dynamically selects and combines the most suitable features, significantly enhancing the model's performance in multimodal tasks. Experimental results demonstrate that LLaVA-Octopus achieves excellent performance across multiple benchmarks, especially in tasks such as video question answering, long video understanding, and comprehensive multi-choices benchmarks, highlighting its broad application potential.