标题： VideoCap-R1：通过结构化思维增强用于视频字幕的MLLMs 显示英文标题

标题： VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking

摘要： 尽管强化学习的最新进展显著提升了大型语言模型（LLMs）的推理能力，但在视频字幕的多模态LLMs中，这些技术仍未得到充分探索。 本文首次系统性地研究了基于GRPO的RL微调对视频多模态LLMs的影响，旨在增强视频多模态LLMs描述视频中动作的能力。 具体而言，我们开发了VideoCap-R1，它被提示首先执行结构化思考，分析视频主体及其属性和动作，然后生成完整的字幕，支持两种专门的奖励机制：一种是无需LLM的思考评分器评估结构化思考的质量，另一种是辅助LLM的字幕评分器评估输出质量。 RL训练框架有效地建立了结构化推理与全面描述生成之间的联系，使模型能够生成更准确的动作描述。 我们的实验表明，VideoCap-R1在多个视频字幕基准测试（DREAM1K:+4.4事件F1，VDC:+4.2准确率，CAREBENCH:+3.1动作F1，+6.9对象F1）上显著优于Qwen2VL-7B基线模型（使用有限样本1.5k），并且始终优于SFT训练的对应模型，证实了GRPO在提升多模态LLMs字幕能力方面的优越性。 显示英文摘要

摘要： While recent advances in reinforcement learning have significantly enhanced reasoning capabilities in large language models (LLMs), these techniques remain underexplored in multi-modal LLMs for video captioning. This paper presents the first systematic investigation of GRPO-based RL post-training for video MLLMs, with the goal of enhancing video MLLMs' capability of describing actions in videos. Specifically, we develop the VideoCap-R1, which is prompted to first perform structured thinking that analyzes video subjects with their attributes and actions before generating complete captions, supported by two specialized reward mechanisms: a LLM-free think scorer evaluating the structured thinking quality and a LLM-assisted caption scorer assessing the output quality. The RL training framework effectively establishes the connection between structured reasoning and comprehensive description generation, enabling the model to produce captions with more accurate actions. Our experiments demonstrate that VideoCap-R1 achieves substantial improvements over the Qwen2VL-7B baseline using limited samples (1.5k) across multiple video caption benchmarks (DREAM1K: +4.4 event F1, VDC: +4.2 Acc, CAREBENCH: +3.1 action F1, +6.9 object F1) while consistently outperforming the SFT-trained counterparts, confirming GRPO's superiority in enhancing MLLMs' captioning capabilities.