标题： MotionSight：提升多模态大型语言模型中的精细动作理解 显示英文标题

标题： MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal LLMs

摘要： 尽管多模态大型语言模型（MLLMs）取得了进展，但它们在细粒度视频运动理解方面的能力仍然严重受限。它们通常缺乏帧间差异，并倾向于平均或忽略微妙的视觉线索。此外，虽然视觉提示在静态图像中显示出潜力，但将其应用于视频的时间复杂性，特别是针对细粒度运动理解的应用，仍未得到充分探索。我们研究了是否可以解锁MLLMs的固有能力，提升其运动感知，并生成针对分离物体和相机运动线索的独特视觉特征。在这项研究中，我们提出了MotionSight，这是一种开创性的零样本方法，以对象为中心的视觉聚光灯和运动模糊作为视觉提示，有效提升了细粒度运动理解，而无需训练。为了将其转化为有价值的数据资产，我们创建了MotionVid-QA，这是首个用于细粒度视频运动理解的大规模数据集，包含分层注释，包括SFT和偏好数据，以及{\Theta }（40K）个视频片段和{\Theta }（87K）个问答。实验表明，MotionSight实现了最先进的开源性能，并与商业模型具有竞争力。特别是，我们提出了一种新的零样本技术以及一个大规模、高质量的数据集。所有代码和注释都将公开可用。 显示英文摘要

摘要： Despite advancements in Multimodal Large Language Models (MLLMs), their proficiency in fine-grained video motion understanding remains critically limited. They often lack inter-frame differencing and tend to average or ignore subtle visual cues. Furthermore, while visual prompting has shown potential in static images, its application to video's temporal complexities, particularly for fine-grained motion understanding, remains largely unexplored. We investigate whether inherent capability can be unlocked and boost MLLMs' motion perception and enable distinct visual signatures tailored to decouple object and camera motion cues. In this study, we introduce MotionSight, a novel zero-shot method pioneering object-centric visual spotlight and motion blur as visual prompts to effectively improve fine-grained motion understanding without training. To convert this into valuable data assets, we curated MotionVid-QA, the first large-scale dataset for fine-grained video motion understanding, with hierarchical annotations including SFT and preference data, {\Theta}(40K) video clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves state-of-the-art open-source performance and competitiveness with commercial models. In particular, for fine-grained motion understanding we present a novel zero-shot technique and a large-scale, high-quality dataset. All the code and annotations will be publicly available.