标题： 思考后再分割：一种针对指代音频-视觉分割的物体感知推理代理 显示英文标题

标题： Think Before You Segment: An Object-aware Reasoning Agent for Referring Audio-Visual Segmentation

摘要： 参考音频-视觉分割（Ref-AVS）旨在根据给定的参考表达式对可听视频中的目标对象进行分割。 以往的工作通常依赖于通过多模态融合学习潜在嵌入，以提示可调节的SAM/SAM2解码器进行分割，这需要强大的像素级监督且缺乏可解释性。 从显式参考理解的新视角出发，我们提出了TGS-Agent，它将任务分解为思考-定位-分割的过程，通过首先通过多模态分析识别所指对象，然后进行粗粒度定位和精确分割，模仿人类的推理过程。 为此，我们首先提出了Ref-Thinker，一个能够对文本、视觉和听觉线索进行推理的多模态语言模型。 我们构建了一个带有显式对象感知思考-回答链的指令微调数据集，用于Ref-Thinker的微调。 由Ref-Thinker推断出的对象描述被用作Grounding-DINO和SAM2的显式提示，这些模型在不依赖像素级监督的情况下执行定位和分割。 此外，我们引入了R\textsuperscript{2}-AVSBench，一个新的基准测试，具有语言多样性和推理密集型的参考，以更好地评估模型的泛化能力。 我们的方法在标准Ref-AVSBench和提出的R\textsuperscript{2}-AVSBench上均取得了最先进的结果。 代码将在https://github.com/jasongief/TGS-Agent上提供。 显示英文摘要

摘要： Referring Audio-Visual Segmentation (Ref-AVS) aims to segment target objects in audible videos based on given reference expressions. Prior works typically rely on learning latent embeddings via multimodal fusion to prompt a tunable SAM/SAM2 decoder for segmentation, which requires strong pixel-level supervision and lacks interpretability. From a novel perspective of explicit reference understanding, we propose TGS-Agent, which decomposes the task into a Think-Ground-Segment process, mimicking the human reasoning procedure by first identifying the referred object through multimodal analysis, followed by coarse-grained grounding and precise segmentation. To this end, we first propose Ref-Thinker, a multimodal language model capable of reasoning over textual, visual, and auditory cues. We construct an instruction-tuning dataset with explicit object-aware think-answer chains for Ref-Thinker fine-tuning. The object description inferred by Ref-Thinker is used as an explicit prompt for Grounding-DINO and SAM2, which perform grounding and segmentation without relying on pixel-level supervision. Additionally, we introduce R\textsuperscript{2}-AVSBench, a new benchmark with linguistically diverse and reasoning-intensive references for better evaluating model generalization. Our approach achieves state-of-the-art results on both standard Ref-AVSBench and proposed R\textsuperscript{2}-AVSBench. Code will be available at https://github.com/jasongief/TGS-Agent.