Title: Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification Show Chinese title

Title: 基于MLLMs的人类中心交互学习用于文本到图像的人再识别

Abstract: Despite remarkable advancements in text-to-image person re-identification (TIReID) facilitated by the breakthrough of cross-modal embedding models, existing methods often struggle to distinguish challenging candidate images due to intrinsic limitations, such as network architecture and data quality. To address these issues, we propose an Interactive Cross-modal Learning framework (ICL), which leverages human-centered interaction to enhance the discriminability of text queries through external multimodal knowledge. To achieve this, we propose a plug-and-play Test-time Humane-centered Interaction (THI) module, which performs visual question answering focused on human characteristics, facilitating multi-round interactions with a multimodal large language model (MLLM) to align query intent with latent target images. Specifically, THI refines user queries based on the MLLM responses to reduce the gap to the best-matching images, thereby boosting ranking accuracy. Additionally, to address the limitation of low-quality training texts, we introduce a novel Reorganization Data Augmentation (RDA) strategy based on information enrichment and diversity enhancement to enhance query discriminability by enriching, decomposing, and reorganizing person descriptions. Extensive experiments on four TIReID benchmarks, i.e., CUHK-PEDES, ICFG-PEDES, RSTPReid, and UFine6926, demonstrate that our method achieves remarkable performance with substantial improvement. Show Chinese abstract

Abstract: 尽管得益于跨模态嵌入模型的突破，文本到图像人物再识别（TIReID）取得了显著进展，但现有方法往往由于网络架构和数据质量等固有限制，难以区分具有挑战性的候选图像。为了解决这些问题，我们提出了一种交互式跨模态学习框架（ICL），利用以人为中心的交互来通过外部多模态知识增强文本查询的可分辨性。 为了实现这一点，我们提出了一个即插即用的测试时以人为中心的交互（THI）模块，该模块专注于人类特征的视觉问答，促进与多模态大型语言模型（MLLM）的多轮交互，以使查询意图与潜在的目标图像对齐。具体而言，THI 根据MLLM的响应优化用户查询，减少与最佳匹配图像之间的差距，从而提高排名准确性。 此外，为了解决训练文本质量低的问题，我们引入了一种基于信息丰富化和多样性增强的新颖数据重组织增强（RDA）策略，通过丰富、分解和重新组织人物描述来增强查询的可分辨性。 在四个TIReID基准数据集（即CUHK-PEDES、ICFG-PEDES、RSTPReid和UFine6926）上的大量实验表明，我们的方法取得了显著的性能提升。