Title: Dynamic Sub-region Search in Homogeneous Collections Using CLIP Show Chinese title

Title: 使用CLIP在同质集合中进行动态子区域搜索

Abstract: Querying with text-image-based search engines in highly homogeneous domain-specific image collections is challenging for users, as they often struggle to provide descriptive text queries. For example, in an underwater domain, users can usually characterize entities only with abstract labels, such as corals and fish, which leads to low recall rates. Our work investigates whether recall can be improved by supplementing text queries with position information. Specifically, we explore dynamic image partitioning approaches that divide candidates into semantically meaningful regions of interest. Instead of querying entire images, users can specify regions they recognize. This enables the use of position constraints while preserving the semantic capabilities of multimodal models. We introduce and evaluate strategies for integrating position constraints into semantic search models and compare them against static partitioning approaches. Our evaluation highlights both the potential and the limitations of sub-region-based search methods using dynamic partitioning. Dynamic search models achieve up to double the retrieval performance compared to static partitioning approaches but are highly sensitive to perturbations in the specified query positions. Show Chinese abstract

Abstract: 在高度同质的领域特定图像集合中使用基于文本-图像的搜索引擎对用户来说具有挑战性，因为他们常常难以提供描述性的文本查询。 例如，在水下领域，用户通常只能用抽象标签（如珊瑚和鱼类）来描述实体，这导致召回率较低。 我们的工作研究了通过补充位置信息是否可以提高召回率。 具体而言，我们探索了动态图像分区方法，这些方法将候选项划分为语义上有意义的兴趣区域。 用户无需查询整个图像，而是可以指定他们识别出的区域。 这使得在保留多模态模型语义能力的同时能够使用位置约束。 我们介绍了将位置约束集成到语义搜索模型中的策略，并将其与静态分区方法进行了比较。 我们的评估突显了基于子区域搜索方法（采用动态分区）的潜力和局限性。 与静态分区方法相比，动态搜索模型的检索性能最高可提升一倍，但对指定查询位置的扰动非常敏感。