Title: Foreground-Covering Prototype Generation and Matching for SAM-Aided Few-Shot Segmentation Show Chinese title

Title: 前景覆盖原型生成与匹配用于SAM辅助的少样本分割

Abstract: We propose Foreground-Covering Prototype Generation and Matching to resolve Few-Shot Segmentation (FSS), which aims to segment target regions in unlabeled query images based on labeled support images. Unlike previous research, which typically estimates target regions in the query using support prototypes and query pixels, we utilize the relationship between support and query prototypes. To achieve this, we utilize two complementary features: SAM Image Encoder features for pixel aggregation and ResNet features for class consistency. Specifically, we construct support and query prototypes with SAM features and distinguish query prototypes of target regions based on ResNet features. For the query prototype construction, we begin by roughly guiding foreground regions within SAM features using the conventional pseudo-mask, then employ iterative cross-attention to aggregate foreground features into learnable tokens. Here, we discover that the cross-attention weights can effectively alternate the conventional pseudo-mask. Therefore, we use the attention-based pseudo-mask to guide ResNet features to focus on the foreground, then infuse the guided ResNet feature into the learnable tokens to generate class-consistent query prototypes. The generation of the support prototype is conducted symmetrically to that of the query one, with the pseudo-mask replaced by the ground-truth mask. Finally, we compare these query prototypes with support ones to generate prompts, which subsequently produce object masks through the SAM Mask Decoder. Our state-of-the-art performances on various datasets validate the effectiveness of the proposed method for FSS. Our official code is available at https://github.com/SuhoPark0706/FCP Show Chinese abstract

Abstract: 我们提出前景覆盖原型生成与匹配来解决少样本分割（FSS），其目标是基于标记的支持图像对未标记查询图像中的目标区域进行分割。 不同于以往的研究，通常使用支持原型和查询像素来估计查询中的目标区域，我们利用支持和查询原型之间的关系。 为了实现这一点，我们利用两种互补特征：SAM图像编码器特征用于像素聚合，ResNet特征用于类别一致性。 具体来说，我们使用SAM特征构建支持和查询原型，并基于ResNet特征区分目标区域的查询原型。 对于查询原型构建，我们首先使用传统的伪掩码大致引导SAM特征中的前景区域，然后使用迭代交叉注意力将前景特征聚合到可学习的标记中。 在这里，我们发现交叉注意力权重可以有效地交替传统的伪掩码。 因此，我们使用基于注意力的伪掩码引导ResNet特征关注前景，然后将引导的ResNet特征注入可学习的标记中以生成类别一致的查询原型。 支持原型的生成与查询原型的生成对称进行，其中伪掩码被真实掩码取代。 最后，我们将这些查询原型与支持原型进行比较以生成提示，随后通过SAM掩码解码器生成对象掩码。 我们在各种数据集上的最先进性能验证了所提方法在FSS中的有效性。 我们的官方代码可在 https://github.com/SuhoPark0706/FCP 获取。