Title: Amodal Instance Segmentation with Diffusion Shape Prior Estimation Show Chinese title

Title: 基于扩散形状先验估计的非模实例分割

Abstract: Amodal Instance Segmentation (AIS) presents an intriguing challenge, including the segmentation prediction of both visible and occluded parts of objects within images. Previous methods have often relied on shape prior information gleaned from training data to enhance amodal segmentation. However, these approaches are susceptible to overfitting and disregard object category details. Recent advancements highlight the potential of conditioned diffusion models, pretrained on extensive datasets, to generate images from latent space. Drawing inspiration from this, we propose AISDiff with a Diffusion Shape Prior Estimation (DiffSP) module. AISDiff begins with the prediction of the visible segmentation mask and object category, alongside occlusion-aware processing through the prediction of occluding masks. Subsequently, these elements are inputted into our DiffSP module to infer the shape prior of the object. DiffSP utilizes conditioned diffusion models pretrained on extensive datasets to extract rich visual features for shape prior estimation. Additionally, we introduce the Shape Prior Amodal Predictor, which utilizes attention-based feature maps from the shape prior to refine amodal segmentation. Experiments across various AIS benchmarks demonstrate the effectiveness of our AISDiff. Show Chinese abstract

Abstract: 无模式实例分割（AIS）提出了一个引人入胜的挑战，包括对图像中可见和遮挡部分的分割预测。 以往的方法常常依赖于从训练数据中获得的形状先验信息来增强无模式分割。 然而，这些方法容易过拟合，并且忽略了物体类别细节。 最近的进展表明，基于大规模数据集预训练的条件扩散模型在从潜在空间生成图像方面具有潜力。 受此启发，我们提出了AISDiff，其中包含一个扩散形状先验估计（DiffSP）模块。 AISDiff首先预测可见分割掩码和物体类别，并通过预测遮挡掩码进行遮挡感知处理。 随后，这些元素被输入到我们的DiffSP模块中以推断物体的形状先验。 DiffSP利用在大规模数据集上预训练的条件扩散模型来提取丰富的视觉特征用于形状先验估计。 此外，我们引入了形状先验无模式预测器，该预测器利用形状先验的注意力特征图来细化无模式分割。 在各种AIS基准测试中的实验展示了我们AISDiff的有效性。