Title: Deep Object Co-segmentation via Spatial-Semantic Network Modulation Show Chinese title

Title: 通过空间语义网络调制的深度物体共分割

Abstract: Object co-segmentation is to segment the shared objects in multiple relevant images, which has numerous applications in computer vision. This paper presents a spatial and semantic modulated deep network framework for object co-segmentation. A backbone network is adopted to extract multi-resolution image features. With the multi-resolution features of the relevant images as input, we design a spatial modulator to learn a mask for each image. The spatial modulator captures the correlations of image feature descriptors via unsupervised learning. The learned mask can roughly localize the shared foreground object while suppressing the background. For the semantic modulator, we model it as a supervised image classification task. We propose a hierarchical second-order pooling module to transform the image features for classification use. The outputs of the two modulators manipulate the multi-resolution features by a shift-and-scale operation so that the features focus on segmenting co-object regions. The proposed model is trained end-to-end without any intricate post-processing. Extensive experiments on four image co-segmentation benchmark datasets demonstrate the superior accuracy of the proposed method compared to state-of-the-art methods. Show Chinese abstract

Abstract: 对象共分割是分割多个相关图像中的共享对象，这在计算机视觉中有许多应用。 本文提出了一种空间和语义调制的深度网络框架用于对象共分割。 采用主干网络来提取多分辨率图像特征。 以相关图像的多分辨率特征作为输入，我们设计了一个空间调制器来为每张图像学习一个掩码。 空间调制器通过无监督学习捕捉图像特征描述符的相关性。 学习到的掩码可以粗略地定位共享前景对象同时抑制背景。 对于语义调制器，我们将它建模为一个监督图像分类任务。 我们提出了一种分层的二阶池化模块，以将图像特征转换为分类使用。 两个调制器的输出通过移位和缩放操作操纵多分辨率特征，从而使特征专注于分割共对象区域。 所提出的模型端到端训练，无需任何复杂的后处理。 在四个图像共分割基准数据集上的大量实验表明，所提出的方法相比最先进方法具有更高的准确性。