Title: SalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Images Show Chinese title

Title: SalyPath360：全方位图像的显著性和注视路径预测框架

Abstract: This paper introduces a new framework to predict visual attention of omnidirectional images. The key setup of our architecture is the simultaneous prediction of the saliency map and a corresponding scanpath for a given stimulus. The framework implements a fully encoder-decoder convolutional neural network augmented by an attention module to generate representative saliency maps. In addition, an auxiliary network is employed to generate probable viewport center fixation points through the SoftArgMax function. The latter allows to derive fixation points from feature maps. To take advantage of the scanpath prediction, an adaptive joint probability distribution model is then applied to construct the final unbiased saliency map by leveraging the encoder decoder-based saliency map and the scanpath-based saliency heatmap. The proposed framework was evaluated in terms of saliency and scanpath prediction, and the results were compared to state-of-the-art methods on Salient360! dataset. The results showed the relevance of our framework and the benefits of such architecture for further omnidirectional visual attention prediction tasks. Show Chinese abstract

Abstract: 本文介绍了一种新框架，用于预测全向图像的视觉注意。 我们架构的关键设置是同时预测给定刺激的显著性图和相应的扫描路径。 该框架实现了一个完全编码器-解码器卷积神经网络，通过注意力模块进行增强以生成具有代表性的显著性图。 此外，采用了一个辅助网络，通过SoftArgMax函数生成可能的视口中心注视点。 后者允许从特征图中推导出注视点。 为了利用扫描路径预测，然后应用了一个自适应联合概率分布模型，通过利用基于编码器解码器的显著性图和基于扫描路径的显著性热图来构建最终的无偏显著性图。 所提出的框架在显著性和扫描路径预测方面进行了评估，并与Salient360!数据集上的最先进方法进行了比较。 结果表明了我们框架的相关性以及这种架构对进一步全向视觉注意预测任务的好处。