Title: Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues Show Chinese title

Title: 通过高效解码器和时空动作线索的极简视频显著性预测

Abstract: This paper introduces ViNet-S, a 36MB model based on the ViNet architecture with a U-Net design, featuring a lightweight decoder that significantly reduces model size and parameters without compromising performance. Additionally, ViNet-A (148MB) incorporates spatio-temporal action localization (STAL) features, differing from traditional video saliency models that use action classification backbones. Our studies show that an ensemble of ViNet-S and ViNet-A, by averaging predicted saliency maps, achieves state-of-the-art performance on three visual-only and six audio-visual saliency datasets, outperforming transformer-based models in both parameter efficiency and real-time performance, with ViNet-S reaching over 1000fps. Show Chinese abstract

Abstract: 本文介绍了ViNet-S，一个基于ViNet架构的36MB模型，采用U-Net设计，具有轻量级解码器，显著减少了模型大小和参数而不会影响性能。 此外， ViNet-A（148MB）结合了时空动作定位（STAL）特征，不同于传统视频显著性模型使用的动作分类主干。 我们的研究表明，通过平均预测的显著性图，ViNet-S和ViNet-A的集成在三个仅视觉和六个视听显著性数据集上实现了最先进的性能，其参数效率和实时性能均优于基于变压器的模型，ViNet-S的帧率超过1000fps。