标题： 通过正则化Gromov-Wasserstein最优传输的程序学习 显示英文标题

标题： Procedure Learning via Regularized Gromov-Wasserstein Optimal Transport

摘要： 我们研究自监督过程学习的问题，该问题从一组未标记的过程视频中发现关键步骤并建立它们的顺序。 以往的过程学习方法通常在确定关键步骤及其顺序之前学习视频之间的帧到帧对应关系。 然而，它们的性能常常受到顺序变化、背景/冗余帧和重复动作的影响。 为了克服这些挑战，我们提出了一种自监督过程学习框架，该框架利用融合的Gromov-Wasserstein最优传输公式，并结合结构先验来计算视频之间的帧到帧映射。 然而，仅优化上述时间对齐项可能导致退化解，其中所有帧都被映射到嵌入空间中的一个小簇，因此每个视频仅与一个关键步骤相关。 为了解决这一限制，我们进一步集成了一个对比正则化项，该项将不同的帧映射到嵌入空间中的不同点，避免陷入平凡解。 最后，我们在大规模的第一视角（即EgoProceL）和第三人称（即ProceL和CrossTask）基准上进行了广泛的实验，以证明我们的方法相对于以前的方法（包括依赖于传统Kantorovich最优传输公式和最优性先验的OPEL）具有优越的性能。 显示英文摘要

摘要： We study the problem of self-supervised procedure learning, which discovers key steps and establishes their order from a set of unlabeled procedural videos. Previous procedure learning methods typically learn frame-to-frame correspondences between videos before determining key steps and their order. However, their performance often suffers from order variations, background/redundant frames, and repeated actions. To overcome these challenges, we propose a self-supervised procedure learning framework, which utilizes a fused Gromov-Wasserstein optimal transport formulation with a structural prior for computing frame-to-frame mapping between videos. However, optimizing exclusively for the above temporal alignment term may lead to degenerate solutions, where all frames are mapped to a small cluster in the embedding space and hence every video is associated with only one key step. To address that limitation, we further integrate a contrastive regularization term, which maps different frames to different points in the embedding space, avoiding the collapse to trivial solutions. Finally, we conduct extensive experiments on large-scale egocentric (i.e., EgoProceL) and third-person (i.e., ProceL and CrossTask) benchmarks to demonstrate superior performance by our approach against previous methods, including OPEL which relies on a traditional Kantorovich optimal transport formulation with an optimality prior.