标题： 图像目标导航中真正重要的是什么？ 显示英文标题

标题： What does really matter in image goal navigation?

摘要： 图像目标导航需要两种不同的技能：首先，核心导航技能，包括自由空间和障碍物的检测，以及基于内部表示做出决策；其次，通过将视觉观察与目标图像进行比较来计算方向信息。 当前最先进的方法要么依赖于专门的图像匹配，要么依赖于计算机视觉模块在相对姿态估计上的预训练。 在本文中，我们研究了是否可以通过强化学习对完整智能体进行端到端训练来高效解决此任务，这已被近期的工作所声称。 肯定的答案将超越具身人工智能的影响，并允许仅通过奖励训练相对姿态估计用于导航。 在一项大规模研究中，我们调查了架构选择如后期融合、通道堆叠、空间到深度投影和交叉注意力的影响，以及它们在从导航训练中出现相对姿态估计器中的作用。 我们表明，最近方法的成功在一定程度上受到模拟器设置的影响，导致了模拟中的捷径。 然而，我们还表明这些能力可以在一定程度上转移到更现实的环境中。 我们还发现了导航性能与探测（出现的）相对姿态估计性能之间的相关性，这是一个重要的子技能。 显示英文摘要

摘要： Image goal navigation requires two different skills: firstly, core navigation skills, including the detection of free space and obstacles, and taking decisions based on an internal representation; and secondly, computing directional information by comparing visual observations to the goal image. Current state-of-the-art methods either rely on dedicated image-matching, or pre-training of computer vision modules on relative pose estimation. In this paper, we study whether this task can be efficiently solved with end-to-end training of full agents with RL, as has been claimed by recent work. A positive answer would have impact beyond Embodied AI and allow training of relative pose estimation from reward for navigation alone. In a large study we investigate the effect of architectural choices like late fusion, channel stacking, space-to-depth projections and cross-attention, and their role in the emergence of relative pose estimators from navigation training. We show that the success of recent methods is influenced up to a certain extent by simulator settings, leading to shortcuts in simulation. However, we also show that these capabilities can be transferred to more realistic setting, up to some extend. We also find evidence for correlations between navigation performance and probed (emerging) relative pose estimation performance, an important sub skill.