标题： SEMNAV：一种以语义分割为导向的视觉语义导航方法 显示英文标题

标题： SEMNAV: A Semantic Segmentation-Driven Approach to Visual Semantic Navigation

摘要： 视觉语义导航（VSN）是机器人学中的一个基础问题，其中智能体必须在一个未知环境中主要利用视觉信息朝向目标物体导航。 大多数最先进的VSN模型是在模拟环境中训练的，这些环境最多使用真实世界的渲染场景。 这些方法通常依赖于虚拟场景的原始RGB数据，由于领域适应问题，这限制了它们在现实世界环境中的泛化能力。 为了解决这个问题，我们在本工作中提出了一种名为SEMNAV的新方法，该方法利用语义分割作为环境的主要视觉输入表示，以增强智能体的感知和决策能力。 通过明确引入高层次的语义信息，我们的模型学习到了在模拟和现实世界环境中都具有较高泛化能力的鲁棒导航策略。 我们还引入了一个新整理的数据集，即SEMNAV数据集，专门用于训练像SEMNAV这样的语义分割感知导航模型。 我们的方法在模拟环境和实际机器人平台上进行了广泛的评估。 实验结果表明，SEMNAV在Habitat 2.0仿真环境中使用HM3D数据集时，超越了现有的最先进的VSN模型，成功率达到更高水平。 此外，我们的现实世界实验突显了语义分割在缩小仿真到现实差距方面的有效性，使我们的模型成为实际基于VSN的机器人应用的有前景解决方案。 我们在https://github.com/gramuah/semnav上发布了SEMNAV数据集、代码和训练好的模型。 显示英文摘要

摘要： Visual Semantic Navigation (VSN) is a fundamental problem in robotics, where an agent must navigate toward a target object in an unknown environment, mainly using visual information. Most state-of-the-art VSN models are trained in simulation environments, where rendered scenes of the real world are used, at best. These approaches typically rely on raw RGB data from the virtual scenes, which limits their ability to generalize to real-world environments due to domain adaptation issues. To tackle this problem, in this work, we propose SEMNAV, a novel approach that leverages semantic segmentation as the main visual input representation of the environment to enhance the agent's perception and decision-making capabilities. By explicitly incorporating high-level semantic information, our model learns robust navigation policies that improve generalization across unseen environments, both in simulated and real world settings. We also introduce a newly curated dataset, i.e. the SEMNAV dataset, designed for training semantic segmentation-aware navigation models like SEMNAV. Our approach is evaluated extensively in both simulated environments and with real-world robotic platforms. Experimental results demonstrate that SEMNAV outperforms existing state-of-the-art VSN models, achieving higher success rates in the Habitat 2.0 simulation environment, using the HM3D dataset. Furthermore, our real-world experiments highlight the effectiveness of semantic segmentation in mitigating the sim-to-real gap, making our model a promising solution for practical VSN-based robotic applications. We release SEMNAV dataset, code and trained models at https://github.com/gramuah/semnav