标题： SIU3R：超越特征对齐的同步场景理解和三维重建 显示英文标题

标题： SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignment

摘要： 同时理解和3D重建在开发端到端具身智能系统中起着重要作用。 为了实现这一点，最近的方法采用2D到3D特征对齐范式，这导致了有限的3D理解能力和潜在的语义信息丢失。 鉴于此，我们提出了SIU3R，这是第一个从无姿势图像中进行可推广的同时理解和3D重建的无对齐框架。 具体来说，SIU3R通过像素对齐的3D表示连接重建和理解任务，并将多个理解任务统一为一组统一的可学习查询，从而无需与2D模型对齐即可实现原生的3D理解。 为了促进两个任务之间共享表示的合作，我们进一步深入分析了它们的相互好处，并提出了两个轻量级模块来促进它们的交互。 大量实验表明，我们的方法不仅在3D重建和理解的单独任务上取得了最先进的性能，而且在同时理解和3D重建的任务上也取得了最先进的性能，突显了我们无对齐框架的优势和相互好处设计的有效性。 显示英文摘要

摘要： Simultaneous understanding and 3D reconstruction plays an important role in developing end-to-end embodied intelligent systems. To achieve this, recent approaches resort to 2D-to-3D feature alignment paradigm, which leads to limited 3D understanding capability and potential semantic information loss. In light of this, we propose SIU3R, the first alignment-free framework for generalizable simultaneous understanding and 3D reconstruction from unposed images. Specifically, SIU3R bridges reconstruction and understanding tasks via pixel-aligned 3D representation, and unifies multiple understanding tasks into a set of unified learnable queries, enabling native 3D understanding without the need of alignment with 2D models. To encourage collaboration between the two tasks with shared representation, we further conduct in-depth analyses of their mutual benefits, and propose two lightweight modules to facilitate their interaction. Extensive experiments demonstrate that our method achieves state-of-the-art performance not only on the individual tasks of 3D reconstruction and understanding, but also on the task of simultaneous understanding and 3D reconstruction, highlighting the advantages of our alignment-free framework and the effectiveness of the mutual benefit designs.