标题： POMATO：将点图匹配与时间运动相结合用于动态3D重建 显示英文标题

标题： POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstruction

摘要： 动态场景中的3D重建主要依赖于几何估计和匹配模块的结合，其中后者任务对于区分动态区域至关重要，这有助于减轻由相机和物体运动引入的干扰。 此外，匹配模块显式建模物体运动，使特定目标的跟踪成为可能，并推动了复杂场景中的运动理解。 最近，DUSt3R中提出的点图表示建议了一种统一几何估计和匹配的潜在解决方案，但它在动态区域中仍存在模糊匹配的问题，这可能会阻碍进一步的改进。 在本工作中，我们提出了POMATO，这是一种通过将点图匹配与时间运动相结合的动态3D重建统一框架。 具体来说，我们的方法首先通过将不同视图中动态和静态区域的RGB像素映射到统一坐标系内的3D点图，学习显式的匹配关系。 此外，我们引入了一个用于动态运动的时间运动模块，确保不同帧之间的尺度一致性，并提升需要精确几何和可靠匹配的任务性能，尤其是在3D点跟踪方面。 我们通过在多个下游任务中展示出色的表现，证明了所提出的点图匹配和时间融合范式的有效性，包括视频深度估计、3D点跟踪和姿态估计。 代码和模型可在https://github.com/wyddmw/POMATO公开获取。 显示英文摘要

摘要： 3D reconstruction in dynamic scenes primarily relies on the combination of geometry estimation and matching modules where the latter task is pivotal for distinguishing dynamic regions which can help to mitigate the interference introduced by camera and object motion. Furthermore, the matching module explicitly models object motion, enabling the tracking of specific targets and advancing motion understanding in complex scenarios. Recently, the proposed representation of pointmap in DUSt3R suggests a potential solution to unify both geometry estimation and matching in 3D space, but it still struggles with ambiguous matching in dynamic regions, which may hamper further improvement. In this work, we present POMATO, a unified framework for dynamic 3D reconstruction by marrying pointmap matching with temporal motion. Specifically, our method first learns an explicit matching relationship by mapping RGB pixels from both dynamic and static regions across different views to 3D pointmaps within a unified coordinate system. Furthermore, we introduce a temporal motion module for dynamic motions that ensures scale consistency across different frames and enhances performance in tasks requiring both precise geometry and reliable matching, most notably 3D point tracking. We show the effectiveness of the proposed pointmap matching and temporal fusion paradigm by demonstrating the remarkable performance across multiple downstream tasks, including video depth estimation, 3D point tracking, and pose estimation. Code and models are publicly available at https://github.com/wyddmw/POMATO.