标题： TRACE：从多视角视频中学习3D高斯物理动力学 显示英文标题

标题： TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos

摘要： 在本文中，我们旨在仅从动态多视角视频中建模3D场景的几何、外观和物理信息，而无需任何人工标签。 通过利用物理信息损失作为软约束或将简单的物理模型集成到神经网络中，现有工作往往无法学习复杂的运动物理规律，或者这样做需要额外的标签，如物体类型或掩码。 我们提出了一种名为TRACE的新框架来建模复杂动态3D场景的运动物理规律。 我们方法的关键创新之处在于，通过将每个3D点表示为具有大小和方向的刚性粒子，我们直接为每个粒子学习一个平移旋转动力学系统，显式估计一组完整的物理参数来控制粒子随时间的运动。 在三个现有的动态数据集和一个新创建的具有挑战性的合成数据集上的大量实验表明，我们的方法在未来的帧外推任务中优于基线方法。 我们框架的一个优点是，只需通过聚类学习到的物理参数就可以轻松分割多个对象或部分。 显示英文摘要

摘要： In this paper, we aim to model 3D scene geometry, appearance, and physical information just from dynamic multi-view videos in the absence of any human labels. By leveraging physics-informed losses as soft constraints or integrating simple physics models into neural nets, existing works often fail to learn complex motion physics, or doing so requires additional labels such as object types or masks. We propose a new framework named TRACE to model the motion physics of complex dynamic 3D scenes. The key novelty of our method is that, by formulating each 3D point as a rigid particle with size and orientation in space, we directly learn a translation rotation dynamics system for each particle, explicitly estimating a complete set of physical parameters to govern the particle's motion over time. Extensive experiments on three existing dynamic datasets and one newly created challenging synthetic datasets demonstrate the extraordinary performance of our method over baselines in the task of future frame extrapolation. A nice property of our framework is that multiple objects or parts can be easily segmented just by clustering the learned physical parameters.