Title: View-Dependent Deformation Fields for 2D Editing of 3D Models Show Chinese title

Title: 视点相关的变形场用于三维模型的二维编辑

Abstract: We propose a method for authoring non-realistic 3D objects (represented as either 3D Gaussian Splats or meshes), that comply with 2D edits from specific viewpoints. Namely, given a 3D object, a user chooses different viewpoints and interactively deforms the object in the 2D image plane of each view. The method then produces a "deformation field" - an interpolation between those 2D deformations in a smooth manner as the viewpoint changes. Our core observation is that the 2D deformations do not need to be tied to an underlying object, nor share the same deformation space. We use this observation to devise a method for authoring view-dependent deformations, holding several technical contributions: first, a novel way to compositionality-blend between the 2D deformations after lifting them to 3D - this enables the user to "stack" the deformations similarly to layers in an editing software, each deformation operating on the results of the previous; second, a novel method to apply the 3D deformation to 3D Gaussian Splats; third, an approach to author the 2D deformations, by deforming a 2D mesh encapsulating a rendered image of the object. We show the versatility and efficacy of our method by adding cartoonish effects to objects, providing means to modify human characters, fitting 3D models to given 2D sketches and caricatures, resolving occlusions, and recreating classic non-realistic paintings as 3D models. Show Chinese abstract

Abstract: 我们提出了一种方法，用于创建符合特定视点的2D编辑要求的非写实3D对象（表示为3D高斯点云或网格）。具体来说，给定一个3D对象后，用户选择不同的视点并交互式地变形每个视图的2D图像平面中的对象。然后该方法生成一个“变形场”——以平滑的方式在这些2D变形之间进行插值，随着视点的变化而变化。 我们的核心观察是，2D变形不需要与底层对象绑定，也不需要共享相同的变形空间。我们利用这一观察来设计一种用于创建视点相关的变形的方法，具有多个技术贡献：首先，提出了一种新颖的方式，在将2D变形提升到3D后组合性地混合它们——这使得用户可以像编辑软件中的图层一样“堆叠”变形，每个变形都基于前一个的结果进行操作；其次，提出了一种新颖的方法，将3D变形应用于3D高斯点云；第三，通过变形封装对象渲染图像的2D网格来创建2D变形的方法。 我们通过向对象添加卡通效果、提供修改人物角色的手段、将3D模型拟合到给定的2D草图和漫画、解决遮挡问题以及重新创建经典非写实绘画作为3D模型，展示了我们方法的多样性和有效性。