标题： 让前馈重建模型作为三维生成模型的潜在编码器 显示英文标题

标题： Taming Feed-forward Reconstruction Models as Latent Encoders for 3D Generative Models

摘要： 基于人工智能的3D内容创作最近主要沿着两个方向发展：前馈图像到3D重建方法和使用2D或3D监督训练的3D生成模型。在本工作中，我们表明现有的前馈重建方法可以作为训练3D生成模型的有效潜在编码器，从而弥合这两种范式。通过重用强大的预训练重建模型，我们避免了计算成本高昂的编码器网络训练，并免费获得了丰富的3D潜在特征用于生成建模。然而，由于重建模型的潜在空间具有无结构特性，因此并不适合生成建模。为了在这些潜在特征上进行基于流的模型训练，我们开发了后处理管道，包括标准化特征的协议和空间加权以专注于重要区域。我们进一步结合了2D图像空间感知渲染损失来处理高维潜在空间。最后，我们提出了一种多流变压器修正流架构，以实现线性扩展和高质量的文本条件3D生成。我们的框架利用前馈重建模型的进展来增强3D生成建模的可扩展性，在文本到3D生成中实现了高计算效率和最先进的性能。 显示英文摘要

摘要： Recent AI-based 3D content creation has largely evolved along two paths: feed-forward image-to-3D reconstruction approaches and 3D generative models trained with 2D or 3D supervision. In this work, we show that existing feed-forward reconstruction methods can serve as effective latent encoders for training 3D generative models, thereby bridging these two paradigms. By reusing powerful pre-trained reconstruction models, we avoid computationally expensive encoder network training and obtain rich 3D latent features for generative modeling for free. However, the latent spaces of reconstruction models are not well-suited for generative modeling due to their unstructured nature. To enable flow-based model training on these latent features, we develop post-processing pipelines, including protocols to standardize the features and spatial weighting to concentrate on important regions. We further incorporate a 2D image space perceptual rendering loss to handle the high-dimensional latent spaces. Finally, we propose a multi-stream transformer-based rectified flow architecture to achieve linear scaling and high-quality text-conditioned 3D generation. Our framework leverages the advancements of feed-forward reconstruction models to enhance the scalability of 3D generative modeling, achieving both high computational efficiency and state-of-the-art performance in text-to-3D generation.