标题： Scene4U：基于单张全景图像的分层3D场景重建，用于沉浸式探索 显示英文标题

标题： Scene4U: Hierarchical Layered 3D Scene Reconstruction from Single Panoramic Image for Your Immerse Exploration

摘要： 沉浸式和真实的三维场景重建在计算机视觉和计算机图形学的各个领域具有重要的实际意义。 通常，沉浸式和真实的场景应该不受动态物体遮挡的影响，保持全局纹理一致性，并允许无限制地探索。 当前主流的图像驱动场景构建方法涉及使用移动虚拟相机迭代优化初始图像以生成场景。 然而，先前的方法由于相机姿态变化下的全局纹理不一致而难以避免视觉不连续性，并且经常出现由前景-背景遮挡引起的场景空洞。 为此，我们提出了一种基于全景图的新颖分层三维场景重建框架，名为Scene4U。 具体而言，Scene4U将开放词汇分割模型与大型语言模型相结合，将真实全景图分解为多个层次。 然后，我们采用基于扩散模型的分层修复模块，利用视觉线索和深度信息恢复被遮挡的区域，生成场景的分层表示。 多层全景图随后初始化为三维高斯点阵表示，接着进行分层优化，最终生成支持自由探索的具有语义和结构一致性的沉浸式三维场景。 Scene4U在LPIPS指标上比最先进的方法提高了24.24%，在BRISQUE指标上提高了24.40%，同时实现了最快的训练速度。 此外，为了展示Scene4U的鲁棒性并让用户体验来自各种地标的真实场景，我们构建了WorldVista3D数据集用于三维场景重建，该数据集包含了全球著名地点的全景图像。 实现代码和数据集将在https://github.com/LongHZ140516/Scene4U发布。 显示英文摘要

摘要： The reconstruction of immersive and realistic 3D scenes holds significant practical importance in various fields of computer vision and computer graphics. Typically, immersive and realistic scenes should be free from obstructions by dynamic objects, maintain global texture consistency, and allow for unrestricted exploration. The current mainstream methods for image-driven scene construction involves iteratively refining the initial image using a moving virtual camera to generate the scene. However, previous methods struggle with visual discontinuities due to global texture inconsistencies under varying camera poses, and they frequently exhibit scene voids caused by foreground-background occlusions. To this end, we propose a novel layered 3D scene reconstruction framework from panoramic image, named Scene4U. Specifically, Scene4U integrates an open-vocabulary segmentation model with a large language model to decompose a real panorama into multiple layers. Then, we employs a layered repair module based on diffusion model to restore occluded regions using visual cues and depth information, generating a hierarchical representation of the scene. The multi-layer panorama is then initialized as a 3D Gaussian Splatting representation, followed by layered optimization, which ultimately produces an immersive 3D scene with semantic and structural consistency that supports free exploration. Scene4U outperforms state-of-the-art method, improving by 24.24% in LPIPS and 24.40% in BRISQUE, while also achieving the fastest training speed. Additionally, to demonstrate the robustness of Scene4U and allow users to experience immersive scenes from various landmarks, we build WorldVista3D dataset for 3D scene reconstruction, which contains panoramic images of globally renowned sites. The implementation code and dataset will be released at https://github.com/LongHZ140516/Scene4U .