标题： SDGOCC：用于3D多模态占用预测的语义和深度引导的鸟瞰图变换 显示英文标题

标题： SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction

摘要： 多模态3D占用预测因其在自动驾驶中的潜力而受到广泛关注。 然而，大多数现有方法是单模态的：基于相机的方法缺乏深度信息，而基于LiDAR的方法则难以处理遮挡问题。 当前轻量级方法主要依赖于Lift-Splat-Shoot（LSS）流程，该流程存在深度估计不准确的问题，并且未能充分利用3D LiDAR点的几何和语义信息。 因此，我们提出了一种名为SDG-OCC的新型多模态占用预测网络，该网络结合了联合语义和深度引导的视角变换以及融合到占用驱动的主动蒸馏。 增强的视角变换通过整合像素语义和共点深度信息，利用扩散和双线性离散化构建准确的深度分布。 融合到占用驱动的主动蒸馏从多模态数据中提取丰富的语义信息，并根据LiDAR识别的区域，有选择地将知识转移到图像特征中。 最后，为了获得最佳性能，我们引入了SDG-Fusion，它仅使用融合，以及SDG-KL，它结合了融合和蒸馏以实现更快的推理。 我们的方法在Occ3D-nuScenes数据集上实现了实时处理的最先进（SOTA）性能，并在更具挑战性的SurroundOcc-nuScenes数据集上表现出可比的性能，证明了其有效性和鲁棒性。 代码将在https://github.com/DzpLab/SDGOCC发布。 显示英文摘要

摘要： Multimodal 3D occupancy prediction has garnered significant attention for its potential in autonomous driving. However, most existing approaches are single-modality: camera-based methods lack depth information, while LiDAR-based methods struggle with occlusions. Current lightweight methods primarily rely on the Lift-Splat-Shoot (LSS) pipeline, which suffers from inaccurate depth estimation and fails to fully exploit the geometric and semantic information of 3D LiDAR points. Therefore, we propose a novel multimodal occupancy prediction network called SDG-OCC, which incorporates a joint semantic and depth-guided view transformation coupled with a fusion-to-occupancy-driven active distillation. The enhanced view transformation constructs accurate depth distributions by integrating pixel semantics and co-point depth through diffusion and bilinear discretization. The fusion-to-occupancy-driven active distillation extracts rich semantic information from multimodal data and selectively transfers knowledge to image features based on LiDAR-identified regions. Finally, for optimal performance, we introduce SDG-Fusion, which uses fusion alone, and SDG-KL, which integrates both fusion and distillation for faster inference. Our method achieves state-of-the-art (SOTA) performance with real-time processing on the Occ3D-nuScenes dataset and shows comparable performance on the more challenging SurroundOcc-nuScenes dataset, demonstrating its effectiveness and robustness. The code will be released at https://github.com/DzpLab/SDGOCC.