Title: A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems Show Chinese title

Title: 自动驾驶系统中现实差距的多模态评估

Abstract: Simulation-based testing is a cornerstone of Autonomous Driving System (ADS) development, offering safe and scalable evaluation across diverse driving scenarios. However, discrepancies between simulated and real-world behavior, known as the reality gap, challenge the transferability of test results to deployed systems. In this paper, we present a comprehensive empirical study comparing four representative testing modalities: Software-in-the-Loop (SiL), Vehicle-in-the-Loop (ViL), Mixed-Reality (MR), and full real-world testing. Using a small-scale physical vehicle equipped with real sensors (camera and LiDAR) and its digital twin, we implement each setup and evaluate two ADS architectures (modular and end-to-end) across diverse indoor driving scenarios involving real obstacles, road topologies, and indoor environments. We systematically assess the impact of each testing modality along three dimensions of the reality gap: actuation, perception, and behavioral fidelity. Our results show that while SiL and ViL setups simplify critical aspects of real-world dynamics and sensing, MR testing improves perceptual realism without compromising safety or control. Importantly, we identify the conditions under which failures do not transfer across testing modalities and isolate the underlying dimensions of the gap responsible for these discrepancies. Our findings offer actionable insights into the respective strengths and limitations of each modality and outline a path toward more robust and transferable validation of autonomous driving systems. Show Chinese abstract

Abstract: 基于仿真的测试是自动驾驶系统（ADS）开发的核心，它在各种驾驶场景中提供了安全且可扩展的评估。然而，仿真与现实世界行为之间的差异，即现实差距，挑战了测试结果向部署系统的转移性。在本文中，我们提出了一项全面的实证研究，比较了四种典型的测试模式：软件在环（SiL）、车辆在环（ViL）、混合现实（MR）和全真实世界测试。使用配备真实传感器（摄像头和激光雷达）的小规模物理车辆及其数字孪生体，我们实现了每种设置，并在涉及真实障碍物、道路拓扑和室内环境的多样化室内驾驶场景中评估了两种ADS架构（模块化和端到端）。我们系统地评估了每种测试模式在现实差距三个维度（执行、感知和行为保真度）上的影响。我们的结果表明，尽管SiL和ViL设置简化了现实世界动态和感知的关键方面，但MR测试在不牺牲安全或控制的情况下提高了感知的真实性。重要的是，我们确定了失败在不同测试模式之间不转移的条件，并隔离了导致这些差异的差距的基本维度。我们的发现提供了对每种模式各自优势和局限性的实际见解，并指明了更强大和可转移的自动驾驶系统验证路径。