For developing safe Autonomous Driving Systems (ADS), rigorous testing is required before they are deemed safe for road deployments. Since comprehensive conventional physical testing is impractical due to cost and safety concerns, Virtual Testing Environments (VTE) can be adopted as an alternative. Comparing VTE-generated sensor outputs against their real-world analogues can be a strong indication that the VTE accurately represents reality. Correspondingly, this work explores a comprehensive experimental approach to finding evaluation metrics suitable for comparing real-world and simulated LiDAR scans. The metrics were tested in terms of sensitivity and accuracy with different noise, density, distortion, sensor orientation, and channel settings. From comparing the metrics, we found that Density Aware Chamfer Distance (DCD) works best across all cases. In the second step of the research, a Virtual Testing Environment was generated using real LiDAR scan data. The data was collected in a controlled environment with only static objects using an instrumented vehicle equipped with LiDAR, IMU and cameras. Simulated LiDAR scans were generated from the VTEs using the same pose as real LiDAR scans. The simulated and LiDAR scans were compared in terms of model perception and geometric similarity. Actual and simulated LiDAR scans have a similar semantic segmentation output with a mIoU of 21\% with corrected intensity and an average density aware chamfer distance (DCD) of 0.63. This indicates a slight difference in the geometric properties of simulated and real LiDAR scans and a significant difference between model outputs. During the comparison, density-aware chamfer distance was found to be the most correlated among the metrics with perception methods.

翻译：为确保自动驾驶系统（ADS）的安全性，需在其上路部署前进行严格测试。鉴于传统物理测试因成本与安全限制难以全面实施，虚拟测试环境（VTE）可作为替代方案。通过对比VTE生成的传感器输出与其真实世界对应数据，可有效验证VTE对现实表征的准确性。为此，本研究探索了一种综合性实验方法，旨在寻找适用于对比真实与模拟激光雷达扫描数据的评估指标。这些指标在不同噪声、密度、畸变、传感器朝向及通道设置下进行了灵敏度与准确性测试。通过指标对比，我们发现密度感知倒角距离（DCD）在所有测试场景中表现最优。在研究的第二阶段，利用真实激光雷达扫描数据构建了虚拟测试环境。数据采集于受控静态场景中，使用搭载激光雷达、惯性测量单元（IMU）和相机的仪器化车辆完成。基于VTE生成与真实扫描位姿一致的模拟激光雷达数据，并从模型感知与几何相似性两个维度对比模拟与真实扫描。结果显示，经强度校正后，真实与模拟扫描的语义分割输出具有相似性（平均交并比mIoU为21%），平均密度感知倒角距离（DCD）为0.63。这表明模拟与真实激光雷达扫描在几何特性上存在细微差异，而模型输出结果差异显著。在对比过程中，密度感知倒角距离被证实是与感知方法相关性最高的评估指标。