3D semantic scene understanding remains a long-standing challenge in the 3D computer vision community. One of the key issues pertains to limited real-world annotated data to facilitate generalizable models. The common practice to tackle this issue is to simulate new data. Although synthetic datasets offer scalability and perfect labels, their designer-crafted scenes fail to capture real-world complexity and sensor noise, resulting in a synthetic-to-real domain gap. Moreover, no benchmark provides synchronized real and simulated point clouds for segmentation-oriented domain shift analysis. We introduce TrueCity, the first urban semantic segmentation benchmark with cm-accurate annotated real-world point clouds, semantic 3D city models, and annotated simulated point clouds representing the same city. TrueCity proposes segmentation classes aligned with international 3D city modeling standards, enabling consistent evaluation of synthetic-to-real gap. Our extensive experiments on common baselines quantify domain shift and highlight strategies for exploiting synthetic data to enhance real-world 3D scene understanding. We are convinced that the TrueCity dataset will foster further development of sim-to-real gap quantification and enable generalizable data-driven models. The data, code, and 3D models are available online: https://tum-gis.github.io/TrueCity/

翻译：三维语义场景理解一直是三维计算机视觉领域长期存在的挑战。关键问题之一在于现实世界标注数据的匮乏，难以支撑泛化模型的训练。解决此问题的常见方法是模拟生成新数据。尽管合成数据集具备可扩展性和完美标注，但其人工设计的场景无法捕捉现实世界的复杂性和传感器噪声，导致合成到真实数据的域间差距。此外，现有基准测试均未提供用于分割导向域偏移分析的同步真实与模拟点云数据。本文提出TrueCity，首个包含厘米级精度标注的真实世界点云、语义三维城市模型及对应同一城市的标注模拟点云的城市语义分割基准。TrueCity采用与国际三维城市建模标准对齐的分割类别，支持对合成到真实域差距的一致性评估。我们在常见基线模型上的大量实验量化了域偏移现象，并揭示了利用合成数据增强现实世界三维场景理解的有效策略。我们确信TrueCity数据集将推动模拟到真实域差距量化研究的深入发展，并为构建泛化数据驱动模型提供支撑。数据、代码及三维模型已在线发布：https://tum-gis.github.io/TrueCity/