Research QSIM-DL

LiDAR point clouds acquired by remote and close-range sensing have become an increasingly important and often primary source of 3D data in research and in operational applications. Recent advances in deep learning (DL)-based methods for automatic topographic mapping, geo-information extraction, change detection, and autonomous operations work directly on the LiDAR representations of the environment, i.e., 3D point clouds, marking a disruptive shift in methodology compared to previous 2D approaches. Supervised DL approaches require significant amounts of training data in the form of annotated point clouds which were often created for specific use cases with extensive and tedious manual labeling. With geodetic and physics-based LiDAR simulators (e.g., HELIOS++, BlAInder), we have a method to overcome several challenges (e.g., limited training data, option of future sensor testing), but we still face the problem of unknown task-aware quality of simulated LiDAR point clouds. The main objective of QSIM-DL is to systematically investigate, quantify and trace the task-aware quality of simulated LiDAR point clouds in DL tasks. We will build and analyze a database of corresponding real-world LiDAR data and their simulated point cloud counterparts in different quality levels. We will cover representative DL applications (e.g., semantic, instance segmentation) for outdoor scenarios (with TLS, UAV LiDAR of vegetation) and indoor close-range mapping applications (short-range, solid-state LiDAR). We will investigate how the quality level of synthetic data is affected by the applied complexity of LiDAR simulation and the level of detail of the input 3D scene, and how this affects the domain transfer and cross-domain generalization of DL models. The derived diagnostic foundation will enable us to develop a framework to optimize synthetic data generation efforts as a selective feedback loop between training data generation, DL model training and task application. Furthermore, we will identify DL task-aware critical thresholds of synthetic data quality. We expect our findings to have a broad impact both for optimal combination of real and synthetic data in DL training, and for more effective development of generative AI models with enhanced capabilities. Both the database and our data generation and DL framework will be made openly available.