Research VirtuaLearn3D
Virtual Laser Scanning for Machine Learning Algorithms in Geographic 3D Point Cloud Analysis
Stay up-to-date by following VirtuaLearn3D via our GIScience News Blog and on LinkedIn: #VirtuaLearn3D #3DGeo.
News
We have taken a major step forward in the development of LiDAR-based environmental monitoring: we use animated scenes in LiDAR simulations, enabling them to take motion effects into account and to simulate time series. This new approach is called VLS-4D (virtual laser scanning of dynamic scenes). Read our review paper in the journal “Methods in Ecology and Evolution”, which introduces the concept, summarizes the current state of research and outlines exciting applications in vegetation monitoring. One key application is the generation of training data, where VLS-4D can be used to develop automated, point-cloud-based AI methods that allow dynamic environmental processes to be examined more effectively than before.
From static to dynamic virtual worlds. We present our novel framework VLS-4D and provide demo simulations that can be run with HELIOS++.
Objective
Topographic laser scanning (LS) is a leading remote sensing technique to derive detailed 3D point cloud representations of the Earth’s surface and its objects. Virtual laser scanning (VLS) simulations recreate real-world scenarios of LS acquisitions in a computer environment. VLS is useful when real experiments are not feasible, e.g. due to technical, economic and logistic constraints.
Recent advances in machine learning, in particular supervised deep learning, indicate a huge potential to improve geographic 3D point cloud analysis of complex natural objects (e.g. vegetation) and scenes (e.g. geomorphological settings). The success of deep learning algorithms strongly depends on the availability of high-quality and appropriately large amounts of training data. The main aim of this project is to advance the concept of virtual laser scanning to tackle the lack of training data to enable powerful machine learning algorithms for geographic point cloud analysis.
Links
WORKFLOW
FOCUS
This proposed methodological step will push large-scale usage of VLS simulations for machine learning and opens up completely new fields of applications. Focus of this project will be on airborne laser scanning (incl. UAV-borne LS) and the tasks of object-based tree species classification and semantic urban scene classification, though the relevance of the developed generic concepts is not limited by the investigated examples.
CENTRAL TARGETS
- Optimal Real-Simulated LS Data Fusion for Unsupervised Training: Find effective combinations of real LS data with theoretically unlimited amounts of simulated LS data for supervised training. Effective solutions can close the reality gap from simulated to real data and keep high classification accuracy while reducing costly input data.
- Enhancing Transfer Learning with VLS-Generated Data: Find out to what degree VLS data generation can support transfer learning strategies to enable the usage of pre-trained models for transfer to different geographic characteristics and types of LS data. VLS-supported transfer learning is highly demanded due to drastically increasing availability of LiDAR technology in sciences and also on daily-life devices.
- Development of Dynamic Objects in VLS simulations: Develop and test a new concept of ‘dynamic objects’ in VLS simulations, which enables e.g. to include vegetation with phenological changes and also moving objects (e.g. plants, cars, people).
Therefore: VirtuaLearn3D will push large-scale usage of VLS simulations for machine learning - in particular deep learning approaches - and our results will open up new fields of applications in science and industry.
FUNDING
Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)
Project number: 496418931
Duration: 2022 - 2025
