Unmanned Aerial Vehicles (UAVs) have shown to undeniably be one of the most disruptive emerging technologies over the last decades —touching many aspects in different research domains, and industrial and commercial applications. Within the geo-information and Earth Observation science domain, the applications of UAV range from environmental monitoring to disaster, land, and water management. Hardware and domain expertise aside, the core building block that enables us, researchers and practitioners, to carry out these UAV-related solutions is the photogrammetric pipeline.
This research aims to develop a dense image matching method, one of the key steps in the photogrammetric pipeline that transforms overlapping views of images into dense 3D information, based on an unsupervised deep network utilizing the parallax attention mechanism. The proposed approach effectively extracts dense point cloud from very high resolution UAV images.
Initial development of the point cloud extraction routine was done in the Geospatial Computing Platform of ITC. Figure 1 shows the steps within the point cloud extraction routine together with an overview of the parallax attention stereo matching network used in the dense image matching step.
Figure 1. UAV point cloud extraction pipeline (left) and parallax attention stereo matching network (right).
For the experiments, we used a dataset containing very high-resolution nadir-looking UAV images capturing a scene of a neighborhood in a town in the central Netherlands called Nunspeet. There are 312 images in total, the raw UAV images have a dimension of 4032 × 3024 and an average ground sampling distance of about 17 mm. Figure 2 shows the error maps and error statistics of different setups of the point cloud extraction routine we experimented with for a couple of sample image pairs. There are notable improvements on the accuracy of resulting extracted point cloud when using a parallax attention stereo matching network for the dense image matching step, whether pretrained or finetuned in an unsupervised manner, over the conventional semi-global matching (SGM) method.
Figure 2. Error map and error statistics for sample image pairs.
One of the major challenges in conducting the above work is the limited open-source solutions for photogrammetric pipeline that can easily be modified to plug-in the dense image matching method we are experimenting with. The Geospatial Computing Platform offers a great development environment to easily kickstart coding such a solution from scratch using open source tools and resources readily available in the platform. The user interface was also very intuitive so there is little entry barrier for first time users to get comfortable with using the platform.
For more information:
Bergado, J. R. and Nex, F. C. (2023). Dense point cloud extraction from UAV imagery using parallax attention. In N. El-Sheimy, A. Abdelbary, N. El-Bandary, & Y. Hohasseb (Eds.), ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2023 (W1-2023 ed., Vol. X-1, pp. 1027-1032). (ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences). https://doi.org/10.5194/isprs-annals-X-1-W1-2023-1027-2023