ISPRS TEST ON EXTRACTING DEMs FROM POINT CLOUDS:
A comparison of existing automatic filters
Introduction
Dear Colleagues,
Over the last ten years a variety of algorithms has been developed for automatically extracting DEMs from point clouds produced by laser scanning and InSAR. While this collection of algorithms exists, a comparative study of the performance of the filters is yet to be done. In line with the framework of ISPRS Commission III, the Working Group III/3 "3D Reconstruction from Airborne Laser Scanner and InSAR Data" initiated a study to compare the performance of the various automatic filters developed to date with the aim of identifying future research directions in filtering of point clouds for DEM production.
The results of this study are now available. Below you find the original invitation for participation. The data of the test can still be downloaded.
Delft, August 5th, 2003
George Vosselman
Objectives
The study is driven by three main objectives, which are:
- To determine the comparative performance of existing filters - one feature of filters is that they are not universally applicable. They will work under most scenarios (combination and distribution of features on the terrain), but there are situations in which they will fail. Therefore it is of interest to find what filter strategy will work under what circumstances.
- To determine the sensitivity of filtering algorithms under varrying point densities.
- To determine problems in the filtering of point clouds that still need further attention.
Participation
Both developers and users of filters are welcome to participate in this study. Participants are invited to download the data provided on this site. More information about the format and structure of the data files is provided on the download page.
Those who do decide to participate in the study are kindly asked to inform us as soon as they download the data. In this way we will be able to keep a record of participants, and provide information in case we need to make any changes to the parameters of the study.
Twelve sets of data are provided (two sites have data at three different resolutions). Participants in the study are kindly requested to process all twelve data sets if possible.
The deadline for submitting passed on 1 June 2002.
Methodology
The analyses to be performed will be both quantitative and qualitative. For the purpose of the analyses eight sites have been chosen (the choice of the sites is explained below).
Quantitative analysis
The data from the eight sites mentioned above have been manually filtered. From these manually filtered sites TINs will be produced.
The height differences between the filter results (submitted by the participants)and their corresponding TINs (derived from the manually filtered data) will be computed and quantitative measures (e.g., rms) derived. The derived measures will then be used to compare the performance of filters.
To ensure that the analysis is as objective as possible both global and local comparisons will be done for each of the sites. The global comparisons will be used to gauge how the filters perfom overally in different regions (urban, rural, etc.,). The local comparison will be used to gauge how the filters perform for specific features of interest (e.g., open spaces, bridges, etc.).
Qualitative analysis
There are some features in the data sets that are open to different treatment by filtering algorithms. An example of this are bridges. A bridge can be treated as part of the terrain or not as part of the terrain depending on the application for which the DEM is going to be used. Qualitative analysis of the filters is meant to find how the different filters treat such features. Visual analysis will be employed for qualitative comparisons.
Test sites
As part of second phase of the OEEPE project on laser scanning companies were invited to fly over the Vaihingen/Enz test field and Stuttgart city centre. These areas were chosen because of their diverse feature content (open fields, vegetation, buildings, roads, railroads, rivers, bridges, powerlines, water surfaces, etc.,). However, the areas fall into two groupings, City and Forest. From within the two groupings eight sites (numbered 1 through 8) have been selected for the comparison of filtering algorithms. The sites represent four regions with urban characteristics and another four with rural characteristics.
The data for the sites is extracted from laser scanning data produced by FOTONOR AS. The areas were scanned with an Optech laser scanner, and both first and last pulse data were recorded.
Submitting results
The submission deadline was June 1, 2002. Submission of results is no longer possible.
Results
On completion of the study the results will be published in a scientific journal and presented at a conference (with acknowledgement of all participants). Furthermore, a summary of the results will also be posted on this site.
It is emphasised again that the purpose of the study is to identify future research directions in the area of filtering laser scanner data based on the performance of current filters. Therefore, in the publication of results the performance of filtering algorithms will be reported as per the findings in the test and no filtering algorithm will be endorsed over others.