Greetings from the Big Geodata Newsletter!
In this issue you will find information on 10th anniversary of PDOK and its new services, recent developments in efficient creation of multi-scale Zarr pyramids to boost big data storage and access, global urban green space dataset covering more than 1000 cities, and the GeoAI challenges to accelerate the implementation and monitoring of SDGs.
Happy reading!
You can access the previous issues of the newsletter on our web portal. If you find the newsletter useful, please share the subscription link below with your network.
PDOK: A Decade of Success with OGC Services
Image credits: OGC, 2024
Public Services on the Map (Publieke Dienstverlening Op de Kaart) or PDOK, maintained by Kadaster in The Netherlands, celebrated its 10th anniversary recently. The Open Geospatial Consortium (OGC) highlights its milestone achievements in this article. Launched in 2013, the PDOK platform makes the Dutch Government’s geospatial datasets Findable, Accessible, Interoperable, and Reusable (FAIR) for both the public and private sectors. From hosting 40 datasets and handling 580 million server requests annually in 2013, the platform now hosts 210 datasets and handles 30 billion server requests annually! Majority of the datasets on PDOK are made available using OGC standards like Web Map Service (WMS) and Web Feature Service (WFS) and the Web Map Tile Service (WMTS). This includes, the Basic Topography Register, aerial imagery (updated annually at 8 and 25 cm GSD), Large Scale Topography Register at 1:500 and 1:5,000 scales, 3D dataset of buildings and many more. Kadaster is now in the process of transitioning to newer OGC API Standards, including OGC API – Tiles for Vector Tiles and OGC API – Features for query services. Earth Observation data is made available using WMTS is stored as OGC Cloud Optimized GeoTIFFs (CoG) and the 3D Building data encoded as OGC 3D Tiles is served using the OGC API – 3D GeoVolumes standard. The usage of new OGC APIs is said to have not only helped in handling the increasing volume of server requests but has also made it easier to track compute and memory usage on servers and optimise performance.
More information on the recent developments can be found in this article. Explore PDOK datasets for The Netherlands on the 2D and 3D data viewing platforms.
Zarr Pyramids at Scale: Boosting Data Storage and Access on the Cloud
Image credits: Raphael Hagen, 2024
CarbonPlan has unveiled major updates to its toolkit for web-based data visualization, focusing on the efficient creation of multi-scale Zarr pyramids. These pyramids enable visualizing high-resolution Earth science data, allowing seamless transitions from global overviews to detailed zoom-ins. The updated ndpyramid Python library now features methods, which use the pyresample library and Dask to achieve significant performance boosts—over five times faster for bilinear interpolation—cutting cloud processing costs and mitigating memory issues. Moreover, the pangeo-forge-ndpyramid extension to Pangeo Forge simplifies pyramid creation from existing Zarr stores or collections of archival files, enabling scalable processing in large-data frameworks like Apache Spark and Google Dataflow.
For more details, check out here. Detailed demo notebooks are available in the ndpyramid and pangeo-forge-ndpyramid repositories.
GeoAI Challenges to Accelerate the Implementation and Monitoring of SDGs
Image credits: AI for Good, 2023
Organised by the National Institute of Statistics and Geography (INEGI), the International Telecommunication Union (ITU), AI for Good, the United Nations Global Geospatial Information Management (UN-GGIM) Academic Network and UN Geospatial Network, the GeoAI Challenges aim to collaboratively address real-world geospatial problems by applying Artificial Intelligence. Open to students and professionals from around the world, the 3rd edition of the challenges features five problem statements that attempt to accelerate progress on the implementation and monitoring of the United Nations Sustainable Development Goals (SDGs). The challenges include human settlement detection, vegetation mapping, agricultural plastic cover mapping, ground-level NO2 estimation, and clandestine runways detection in the Peruvian Amazonian basin. The winners of these challenges will be awarded at the prestigious UN-GGIM Seventh High-Level Forum, to be held in Mexico City, on October 8-10, 2024.
More information on the timeline and guidelines for each challenge is available on the GeoAI Challenge page and in the recent webinar conducted introducing the Human Settlement Detection and Vegetation Mapping Challenges. ITU also provides free-of-charge computing platforms (GPUs and CPUs) to participants with a hosted Jupyter Notebook server and pre-installed machine learning packages.
Upcoming Meetings
- SURF Training: Introduction to Supercomputing, part I
3 September 2024, Amsterdam
- SURF Training: High-Performance Deep Learning
9-10 September 2024, Amsterdam - QGIS User Conference
9-10 September 2024, Bratislava - SURF Training: Introduction to SURF Research Cloud
17 September 2024, Amsterdam - CRIB Training: Introduction to Geospatial Raster and Vector Data with Python
18-19 September 2024, ITC, Enschede - Call for Abstracts: Living Planet Symposium
14 October – 22 November 2024, (Vienna, Austria, 23-27 June, 2025) - National Open Science Festival
22 October 2024, Maastricht University
Recent Releases
- cuGraph: RAPIDS GPU-accelerated Graph Analytics Library
24.08.00 (8/8/2024) - OpenBLAS: Optimized BLAS library
0.3.28 (8/8/2024) - Keras: Deep Learning for humans
3.5.0 (12/8/2024) - cuPy: NumPy/SciPy-compatible array library for GPU-accelerated computing
13.3.0 (22/8/2024)
The "Big" Picture
Image credits: Wu et al., 2024
Urban green spaces are crucial for ecological balance and social well-being, forming a key element of the UN's 11th Sustainable Development Goal. Monitoring of such areas are important in this regard, and a new approach using a high-resolution "greenspace dynamic data cube" created from Sentinel-2 satellite imagery, addresses the gap in data availability by covering 1,028 major cities globally from 2019 to 2022. The data cube tracks green space changes every 10 days, providing pixel-level data suitable for cross-scale analysis—from parcels to entire cities. It supports urban biodiversity conservation and helps planners manage green space supply-demand dynamics, including urban cooling strategies to counter local warming. However, despite Sentinel-2's high spatial resolution, shadow effects from tall buildings or trees can introduce errors by underestimating green space coverage. To mitigate this, the authors propose integrating high-resolution LiDAR data and 3D models in the future, which can provide more accurate shadow maps, improving greenspace assessments under shaded conditions.
The dataset can be freely accessed via figshare. Also, the code for extracting and displaying greenspace coverage is publicly available here.
Wu, S., Song, Y., An, J., Lin, C., and Chen, B. (2024). High-resolution greenspace dynamic data cube from Sentinel-2 satellites over 1028 global major cities. Scientific Data, 11(1), doi:10.1038/s41597-024-03746-7