Greetings from the Big Geodata Newsletter!
You might have heard about it already, but if not, during these rather unusual Corona days a new initiative has started at ITC: Center of Expertise in Big Geodata Science (CRIB). CRIB aims to be a horizontal facility that serves and supports all our departments in the use of big data technologies in education, research and institutional strengthening activities. In addition to developing a common infrastructure, providing assistance to your projects, and organizing trainings, we also want to keep you informed about the developments in the rapidly changing landscape of big geodata. That's why you are receiving this newsletter!
The Big Geodata Newsletter will provide a quick update on the recent news and developments. We will try to keep it concise, informative and interesting. You will also receive them not frequently, probably one in a month. Please have a quick look at this first edition and if you want to receive more, please simply subscribe here!
Landsat Collection 2 will be available in mid-2020
This may look like an ordinary Landsat 8 scene, but actually it marks an important milestone: 100 millionth download from the USGS Landsat archive on March 9, 2020!
Data acquired since 1972 to present from Landsat 1-8 is one of the largest publicly available Earth observation data sets. USGS is currently conducting a massive reprocessing effort for Landsat Collection 2 to replace the existing collection. It will be available in a cloud optimized file format with updated and consistent metadata, consistent quality assessment bands, global Level-2 auxiliary products, and improved geometric accuracy, digital elevation modeling, and radiometric calibration. The new collection will also improve the interoperability with ESA Sentinel products.
USGS is implementing a hybrid approach for processing and storage of the new collection by utilizing commercial cloud services besides its private infrastructure inline with the Federal Cloud Smart strategy. It is very likely that we will see a similar pattern in other projects in the near future.
Cloud credits for Earth observation projects
Do you know that most of the major cloud service providers offer grants or credits for academic research? Here are two options for big geodata:
Amazon has a specific call for proposals that utilize Earth on AWS datasets. Awards are provided as AWS credits applicable to AWS services. Decisions are typically communicated 30 days following submission.
Microsoft has AI for Earth grants to provide access to Azure resources with credits worth 5,000-15,000 USD. There are also benefits such as technical advice and support, online Azure training materials, and invitations to the AI for Earth Summit for networking and education opportunities. Proposals no longer than 3 pages are reviewed four times a year, the next deadline being July 6, 2020.
We already have an AI for Earth grant holder this year! Dr. Claudio Persello (EOS) and his team will use Azure credits worth of 15,000 USD to map subsistence farms in Africa by using Deep Learning on the platform.
If you have project ideas, but need provider-specific technical support just let us know. We can help!
EUMETSAT phasing in new data services
EUMETSAT with a data catalogue spanning over 35 years of earth observation data is reshaping its data services portfolio by leveraging big data and cloud computing technologies. New services that will be operational later this year aim to facilitate near real-time data access, data visualisation, data transformation, and cloud processing. The first service, Data Tailor, is already available and enables users to subset and aggregate EUMETSAT data products in space and time, filter layers, generate quick looks, project onto new coordinate reference systems, and reformat data into standard GIS formats. In addition to a web-based interface, Data Tailor also supports Python and REST APIs and command line interface (CLI). For more information you can check this presentation or this video!
ITC has extensive experience with EUMETSAT products and also developed GEONETCast Toolbox plug-in for ILWIS to make it easier to import products disseminated via GEONETCast.
Did you know that we also operate a GEONETCast reception station?
Do you have 5 minutes?
To better support you, we would like to know more about your current and future needs. For this purpose, we prepared a short anonymous survey which can be completed in 5 minutes.
Similar surveys were sent to the participants of the ITC DS & AI Workshop in November and PhD students in March. Overall we had 36 replies and we seek further contributions to get a more complete and representative view. If you didn't participate already, here is the link for you.
Thanks!
cuSpatial: CUDA-accelerated GIS and spatiotemportal algorithms
Nowadays it is quite common to use GPUs to accelerate machine learning tasks, but they can also be useful for other tasks (besides gaming!). cuSpatial library provides significant GPU acceleration to common spatial and spatiotemporal operations, such as point-in-polygon tests, distance calculations, and trajectory clustering. Reported speed-ups are enormous, e.g. up to 114,000x for Haversine distance computation for 14+ million points. More advanced operations, such as point-to-polygon nearest neighbor and spatial indices, are planned for future releases.
Recent Major Releases
- RasterFrames: Spark DataFrame-centric view over geospatial raster data
0.9.0 (2020/04/24) - CuPY: NumPy-compatible matrix library accelerated by CUDA
8.0.0 (2020/04/23) - PyTorch: Open source machine learning library
1.5.0 (2020/04/20) - GeoTrellis: Distributed geographic data processing engine
3.3.0 (2020/04/07)
Enjoy the "Big" Picture
Bolton et al. (2020) developed an algorithm to create high quality time series of vegetation indices from harmonized Landsat 8 and Sentinel-2 (HLS) imagery and used them to estimate the timing of continental-scale vegetation phenophase transitions at 30 m spatial resolution. Because of the very large data volumes, all data processing was performed on Amazon Web Services (AWS).