WHAT IS THE OPEN DATA CUBE?
The Open Data Cube (ODC) is an open source solution for accessing, managing, and analyzing large quantities of Geographic Information System (GIS) data - namely Earth observation (EO) data. It presents a common analytical framework composed of a series of data structures and tools which facilitate the organization and analysis of large gridded data collections.
The Open Data Cube was developed for the analysis of temporally-rich earth observation data, however the flexibility of the platform also allows other gridded data collections to be included and analyzed. Such data may include elevation models, geophysical grids, interpolated surfaces and model outputs.
A key characteristic of the Open Data Cube is that every unique observation is kept, which contrasts with many other methods used to handle large gridded data collections. Some of the major advantages of ODC are the following:
User maintains control and ownership over their data
Paradigm shift from scene-based analysis to pixel based
Lower barrier to entry for remote sensing data analysis.
In our technical overview document, we briefly describe and illustrate the high-level architecture and ecosystem of the ODC framework in order to provide a better understanding to those who are new to ODC. This document only covers major components of the ODC and the relationships between them. Find it here>>
What can it do?
Project Open Data Cube (ODC) is an open source project born out of the need to better manage Satellite Data. It provides the foundation of several international, regional to national scale data architecture solutions, such as Digital Earth Australia, Africa Regional Data Cube, and others. The Data Cube works well with Analysis Ready Data (ARD), pre-processed, ready to use data made available by data providers. While providers work on making global ARD products available on the cloud, the Data Cube typically uses the USGS collection 1 Landsat 8 PDS for demonstrations. These data are not ARD and should not be used for scientific analysis. In 2019, it is expected ARD data will become easily available on the Cloud, and until then a user can simply add and index their own processed data. Any data available to you can be installed to your cube, including commercial, in situ, or derived products.
The Open Data Cube system is designed to:
- Catalogue large amounts of Earth Observation data
- Provide a Python based API for high performance querying and data access
- Give scientists and other users easy ability to perform Exploratory Data Analysis
- Allow scalable continent scale processing of the stored data
- Track the provenance of all the contained data to allow for quality control and updates
It has evolved to support interactive data science and scientific computing. ODC will always be 100% open source software, free for all to use and released under the liberal terms of the Apache 2.0 license.
GEOSPATIAL DATA MANAGEMENT & ANALYSIS SOFTWARE
Depending on your application, the Open Data Cube can be deployed on HPC, Cloud, and local installations. Typical installations run on Linux, MacOS, and Windows.
Accessing the data in your Data Cube
The ODC core serves as a layer between satellite data providers and applications. A set of open source tools exist to help scientists conduct research using data managed by the ODC. Popular tools used within the community that utilizes the ODC Core as its basis include:
Command Line Tools: A tool used by programmers/developers to interface with the ODC.
Open Data Cube Explorer: A visual and interactive web application that lets users explore their inventory of available data.
Open Data Cube Stats: An optimized means of defining and executing advanced analysis on ODC system. This tool is oriented towards scientists.
Web User Interface (UI): A web application that allows developers to interactively showcase and visualize the output of algorithms.
Jupyter Notebooks: Research documents centered around techniques in EO sciences. A notebook contains executable code detailing examples of how the data cube is used in a research setting, and therefore is an invaluable reference material for new users.
Open Geospatial Consortium (OGC) Web Services: Adapters that can connect non-ODC applications to the ODC.
Satellite Data Providers
Command Line Tools
Links to the GitHub repository
ODC ALGORITHMS & APPLICATIONS
The ODC supports a broad range of applications including land, water, cloud, and time series analysis:
Mosaic Creation - A mosaic is a composite image created by combining the most appropriate pixels from a collection of source images. A common use case is to create cloud-free images for applications that are not time-dependent. There are a number of approaches.
Spectral Products - Spectral Index algorithms calculate the relative magnitudes of wavelength components. The particular wavelength components used determine the product calculated.Common Spectral Indices include NDVI, NDBI, NDWI, and products such as Fractional Cover.
Water Mapping - Water Observations from Space (WOfS) - WOfS is an automated water mapping algorithm created by Geoscience Australia. Surface water is detected in satellite images and for each location the number of occurrences is summed through time. The result is a percentage value of the number of times water was observed at the location.
Land Classification and Land Change
Cloud Statistics - Cloud statistics are valuable information for performing analyses. For example, if there are extensive clouds during a season, it may significantly impact mosaic products or index values. Users may also want a way to find dates when there are few clouds, so that they may assess land features.
Time Series Analysis