Traditional approaches for the analysis of Earth Observation (EO) data typically involve several steps, including data discovery, data download, data pre-processing, and data analysis. Especially when working with multiple datasets, handling data discovery, download, and access is a tremendous task, where users need to navigate through different interfaces, adhere to varying access requirements, and manage the heterogeneity of data formats. This approach is often time-consuming and requires significant effort to aggregate and harmonize datasets from different providers for comprehensive analysis.
Figure: EO research without cloud facilities.
EO data volume and the limits of your computer
In the field of Earth Observation, satellite missions like Sentinel-2 provide vast amounts of data that play a crucial role in various applications, including environmental monitoring, land cover mapping, and climate analysis. Understanding the volume of data involved in an analysis is critical for efficient data processing. EO datasets can span terabytes and petabytes, making it impractical to store, manage, and process them entirely on a local computer.
The increasing availability of vast amounts of EO data from multiple satellites presents challenges in terms of the time required for data download and pre-processing on individual computers or infrastructures. Within the Copernicus program of the European Union, around 64 million products have been published, which sums up to more than 25 Petabyte of data volume. In the following slider we have collected some statistics about the amounts of EO data from the Sentinel satellites.
The following interactive exercise assists in estimating the data volume associated with Sentinel-2 data. This calculator allows users to gain insights into the data volumes involved in specific regions and time ranges, further emphasizing the relevance of using EO platforms for scientific analyses.
Figure: EO data volume and the limits of your computer.
The following interactive exercise assists in estimating the data volume associated with Sentinel-2 data. This calculator allows users to gain insights into the data volumes involved in specific regions and time ranges, further emphasizing the relevance of using EO platforms for scientific analyses. Not convinced of clouds yet? Try the volume calculator below to asses how much space you have to free up on your hard drive for your next project.
Figure: Data Volume Calculator
How can we handle such volumes of data?
Cloud infrastructure and platforms have emerged as viable alternatives to the traditional approach of data analyses as described in Figure 1. These solutions combine data storage and compute resources, enabling users to conduct their data analysis in close proximity to the data itself. By leveraging cloud-based infrastructures, researchers and analysts can optimize their workflow by minimizing the time-consuming steps of data transfer and pre-processing, thereby allowing them to focus more efficiently on data analysis tasks.
By utilizing cloud-based resources, users can harness the scalability and flexibility of these platforms to handle the extensive datasets generated by EO missions. Cloud-based EO platforms represent a paradigm shift in EO data analysis, offering a comprehensive ecosystem that seamlessly integrates storage, processing, analysis tools, collaboration, and visualization. These platforms empower users to overcome the challenges posed by large-scale EO data and accelerate scientific advancements in various fields, including environmental monitoring, climate studies, natural resource management, and disaster response.