Photo credit  |  Edgar Nunley on Unsplash

How to make the most of Earth Observation data and not perish in the attempt

Blog  |  10 March, 2021  |  Reading time: 7 minutes
Gerardo Lopez-Saldana has spent more than two decades studying, researching and working in Earth observation. He uses EO data together with models to improve the understanding of land surface processes and to translate this knowledge into real world applications. He part of Space4Climate’s Peatlands Monitoring and Climate Action Task Group and compiled the fascinating UK Mosaic for Assimila. Here he shares his passion for removing barriers to the use of EO data.

The iconic image below provides a previously unseen perspective of our planet, it was one of the first opportunities to see Earth from space and, in this particular picture, to see cloud patterns at a planetary scale.

This unique perspective of Earth is why Earth Observation (EO) data is so important today. The image was taken by one of the five NASA Lunar Orbiters while they were mapping the Moon for potential Apollo landing sites, between 1966 and 1967.

This was the beginning of a fascination with data about our planet gathered from space.

Figure 1: This image, produced by the Lunar Orbiter Image Recovery Project, (LOIRP) in 1966-7 is a picture worth more than thousand words. Image Credit: NASA / LOIRP

Technology has evolved massively since the grainy monotone Lunar Orbiter images of the sixties. Over time, multispectral sensors, dedicated to observing the Earth from satellites, have been developed. The Landsat programme is the longest-running effort for acquisition of images of the Earth. The first Landsat (originally named Earth Resources Technology Satellite) was launched in 1972. Since then, there have been several generations of these satellites hosting better and better instruments. Landsat-8 is the current operational satellite, the next will be Landsat-9, scheduled for launch in September 2021.

Figure 2: This Landsat-7 image shows Mexico City in so-called true colour, meaning that the images used the red, green and blue channels to mimic what the human eye would see. Image Credit: US Geological Survey

If images say more than a thousand words, images that actually represent data then say a whole story. An example of the Assimila Data Cube use is the S4C Peatlands project. The main goal of the project was to gather EO data to support climate actions related to peatlands. We compiled more than 14,600 individual files to create a daily albedo (showing the extent to which a surface reflects or absorbs solar energy) UK mosaic derived from MODIS data, then with literally fewer than five clicks you can get the whole data record for a single location in the UK – feel free to take a look to the Jupyter Notebooks developed for this project.

Consistency holds the key to the value of satellite data

The need for consistency, as offered by the Landsat series, is a powerful driver of investment and innovation in EO. Consistent observations are required to analyse any environmental phenomena. If we want to characterise the land surface or the ocean currents or sea ice extent, we must be able to rely on observations that are gathered in an organised and systematic manner and offer reliable quality. That is one of the main reasons EO programmes exist, for instance Copernicus – Europe’s EO programme – which is based on three components: space, in-situ observations and services.

One of the main missions within the space component is Sentinel which has several satellites in operation, for instance Sentinel-2’s main goal is land monitoring, its first satellite, Sentinel-2A, was launched in June 2015 and Sentinel-2B was launched to follow in its path in 2017. Sentinel-3 is dedicated to marine observations and Sentinel-5P to air quality monitoring. The key element here is that observations from these satellites can be used together – it presents some interesting cross-calibration challenges, but it is possible to do it. For example, we can combine data from Landsat-8 and Sentinel-2 to monitor vegetation health.

This leads to another reason to use EO data – we can, and must, consider the uncertainties in the observations. EO is way more complex than using Google Maps to have a look around our local town! It provides so much more than beautiful pictures. The images that we are becoming familiar with as pictures of Earth taken from space, are in fact, just a pictographic representation of some physical units.

Figure 3: This Sentinel-2 image of Guildford, Surrey, and the surrounding area provides the surface reflectance – the proportion of energy reflected by the land surface after removing atmospheric effects. It is a false colour composite and one of a series that I produced to build up a whole UK mosaic for Assimila using all images available during Spring 2020. I used the short-wave infrared, the red and the green to enhance vegetation features. Image Credit/Source: Assimila visualisations.

EO data is freely available, but how easy it is to use?

And you know what the best thing is? All these amazing images, all these data, everything is freely available. You can download old Landsat images from the seventies or the latest Sentinel-2 from literally a few hours after the satellite overpass, all for free.

Now that you have images that provide a full planetary perspective, consistent observations and (hopefully) a way to assess the quality those observations, how do you use these vast amounts of data? Sadly, there is no straight forward solution, no one-size-fits-all tool to discover, access, process and analyse EO data. What we need is a toolbox, a way to:

  • discover and get access to the data
  • have access to continually updated catalogue of the variables of interest
  • remove the barrier of needing to perform an uncertainty and quality assessment (QA)
  • help perform the analysis required.

The Assimila Data Cube was born from the need to do all of that, to avoid ending up storing a copy of the same image 20 times in different locations; to avoid the need to rewrite a script to perform a QA analysis for similar EO products just changing a few parameters here and there; to be able to create prototypes in an easy way; to handle images in different spatial resolutions; to save data in different formats… etc.

The Assimila Data Cube

So, how can you make the most of EO data and not perish in the attempt? This is a challenge we addressed at Assimila by developing the Data Cube. You can access a wide range of EO and reanalysis datasets such as MODIS and VIIRS data via the LP-DAAC, ERA5 on the Climate Data Store (CDS) and Sentinel-2 on the Open Access Hub. You can create a catalogue, download or access the data in a cloud environment. It is simple to apply all the QA flags needed or associated uncertainties.

There is no need to worry about different spatial reference systems, the Data Cube will handle them for you! How do you need your data? Simple text file with comma-separated values (CSVs) for single point location (just because you want it!)? Yes, Data Cube can do that.  You want a Cloud Optimized GeiTiff (COG)? Yes, Data Cube can do that.  You speak weather and climate data and you want all your analysis results on a NetCDF file? Yes, Data Cube can do that too!

It offers ‘have it your way’ EO data. And last but not least, if you speak Python, you can use a powerful API to access the data. If you prefer a simple Jupyter Notebook interface, Data Cube has that well. You can visualise data and create plots from your web browser. If you prefer a GIS system, there is a QGIS plugin as well.

So while there is no complete one-stop shop solution for accessing and processing EO data, at Assimila we have created a tool that works for our science and product development. We have been making the most of EO data products for more than a decade and we have not perished in the attempt – not yet.

Figure 4: The core tasks that you can perform in the Assimila Data Cube.