Big Data From Space

Last month, the European Space Agency (ESA) in Frascati, Italy, played host to an event titled ‘Big data from space’. Wolfgang Lengert, manager of two separate Earth-observation satellite missions at ESA, gave iSGTW his take on the event:

What was the event about? 

ESA organized the event to address the barriers that hamper the effective use of large volumes of Earth-observation data. The event focused on issues associated with the organization and delivery of large volumes of contemporary and historical Earth observations, either space-based or from the ground (including ubiquitous information-sensing mobile devices, aerial sensory technologies, and wireless sensor networks).

Why is ‘big data from space’ useful?

The global view provided by Earth-observation satellites — together with local data and models — helps us to better understand geophysical or human-caused phenomena. The satellite data are structured, well calibrated, and documented data assets, having the capacity to provide a reference for other data sources (e.g. (A)ATSR measuring sea-surface temperature globally within a precision level of 0.3 degrees) or to create new potential when in synergy with other data (e.g. 20 years of soil moisture data placed in context with epidemic outbreaks). Satellite data within the ‘big data’ paradigm can support both innovation and collaboration, which single and simple data distribution cannot provide.

How, if at all, do the ‘big data’ challenges related to Earth observation differ from those faced by researchers in other scientific fields?

As in other scientific fields, ‘big data’ supports a discovery approach, leading to innovation and inventions. In the Earth-observation field it can be expected that ‘big data’ will help engender more inter-domain research. This will be achieved through the use of tools such as the Geohazard Supersite Exploitation Platform, which is a flagship of Helix Nebula. Here, the space and in-situ community are being enabled to use new data sets, as well as the knowledge of other communities. Also, for climate change, ‘big data’ could help in identifying patterns and correlations. For example, a change of precipitation patterns can impact upon building subsidence, water scarcity and fluvial and pluvial flooding, which then has an impact on pumped drainage, transports and the overall ecosystem.

What then are the barriers that hamper effective use of large volumes of Earth observation data?

In order to exploit  ‘big data’, a cloud is needed. ESA is a co-founder of ‘Helix Nebula - The Science Cloud’ and we believe that only a federated cloud can achieve the maximum contribution and participation of ‘big data’ in science.

What were the outcomes of the conference?

The event concluded with a strong call by all parties for improving the ability to handle and use big Earth-observation data. This could potentially open up new opportunities for research and international cooperation schemes, such as programmatic and industrial coordination.

There was also unanimous support for further developing processing capabilities, and making data more accessible to users, thus complementing more traditional web-service-based approaches. The excellent feedback and contributions received during the event will strongly influence ESA’s plans for managing future Earth-observation data and form the basis for discussion among Earth-observation data owners and suppliers.

What are the biggest challenges that remain and what comes next?

Earth-observation big data will only make sense if we are able to federate all communities involved and to build a data infrastructure that allows analytics and modelling across and between disciplines.

We now need to enable Helix Nebula to become 'the science cloud', as described in the initiative’s strategic plan:

In 2020, all scientists of all disciplines will choose the European Cloud Computing Infrastructure as their first option to store and access data, for data processing and analysis. This infrastructure will be considered as a natural infrastructure for the global science community similar to the road or telecommunication infrastructure for the general public today.

This infrastructure will contain vast quantities of data, an unrivalled array of open source tools, and a literally infinite amount of computing power accessible and usable from any kind of computer, smart phone or tablet device. Science will make significant progresses by applying data sharing and interdisciplinary research using this infrastructure as the fundamental tool. Important articles for leading publications, such as Nature and Science, will be derived from this infrastructure and it will be the source of a drastic increase of patents in Europe

This infrastructure will have such a reliability and worldwide recognition for its implemented security/privacy scheme that also commercial companies will be using this "high security area" to derive patents.

Finally, discussions on ‘big data' will continue at ESA’s Living Planet Symposium in September.

By Andrew Purcel, ©2013 i SGTW