The role of the ISIC facility for Climate and Environmental Monitoring from Space (CEMS) in the development of Quality Assured Datasets and Downstream.

The role of the ISIC facility for Climate and Environmental Monitoring from Space (CEMS) in the development of

Quality Assured Datasets and Downstream Services from Earth

Observation Data

Sam Almond, Logica &

Kevin Talsall, Telespazio Vegahttp://isic-space.com/cems

2

Contents

1. What is CEMS?

2. Why is CEMS needed?

3. What CEMS offers?

4. Overall benefit to the UK

5. How CEMS enables Quality Assurance (QA)

6. CEMS QA / Data Integrity projects

1. TruDAT

2. Data Integrity Pilot Project

7. Looking forward

8. Conclusions

12 September 2012 RSPSoc 2012

3

1. Overview of CEMS

12 September 2012 RSPSoc 2012

• CEMS is an ISIC facility to enable innovative climate and environmental services to be developed, hosted and served to users- These services will be for widespread use by public and private sector

organisations in UK and internationally, supporting excellent science and generating commercial revenues

- CEMS will consolidate the UK’s leadership position in climate science, calibration/validation & downstream applications, and will position the UK for key opportunities e.g. ESA CCI, GMES climate services

• What is unique about CEMS?- It will make the data from satellites, ground and airborne sources

accessible in native & common formats- It will provide tools and information to ensure the integrity of the data

– science and commercial users will be able to rely on the information - It will provide a scalable model by hosting services and applications

close to high performance computing facilities and very large quantities of source data

4

2. Why do we need CEMS?

12 September 2012 RSPSoc 2012

• Climate and global environmental change monitoring from space is an increasingly exciting yet challenging task, leading to potential benefits in science understanding of our planet, as well as commercial opportunities for UK companies.

• The available volume of high-quality Earth observation (EO) data is increasing exponentially, as new satellites are being developed globally from both public and private funds.

• These observations must be processed, quality controlled and managed prior to visualisation and analysis, both independently and in conjunction with state-of-the-art model predictions.

• Leading-edge data processing, management and dissemination capability at the ISIC Facility in Harwell will enable the UK to maximise the value of these data and create economic benefit across a wide spectrum of UK interests.

5

3. What CEMS offers

At its heart CEMS is a facility that provides:

• High-powered processing and data storage:

• Processing: 240 real cores (480 virtual cores) & 2 Terabytes (2000 Gigabytes) of memory

• Storage: 1.7 Petabytes (1.7 Million Gigabytes)

• Accessible through Virtual Machines allowing access from anywhere

• Data processing: Users can process large amounts of data efficiently & cheaply

• Application hosting: organisations can take advantage of CEMS processing and storage capacity and use CEMS to reach a wider audience for their applications

• Data Integrity Facility: provides key QA information to help users determine if their data is fit for purpose

• Consultancy: access to the wide-ranging skills of the ISIC partners

• Visualisation and Application Centre: utilise existing ISIC facilities such as the Applications Innovations Centre and the Visualisation Suite

12 September 2012 RSPSoc 2012

3. CEMS Technical Implementation

Public and Commercial Cloud Infrastructure

Hardware – data storage [NCEO and Commercial Data] and processing

App 2 App 3 App NApp 1…

Business and researchuser communities

Data Access Quality Services

Cor

e S

ervi

ces

App

licat

ions

Clo

ud M

anag

emen

t S

ervi

ces

Data Processing

3. The CEMS Overview

8

4. Overall benefits to the UK

• The implementation of CEMS through ISIC enables the infrastructure to act as a platform for science and commerce to develop further activities which will bring economic benefit to the UK.

• In science, CEMS will support evidence on how Earth system components are changing, through the provision of long-term, quality-controlled data sets for climate and geo-hazards research.

- In commerce, the CEMS core infrastructure will provide the means to develop commercial applications with a focus on UK sectoral strengths:- Financial markets (carbon trading) - informed use of climate and

carbon data - potential revenues/benefits throughout the value chain, rising from £1-2M in 2013/14 to more than £100 million p.a. by 2016.

- Insurance/risk markets – modelling/services using improved knowledge of environmental data; risk/hazard forecasting, etc.

- Environment/energy markets – use of UK and global datasets to inform decision making and forecasting

- Services relevant for many public sector users e.g. Defra, DECC, etc.

9

5. How CEMS enables QA in EO

• QA is important to enable users to make judgements on the quality, provenance and suitability of any potential datasets

• The Data Integrity study recommends that CEMS provides:

• Adopt a DI framework based on QA4EO principles

• A DI Helpdesk and Consultancy Service to assist users in optimally utilising EO and climate data

• A Certification Service to allow for quality certification of data

• The ability to create bespoke tools to enable the completion of ‘missing’ data integrity information not provided by data providers.

• User access to such tools to enable users to perform their own comparative validation exercises of data within CEMS

12 September 2012 RSPSoc 2012

10

5. The CEMS QA / DI process

12 September 2012 RSPSoc 2012

11

6. CEMS QA / Data Integrity projects

12 September 2012 RSPSoc 2012

CEMS has already successfully bid for QA / DIF projects : TruDat – led by NPL: Detailed data integrity procedures will be

developed using optical & SAR Data Integrity Pilot Project – led by Vega: Use SST data as a

basis to produce a prototype added value product and pilot data integrity service.

HIVACROM – led by Logica: Crop monitoring service using EO data to accurately estimate crop yields (initially potato & sugar beet)

CHARM (Characterisation of metadata) (FP7) - Enable climate-relevant data to be consistently characterised as to their provenance, errors, biases and consistency; develop tools to enable users to access information to determine dataset fitness of purpose.

12

6. TruDAT

• TRUsted DATa for Carbon, Science and Finance• TSB project lead by NPL, partnered by University of Sheffield & Logica• Objective is to provide a quality develop strategy / methodology to establish

rigorous Quality Indicators (QI based on QA4EO) on CEMS delivered EO data/products

12 September 2012 RSPSoc 2012

• Forest Carbon Case study • QI used to enable harmonious,

consistent and seamless choice of data sources (Optical and SAR), processing steps and validation data

• Prototype Automation of process within CEMS and also provide visualisation tools

• Applications include carbon trading, validating climate data and low carbon technologies

13

6. Data Integrity Pilot Project

• TSB project lead by Telespazio Vega, partnered by NCEO & Logica• Objective is to create a pilot demonstrator of a Data Integrity Facility within

CEMS using a sample dataset of ESA’s Envisat Advanced Along Track Scanning Radiometer (AATSR) data.

• It will address 3 key Data integrity recommendations:

12 September 2012 RSPSoc 2012

1. Establish a Helpdesk with capability to answer user questions related to the DI data provided with the dataset

2. Establish a DI Certification process through an appropriate metadata schema, by which the sample dataset may be assessed

3. Develop an environment and associated Tools specifically developed for assessing the AATSR data and the output information fed into the DI certification process

14

7. Looking forward: CEMS strategic goals

12 September 2012 RSPSoc 2012

Integrated Delivery of Climate Data• Major processing & delivery node of timely, robust and synthesised satellite

information products from ESA’s CCI• CEMS platform/infrastructure to help enable the UK to become the world

leader in climate science, validation, adaptation, mitigation and commercial climate services

GMES Service Provision• CEMS to be a leading European node for Sentinel data and services, initially

ensuring the GMES Sentinel 1 & 2 data supports a unique set of tools and services.

Stimulating Commercial Applications• ISIC & CEMS to provide the open innovation environment and infrastructure to

stimulate the development of a vast array of commercially hosted applications Next Generation Visualisation Facilities• Enable users to visualise a wealth of data sets using state-of-the-art extraction

and animation tools to monitor the health of our oceans, validate carbon assets and inspire our next generation of young scientists.

15

8. Conclusions

• CEMS provides a unique facility to enable innovative climate and environmental services to be developed, hosted and served to users

• It positions the UK to take a leadership role in key UK and international opportunities in the climate and environmental services domain

• CEMS will help foster growth in the downstream applications that are predicted to increase with availability of vast quantities of EO data

• A key principle of CEMS is focused on the Quality Assurance and Data Integrity of EO and climate data, allowing users to make informed choices about their data

• CEMS has already begun providing additional QA data, as can be seen by the QA / DIF projects started

12 September 2012 RSPSoc 2012

16

Questions