CAMS GA Data Access by Razinger

Copernicus

Atmosphere

Monitoring

Service

CAMS General Assembly, Athens, 14-16 June 2016

Miha Razinger (ECMWF)

Accessing Data and

Products

CAMS Datasets in Numbers

Combined volume of datasets 300 TB

180 individual products

33 distinct atmospheric parameters

9 data servers

2

3

The challenge

Diversity of data producers, variety of data formats and

dataset volumes, existing data serving methods

4

global surface ozone data June 2010

What products and

services are available

from CAMS?

What is the quality

of a product?

How can I access the

data?

Is a product suitable

for operational use?

Can I use a product

in our commercial

service?

How can I read and

interpret the data?

it depends

it depends it depends

it depends

© Copyright Showeet.com

single entry point

single data licence

Questions from a potential user

6

CAMS product catalogue

• Comprehensive and up-to-date inventory of

CAMS products

• 180 individual products and services

accompanied with description, details about

geographical and temporal coverage, data

access links, previews, documentation,

validation reports, contact details ...

• Interoperable: INSPIRE and WMO Core Profile

compliant meta data

• Can be harvested through OGC Catalogue

Service for the Web (CSW) (implemented using

pycsw library)

http://atmosphere.copernicus.eu/catalogue

New catalogue BETA

Product catalogue

FTP servers

WCS

WMS

Regionaldata

servers

Solar radiationservices

ECMWF data

servers

ECPDS

service

Distributed data access

Accessing global data

1. New map application

2. ECMWF data servers

3. ECPDS data delivery

New interactive map application

• OGC WMS and Leaflet library

• Fast development and publishing of maps

• Maps can be composed by various WMS sources

• Global or regional views, panning, zoom,

fullscreen view

• Point forecast chart generation from the maps

• Regional forecasts, observations

• Take it for a spin and send us your feedback at

[email protected]

10

BETA

http://atmosphere.copernicus.eu/maps/global-surface-forecasts

http://atmosphere.copernicus.eu/maps/global-total-column-forecasts

http://atmosphere.copernicus.eu/maps/aerosol-forecasts

http://atmosphere.copernicus.eu/maps/fire-activity

12

Iztočnice

• - catalogue as main entry point, distributed data servers

• - find, preview, access

• - use cases

• - compo announcement: service changes, timing, schedule, data

volume

• - data usage numbers and trends

• - DOIs

• - direct feedback but another survey might be useful

• - data volumes and disk write speed limitations: subsetting, slicing,

extracting - mogoče za na konec

3 March 2015

13

Delivering global model results

• Hosting the global real-time analyses and forecasts, MACC

reanalysis, GFAS and GHG flux inversions datasets

• Very capable of serving big volumes of data

• Services build on top of existing ECMWF systems

• Most of the data is archived in MARS, some on disk

• Not best suitable for data browsing and occasional / light-

weight data usage

ECPDS FTP data service

• Serves global model, boundary

conditions and GFAS data

streams

• FTP-push and –pull service

• Suitable for large data transfers

• 24/7 monitored and supported

• Highly available, load balanced,

scalable service

• User Account Management

16

Global model upgrade

• On 21 June we are going to upgrade the model version

• New horizontal resolution (from about 80 km to about

40 km)

• Two forecast runs per day (00 run available at 10 UTC

and 12 run at 22 UTC)

• 24/7 monitoring

• Test data available

Future plans

• Better data subsetting and slicing services (custom plots,

subareas, time series, vertical profiles ...)

• Exposing catalogue to web crawlers

• Dataset DOIs – improved data citation, better usage

tracking

• Follow CCCS Climate Data Store development