Project number: 283465 Data and Data Requirements Wouter Los University of Amsterdam.

Project number: 283465

Data and Data Requirements

Wouter LosUniversity of Amsterdam

Project number: 283465

Environmental Science

oceanic and atmospheric

processes

long-term development of the

climate system

Biological processesbiodiversity

development of the cryosphere and

lithosphere

Earth as a single complex and coupled system

23/10/12 W. Los - ENVRI @ EUDAT 2

Project number: 283465

ESFRI Environmental Research Infrastructures

• Tropospheric research aircraft

COPAL

• Upgrade of incoherent SCATter facility

EISCAT-3D

• Multidisciplinary seafloor observatory

EMSO

• Plate observing system

EPOS

• Global ocean observing infrastructure

EURO-ARGO

• Aircraft for global observing system

IAGOS

• Integrated carbon observation system

ICOS

• Biodiversity and ecosystem research infra

LIFEWATCH

• Svalbard arctic Earth observing system

SIOS

23/10/2012

W. Losi - ENVRI @ EUDAT 3

Project number: 283465

23/10//12 -ENVRI @ EUDAT 4

Project number: 283465

29/03/12 Pasquale Pagano - ENVRI @ EGI CF 2012 6

Radar interference dataGas (CO2 etc) fluxes

∂ (concentration)

Areal andsatelliteobservation

Species data, distributions, abundance, biomass, etc.

Observations, sensor data,collection data, DNA, etcMarine

sensors

Currents, salinity,deposition, etc

Platetectonics

Seismic data,satellite data,sensors, etc

Project number: 283465

Goal

Enable multidisciplinary scientists to access and study data from multiple domains for “system level” research

by providing solutions and guidelines for the RIs common needs

Multiple data producersMultiple data consumers

723/10/12 W. Los- ENVRI @ EUDAT

Project number: 283465

8

Approach

discover data which are heterogeneous in format, content, and metadata description

harmonise, integrate and analyse data across domains and RIs

Prom

ote

Acce

ssib

ility

Preserve Specificity

23/10/12 W. Los - ENVRI @ EUDAT

PROVIDE SOFTWARE TOOLS TO

Project number: 283465

Data infrastructure requirements

Integrated data discovery across various catalogues

(Near) real-time data handling

Federation over infrastructures/services

Persistent identifiers mechanism

Metadata definition and assignment

Attribution / crediting author/ownership

Quality control of data

Provenance and preservation

Archiving vs. regeneration of data and/or results (processed data)

Single sign-on, delegated authorisation

Running complex models

Data staging or moving computation to data

23/10/12 W. Los- ENVRI @ EUDAT 9

Project number: 283465

First steps - priority areas

Integrated data discovery across various centres / catalogues

(near) Real-time data handling

Federation over existing (national or international) infrastructures / services

23/10/12 W. Los - ENVRI @ EUDAT 10

Project number: 283465

1: Integrated data discovery

Integrated data discovery across various centres / catalogues

The challenge of being able to easily discover data which are heterogeneous (in format, content, and metadata description) and which are stored at different placesENVRI partners ESA, CNR, UvA and CSC are tackling thisDoes EUDAT see a role to contribute?

Project number: 283465

Geospatial Data Services

Geospatial Repositories

Data Discovery

Data Access Data Process

OGCOpenSearch

Linked Open DataCatalogue Services

OGCWCS

THREDDS

OGCWPS

WPS 52N

P1 P2 P..

WPS Hadoop

Hadoop Cluster

HDFS

Data Pub. /Vis.

OGCWMS, WFS

GeoServer

gCub

e D

ata

stag

ing

by courtesy of P. Pagano

Project number: 283465

2: (Near) Real Time Data Handling

(near) Real-time data handlingThe challenge(s) of being able to handle real-time data Challenges include:

collecting, storing and cataloguing data as it arrives in real-time from sensorsprocessing data into derived data products in real-time analysing data in real-time

It was suggested that EUDAT might take this up

Project number: 283465

3: Federation

Federation over existing (national and international infrastructures / services

The challenge of bringing together existing infrastructure components / services as contributions to the construction of an Research InfrastructureThe challenge of bringing about interoperability (syntactic and semantic) between separately owned and operated facilities that each contribute to the Research InfrastructureIs EUDAT also interested?

Project number: 283465

Data infrastructure requirements

Integrated data discovery across various catalogues

(Near) real-time data handling

Federation over infrastructures/services

Persistent identifiers mechanism

Metadata definition and assignment

Attribution / crediting author/ownership

Quality control of data

Provenance and preservation

Archiving vs. regeneration of data and/or results (processed data)

Single sign-on, delegated authorisation

Running complex models

Data staging or moving computation to data

23/10/12 W. Los- ENVRI @ EUDAT 15

Project number: 283465

Need a Common Data Infrastructure

Managing the growing amount of data

Improving interoperability between infrastructures and

across disciplines

Promoting collaboration and clarifying roles and

responsibilities

23/10/12 W. Los- ENVRI @ EUDAT 16

EUDAT contributions to the ENVRI consortium is welcome!

Project number: 283465

Thank you

http://envri.eu/

23/10/2012

W. Los – ENVRI @ EUDAT 17