Top Banner
NERC Data Grid Helen Snaith and the NDG consortium…
12

NERC Data Grid

Jan 19, 2016

Download

Documents

auryon

NERC Data Grid. Helen Snaith and the NDG consortium …. What is NDG?. NDG provides the infrastructure which allows users to Find data Explore what is known about datasets (including information about the observing or simulating tools) Access, manipulate and visualise data!. NDG Services. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: NERC Data Grid

NERC Data Grid

Helen Snaithand the NDG consortium…

Page 2: NERC Data Grid

What is NDG?

• NDG provides the infrastructure which allows users to◦ Find data◦ Explore what is known about datasets

(including information about the observing or simulating tools)

◦ Access, manipulate and visualise data!

Page 3: NERC Data Grid

NDG Services

• The “NDG Discovery Service”◦ A database of discovery information◦ A web-site, to provide a portal to that database◦ A set of web-services, allowing NDG

consumers to exploit that database

• The “NDG Vocabulary Service”◦ Databases of environmental thesauri and

ontology tools to map between terms◦ Supports “machine assisted” browsing: eg,

search for rainfall and find datasets with precipitation

Page 4: NERC Data Grid

What Can NERC Data Grid Do For You?

• Depends on who you are!◦Data User - eg scientist looking for

data◦Data Provider - eg data centre, large

research group◦Developer - providing access tools or

using data sources

Page 5: NERC Data Grid

As a Data User?

• Discovery leads directly to data◦ Simple search page or “google-like” toolbar

gives direct access to data, not just the metadata

• Comprehensive coverage• Seamless access• Hides data format differences and

provides generic visualisation tools• Enhances interdisciplinary research

Page 6: NERC Data Grid

As a Data Provider?

• Provides a standards based metadata hierarchy◦ Exploit the NDG metadata structures to design a

metadata system from the ground up, or use them as an interoperability tool

• No need to run multiple security systems• Increases visibility and usage of datasets

◦ Inclusion in NDG does not prevent inclusion in other data discovery or access initiatives

• Minimise user support load

NDG Security

Page 7: NERC Data Grid

As a Developer?

• Exploit NDG web services• Exploit NDG modular

development• Exploit NDG metadata standards• NDG developments are non-

proprietary• NDG standards compliance

implies less future development

Page 8: NERC Data Grid

• Discovery◦ ISO standard

• MOLES - Metadata Objects for Links in Environmental Science◦Detailed metadata for browsing

• CSML – Climate Science Markup Language◦ For accessing “features” within data

MetaData Framework

Page 9: NERC Data Grid

CSML in Action

Page 10: NERC Data Grid

What Can’t the NDG Do?

• NDG can’t generate the metadata!◦ Automated manipulation and searching relies on

comprehensive metadata, which require quality data management to create and maintain

◦ NDG does provide a clear framework for metadata requirements

• NDG doesn’t provide information services:- it provides data services!◦ NDG does provide a framework within which it is

possible to link data to derived information services:- research publications, reports, policy etc..

Page 11: NERC Data Grid

Competitors!How is NDG different from the

rest? • e2edm, End-to-End Data Management:

• Similar aims but using different technologies• NDG is more standards compliant

• Thredds:• Provides only catalogue services• Thredds is significantly more mature than NDG• Suitable for data providers with hierarchical data /

metadata

• OPeNDAP:• A protocol for accessing data• Limited support for access control• Requires the user to know exactly what the data are

Page 12: NERC Data Grid

NDG Status

• Approaching Beta◦ Discovery Gateway◦ Discovery Service◦ CSML V2◦ MOLES◦ Security Infrastructure◦ Vocabulary Service◦ MOLES browser

• Population (MOLES & CSML) by NDG partners

• Implementing & testing gateways, browsers & interfaces ndg.nerc.ac.uk

Trac wiki site