Towards linked data conventions for delivery of environmental data using netCDF LAND AND WATER FLAGSHIP | OCEANS AND ATMOSPHERE FLAGSHIP Jonathan Yu, Nicholas Car, Adam Leadbetter*, Bruce A. Simons, and Simon J.D. Cox CSIRO LWF and BODC/ Marine Institute (Galway) 25 March 2014 | ISESS … netCDF-LD
37
Embed
netCDF-LD - Towards linked data conventions for delivery of environmental data using netCDF
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Towards linked data conventions for delivery of environmental data using netCDF
LAND AND WATER FLAGSHIP | OCEANS AND ATMOSPHERE FLAGSHIP
Jonathan Yu, Nicholas Car, Adam Leadbetter*, Bruce A. Simons, and Simon J.D. Cox CSIRO LWF and BODC/ Marine Institute (Galway)25 March 2014 | ISESS
… netCDF-LD
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
Outline
1. Big data and netCDF
2. Metadata challenges
3. Linked Data, JSON-LD & CSV-on-the-web
4. netCDF-LD
5. Applications
2 |
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
Big data
• “90% of the world’s data has been produced over the last two years”
• Environmental and earth observation has always been big-data…
3 |
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
National Soil and Landscape Grid
4 |
Hydrodynamic models
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu5 |
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu6 |
Real-time wind data
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
netCDF
• Persistence layer• Simple: self-describing, user
accessible “flat file”
• Java, C++, python, others
• Interfaces to the Data Model
7 |
File format
Software library
API
netCDF-4 Data Model
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
netCDF cont’
• Really good at handling array-oriented data – efficient subsetting• Multi-dimensional grids
• CF conventionsclimatology and weather forecasting domains
9 |
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
Use of netCDF for Chlorophyll observations
11 |
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu
netCDF challenges
• Encoding format and framework
• “Domain agnostic” but conventions exist for communities – e.g. Climate and Forecasting (CF conventions)
• CF conventions limited• Use of ‘standard_name’ attribute to denote the combination of
medium/transformation/substance/state/quantity using a particular syntax• Assumes English• Standard names take a while to be adopted• Standard names variations of each other• Units are from UDUNITS – Unidata managed• Narrow scope of scientific domain – can’t reuse for other domains
12 |
Towards linked data conventions for delivery of environmental data using netCDF | Jonathan Yu