The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 1 New Internet New Internet Developments: Marine Developments: Marine XML XML Peter Pissierssens & Greg Reed (IOC/IODE)
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 1
New Internet Developments: New Internet Developments: Marine XMLMarine XML
Peter Pissierssens & Greg Reed (IOC/IODE)
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 2
SummarySummary
• Background
• Why MarineXML?
• IODE and MarineXML
• Benefits for marine community
• Current initiatives
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 3
BackgroundBackground• Emergence of XML as a data transfer
protocol provides the global oceanographic community with a mechanism to support the exchange of marine data.
• A marine XML structure will encapsulate marine data and provide an efficient means to store, transfer and display marine data.
• XML: describes content, not style
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 4
Why marineXML?Why marineXML?
• Exchange of data
• Extensibility
• Plain Text
• Data Identification
• Stylability
• Hierarchical
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 5
IODE and marineXMLIODE and marineXML• 16th Session of the IOC Committee on
International Oceanographic Data and Information Exchange (IODE) in 2000 proposed to develop an XML interoperability framework based on a ‘Marine Mark-Up Language’ (MML) specification
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 6
Benefits for Ocean CommunityBenefits for Ocean Community• MML interoperability framework will
have considerable benefits to the oceanographic community, supporting the objectives of data interoperability
– Data providers can operate without changing their existing data structures and systems
– Developers can build common programs and instruments to support multiple formats
– Users can access heterogeneous datasets across the internet or local networks
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 7
Current InitiativesCurrent Initiatives• Current international initiatives
include:– The ICES/IOC SG-XML– XML ‘Brick’ concept– The EU Marine XML Project– The MEDI metadata authoring tool– GE-TADE activities– Establishment of a MarineXML community
portal site – OIT Pilot Project
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 8
SG-XMLSG-XML
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 9
SG-XMLSG-XML• Study Group on the Development of
Marine Data Exchange Systems Using XML (SG-XML)
• Jointly established by ICES-IOC
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 10
SG-XMLSG-XML TOR TORThe Terms of Reference for the Group:a) develop a framework and methodology for the use of XML in marine
data exchange in close consultation with IOC and the Marine XML Consortium;
b) develop a workplan that within 4 years will lead to published protocols for XML use in the marine community;
c) explore how to best define XML tags and structures so that many ocean data types can be represented using a common set of tags and structures;
d) test and refine these common tags and structures using designated case studies i.e.:
a) Point (physical/chemical) data (profile, underway, water sample); b) Metadata (cruise information, building from the ROSCOP/Cruise
Summary Report); c) Marine Biology data (integrated tows, e.g., zooplankton-
phytoplankton tows, demonstrate the use of taxonomy).
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 11
SG-XMLSG-XML actions actions• First meeting April 2002• Action plan developed to include:
– Parameter Dictionary. Create, evaluate and discuss intersessional work on SGXML parameter dictionary including the population of the dictionary for distribution via a defined XML structure.
– Point Data Investigation. Evaluate the generalised brick approach with application to various point data types. Evaluate the point data structure from the perspective of the IODE data centres. Provide biological and taxonomic input to the bricks.
– Metadata Investigation. Evaluate linkages to other metadata standards and the implications of a generalised metadata model to existing models.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 12
XML BricksXML Bricks
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 13
XML BricksXML Bricks• Concept developed by R. Keeley of
MEDS• The brick structure represents a basic
building block for packaging data and metadata
• Employs a small number of generally defined structures that can be assembled in different ways to reflect the structures of a variety of data collected over a broad spectrum of disciplines.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 14
XML BricksXML Bricks• Bricks can describe
– data units– measured (or computed) variables– space and time location of the
measurement– data quality flags– supporting information such as sampling
techniques, species taxonomy, etc.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 15
XML Bricks – current listXML Bricks – current listBrick Definition
analysis_method Information about the analysis employed
archive_information Information attached by the recipient of the data
availability A marker to control access to the data
calibration Information about instrument calibration
comment General comments on the data
data_dictionary Used to identify the dictionary in use in the data file
data_point Contains a single data value and supporting information
history Processing history of the data
instrument Information about the instrument used to make the measurements.
location The x, y, z, or t values for a given set of observations.
provenance The originator of the data
quality A marker providing the assessment of data quality
quality_testing Information about how the data quality assessment was made
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 16
XML BricksXML Bricks: Example: Example• Example: the Provenance Brick• Holds information about the originator of the
data. • Where data is collected from a single source,
would take the form: provenance
> originator > project > country > date_received > platform_name > agency > originator_identifier > data_grouping > description
> platform_type
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 17
EU MarineXML ProjectEU MarineXML Project
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 18
EU MarineXML ProjectEU MarineXML Project
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 19
Project OverviewProject Overview• To demonstrate that XML technology
can be used to develop a framework that improves the interoperability of data for the marine community and specifically in support of marine observing systems
• To develop a prototype of an XML-based Marine Mark-up Language (MML)
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 20
Partnerships!Partnerships!• MarineXML is to be developed in
partnership with international agencies, government departments and organisations responsible for data standards to ensure that the research meets the needs of key stakeholders with interests in global ocean observing systems.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 21
Consortium ParticipantsConsortium Participants1. HRW HR Wallingford UK (Coordinator)2. UKMIC UK Marine Information Council UK 3. 7CS SevenCs DE4. NERSC Nansen Environmental and Remote Sensing
Centre NO 5. CLRC Central Laboratory of the Research Council UK 6. RIKZ Rijkswaterstaat NL 7. VLIZ Flemish Marine Institute BE8. SCO Social-change On-line AU 9. SMHI Swedish Metrological and Hydrological Institute
(EuroGOOS) SE 10. IOC/IODE International Oceanographic Data and
Information Exchange Committee INT
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 22
Other InitiativesOther Initiatives
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 23
MEDIMEDI
• Marine Environmental Data Inventory• A directory system for marine related
datasets and data inventories within the framework of the IOC’s International Oceanographic Data and Information Exchange (IODE) programme.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 24
MEDI Authoring ToolMEDI Authoring Tool• The MEDI authoring tool has been
developed to encourage data collectors and scientists to produce metadata descriptions for their datasets.
• MEDI authoring tool is browser-enabled and operates in a client-server configuration. Clients can access MEDI on a local network or over the internet
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 25
MEDI formatMEDI format• MEDI uses the Directory Interchange
Format (DIF) developed by NASA's Global Change Master Directory
• A DIF is a collection of fields which detail specific information about the data which are necessary for users to decide whether a particular dataset would be useful for their needs
• Metadata are stored as DIF-XML files• XML is used to transfer DIF records
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 26
MEDI flowMEDI flow• Operates as a service
under Apache Tomcat 4.0.4 using HTML
• Uses standard HTTP protocol, hence can be accessed via internet or intranet
• Metadata records are stored as DIF-XML files
• Spatial functionality is delivered using Scalable Vector Graphics (SVG)
• Data can be tranferred using XML
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 27
GE-TADEGE-TADE• IODE Group of Experts on Technical
Aspects of Data Exchange• Mission of its work plan:
“The development of an End-to-End Marine Data Management Framework”
• Present situation:– large number of data formats in existence– many different data types– many different systems – result is a high level of duplication of effort.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 28
GE-TADE and MarineXMLGE-TADE and MarineXML• Objective is the development of marine
XML as a mechanism to facilitate format and platform independent information, metadata and data exchange.
• A limited pilot project to demonstrate an activity within the E2EDM framework is currently in progress at the Russian NODC.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 29
Marine XML PortalMarine XML Portal• Established to
provide a discussion forum for MarineXML activities.
• Hosted by IOC
http://www.marinexml.net
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 30
Marine XML PortalMarine XML Portal• News• About marineXML• EU MarineXML
Project• ICES-IOC SGXML• Events• Documents• XML Links• Forum• FAQ
• Uses IODE BeeBox:– You can register– Content informer– Multiple writers
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 31
OIT-PPOIT-PP
• Ocean Information Technology Pilot Project
• JCOMM / GOOS / IODE
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 32
OIT: Why?OIT: Why?• We have only 5-10% of the needed telemetry
capacity• Data Assembly - no broad agreement on how we
ensure coherent, integrated data sets• Lack of agreement on standards, formats and
practices• Poor modes of data transport, both routinely for
operational requirements and for science• Poor or inappropriate uptake of modern information
technology• No proper assessment of the future need.
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 33
OIT VisionOIT Vision
• ““Create an efficient and effective data Create an efficient and effective data and information management system and information management system for the ocean and marine environment, for the ocean and marine environment, based on leading-edge [ocean] based on leading-edge [ocean] information technology, and serving the information technology, and serving the oceanographic community and beyond.”oceanographic community and beyond.”
• Project Leader: Neville Smith
The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 34
ReferencesReferences
MarineXMLMEDI
OITIODE
http://www.marineXML.nethttp://ioc.unesco.org/MEDIhttp://ioc.unesco.org/oithttp://www.iode.org