Delivering Digital Services

Delivering DigitalServices

US Weather Standards and

Codes

Presented By: Matt PeroutkaAaron BraeckelSteve Olson

Date: August 28, 2014

2

Concise History

• July 2011: WMO’s Commission for Basic Systems authorized a task team to create “XML representation of weather data”

• November 2013: AvXML version 1.0 posted to http://schemas.wmo.int

• January 2014: IWXXM-US version 1.0 posted to http://nws.weather.gov/schemas/

2

3

Motivation

• Unique regional requirements in US• Requirements fulfilled and documented in

Traditional Alphanumeric Codes (TAC) via filed differences

3

4

Guiding Principles for IWXXM-US

• Support all filed differences maintained by the US for METAR, SPECI, TAF, and SIGMET

• Enable the development of straightforward techniques that convert products from US-specific formats to formats that comply with international standards

4

5

Implementation of IWXXM-US

• Extensions to IWXXM• Regional namespace

http://nws.weather.gov/schemas/IWXXM-US/1.0/Release

• Three data models/schemas– usMetarSpeci.xsd– usSigmet.xsd– usTaf.xsd

5

6

IWXXM-US Schemas and Code Tables

http://nws.weather.gov/schemas/IWXXM-US/1.0/Release/

http://nws.weather.gov/codes/

6

7

IWXXM-US

• Re-use of IWXXM constructs and style, where possible

• Use of appinfo and quantity attribute, as in IWXXM

• Use of code lists and vocabulary attribute, as in IWXXM

7

8

Sea Level Pressure in usMetar.xsd

8

<element maxOccurs="1" minOccurs="0" name="seaLevelPressure" type="gml:MeasureType"> <annotation> <appinfo> <quantity> http://codes.wmo.int/common/c-15/me/pressureReducedToMeanSeaLevel </quantity> </appinfo> </annotation></element>

9

Pressure Tendency Characteristic in usMetar.xsd

9

<complexType name="PressureTendencyCharacteristicType"> <annotation> <appinfo> <vocabulary> http://codes.wmo.int/bufr4/codeflag/0-10-063 </vocabulary> <extensibility>none</extensibility> </appinfo> </annotation></complexType>

10

Sea Level Pressure in Example US METAR

10

<iwxxm-us:seaLevelPressure uom="hPa">1008.3</iwxxm-us:seaLevelPressure><iwxxm-us:pressureChangeIndicator xlink:href="http://nws.weather.gov/codes/FMH-1/2005/PressureChangingRapidly/ PRESSURE_FALLING_RAPIDLY"/><iwxxm-us:pressureTendency3hr uom="hPa">6.2</iwxxm-us:pressureTendency3hr><iwxxm-us:pressureTendencyCharacteristic3hr xlink:href=" http://codes.wmo.int/bufr4/codeflag/0-10-063/8 "/>

11

US Codes

11

US Codes needed for IWXXM-US and other US products (MDCRS, PIREP, etc.)

Codes so far taken from:• Federal Meteorological Handbook• National Weather Service Instruction (NWSI) publications

12

WMO Codes Registry

12

Authoritative, publicly-accessible definitions

• Relationships between concepts (links, sub-types, etc.)

• Semantics (Skos - TTL, RDF)• Code management policies

(defined lifecycle, deprecated concepts, etc.)

• Allows for distributed management and/or authorities

13

Status

13

Textual codes available at http://nws.weather.gov/codes/

Registry prototyping also underway

Codes Encoding

14 14

US Codes populated on an as-needed basis so farAnalyze documents

Distill specific list of codes

Review with US weather agencies

Encode in some technical form (e.g., semantic triples)

Make available through web pages or a codes registry

A single sustained effort to fully encode each document would probably save effort

15

Guiding Principles for creating Metadata records for NOAA wx Data

• Strong Governance– NAO 212-15 and procedural directives

• Adherence to ISO and OGC standards– Best way to ensure compatibility between difference types

of web services supporting metadata records– Enables flexibility in data discovery methodologies (CSW,

Reg/Rep, CKAN, etc.)• Providing the proper set of tools and training resources– Enable creation, maintenance, conformance, and

transformation of records.

16

Strong GovernanceThe role of NOAA Administrative Order (NAO) 212-15 • NOAA Administrative Order (NAO) 212-15, Management of Environment Data

and Information, states environmental data be managed based upon a lifecycle that includes a data management plan. It also helped establish the Department of Commerce (DOC) National Oceanic and Atmospheric Administration (NOAA) Environmental Data Management Policy.

• NOAA has approved procedural directives for:– Data Documentation (https://www.nosc.noaa.gov/EDMC/PD.DD.php)– Data Management (https://www.nosc.noaa.gov/EDMC/PD.DMP.php)– Data Sharing for NOAA Grants (https://www.nosc.noaa.gov/EDMC/PD.DSP.php)– Data Archival

(https://www.nosc.noaa.gov/EDMC/documents/NOAA_Procedure_document_final_12-16-1.pdf)

– Data Access and Data Citation are in development

17

Strong Governance (Cont’d)• NOAA Data Documentation Procedural Directive established the

ISO 19115 parts 1 and 2 and 19139 as the recommended standards for data and information

• Approved by NOAA’s Environmental Data Management Committee in Oct 2011

• Recommends development of common metadata management tools including mechanisms for evaluating the completeness and quality of data documentation

• Weekly and monthly collaboration activities across the various NOAA line offices

18

Adherence to standards• ISO Technical Committee 211 Geographic

information/Geomatics (TC211) is responsible for the ISO geographic information series of standards.

Standard TopicISO 19115 Geographic information –

metadataISO 19115-2 Geographic information —

Metadata - Part 2: Extensions for imagery and gridded data

ISO 19119 Metadata for servicesISO 19139 Metadata - XML schema

implementation

19

Upcoming changes to standards

ISO 19115-* XML Schema/Transform Status

19115-3 • XML schemas implementation of 19115 -1, 19115-2 and related standards. • ISO 19115-1 and 19115-2 will be the conceptual models• Transform – 19115/19115-2 / 19139 to 19115-3 is done and validates against

schema. It’s available on public github.• Future revised version of ISO 19139 will contain only encoding rules, no UML• Expected to be a International Standard by May 2015

19157-2 • Data Quality XML encoding• Transform done – included in 19115/19115-2 / 19139 to 19115-3

One Key to transition is XSLT stylesheet transforms

20

NOAA Metadata Training and Tool Resources• Training

– NCDDC Metadata Training Webinars – (http://www.ncddc.noaa.gov/metadata-standards/metadata-training/)

• Tools– EMMA webpage – (http://www.ngdc.noaa.gov/metadata/emma/index.html)

• Has links to all metadata tools (guidance, authorship, component management, translation, validation, and metrics and reporting)

– EDM Dashboard -- (https://sites.google.com/a/noaa.gov/edm-dashboard/home)• Show the status and progress of activities guided by the National Oceanic and

Atmospheric Administration (NOAA) Environmental Data Management Committee (EDMC).

– NGDC Metadata Wiki webpage• Goto guide for metadata management

– ISO Explorer• Goto guide for ISO Metadata

– Rubric Scoring for metadata completeness and compliance

21

Questions

21

22

Contact Information

Matt PeroutkaNational Weather Service Office of Science and [email protected]+1 301-713-1768 x144

Aaron BraeckelNational Center for Atmospheric [email protected]+1 303-497-2806

Steve OlsonNational Weather Service Office of Science and [email protected]+1 301-713-0056 x190

22