WMO 1 WMO Regional Association V Fifteenth Session Bali, Indonesia 30 April – 6 May 2010 Proper Data Management Responsibilities to Meet the Global Ocean Observing System (GOOS) Requirements Dr. William Burnett Data Management and Communications U.S. NOAA/National Data Buoy Center Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
85
Embed
WMO 1 WMO Regional Association V Fifteenth Session Bali, Indonesia 30 April – 6 May 2010 Proper Data Management Responsibilities to Meet the Global Ocean.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
WMO
1
WMO Regional Association VFifteenth Session
Bali, Indonesia30 April – 6 May 2010
Proper Data Management Responsibilities to Meet the Global Ocean Observing System (GOOS) Requirements
Dr. William Burnett
Data Management and Communications
U.S. NOAA/National Data Buoy Center
Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
NOAA and Climate/Ocean ObservationsNOAA and Climate/Ocean Observations
• There is an increasing demand for global climate change & ocean information, There is an increasing demand for global climate change & ocean information, services and products – this includes observations from the Indonesian GOOS services and products – this includes observations from the Indonesian GOOS (InaGOOS) and the Indian Ocean equatorial array (RAMA),(InaGOOS) and the Indian Ocean equatorial array (RAMA),
• Partnerships enhance ocean observations in the region and also build the regional Partnerships enhance ocean observations in the region and also build the regional capacity to apply these observations to understand climate risk management, coastal capacity to apply these observations to understand climate risk management, coastal resilience, ecosystems, MPAs, and other socio-economic benefits,resilience, ecosystems, MPAs, and other socio-economic benefits,
• NOAA is interested in advancing a strong, equitable and mutually beneficial NOAA is interested in advancing a strong, equitable and mutually beneficial collaboration with Region V for capacity building, socio-economic applications and collaboration with Region V for capacity building, socio-economic applications and ocean observations in the region, andocean observations in the region, and
• Training & education is important for countries building the next generation ocean Training & education is important for countries building the next generation ocean observatories. NOAA is pleased to be able to provide USA training & educational observatories. NOAA is pleased to be able to provide USA training & educational opportunities through Memorandums Of Understanding or Agreements.opportunities through Memorandums Of Understanding or Agreements.
NOAA is developing a replacement for the ATLAS system Measurements comparable to ATLAS Use more commercially available components Prototype deployment targeted for October 2010 Deploy near existing ATLAS RAMA moorings for comparison As per IA, 2 systems would eventually serve InaGOOS and RAMA
Enhanced capabilities of Members to deliver and improve access to high quality weather, climate and water and related environmental predictions, information and services in response to user’s needs and to enable their use in decision-making by all relevant societal sectors.
Any and all atmospheric, oceanographic and geophysical observations will be considered as a “climate” or high-quality observations – and should be treated as such.
1. Assess impact of new systems or changes to existing systems prior to implementation. 2. Ensure a suitable period of overlap for new and old observing systems. 3. The details and history of local conditions, instruments, operating procedures, data processing
algorithms and other factors pertinent to interpreting data (i.e., metadata) should be documented and treated with the same care as the data themselves.
4. Regularly assess quality and homogeneity of data as a part of routine operations. 5. Integrate into national, regional and global observing priorities the needs for environmental
and climate-monitoring products and assessments, such as IPCC assessments.6. Maintain operation of historically-uninterrupted stations and observing systems.7. Focus on data-poor regions, poorly observed parameters, regions sensitive to change, and
key measurements with inadequate temporal resolution as high priorities for additional observations.
8. Specify to network designers, operators and instrument engineers at the outset of system design and implementation the long-term requirements, including appropriate sampling frequencies.
9. Promote the conversion of research observing systems to long-term operations in a carefully-planned manner.
10. Data management systems that facilitate access, use and interpretation of data and products should be included as essential elements of climate monitoring systems.
1. A quality descriptor will accompany every real-time observation distributed to the ocean community.
2. Subject all observations to some level of automated real-time quality test.3. Sufficiently describe the quality flags and quality test descriptions in the
accompanying metadata.4. Observers should independently verify or calibrate a sensor before
deployment.5. Observers should describe their method / calibration in the real-time
metadata.6. Observers should quantify the level of calibration accuracy and the
associated expected error bounds.7. Manual checks on the automated procedures, the real-time data collected
and the status of the observing system must be provided by the observer on a time-scale appropriate to ensure the integrity of the observing system.
Seven Data Management LawsSeven Data Management Laws
Region V should begin to implement proper data quality techniques into their newly developed marine observation platforms - now – before the instruments are placed in the water.
To provide a real-time, end-to-end capability beginning with the collection of marine atmospheric and oceanographic data and ending with its transmission, quality control and distribution.
• Oceanographers, meteorologists, IT specialists, programmers • Observations ingest, processing, analysis• Data processing and QC algorithm development for new systems• Observation dissemination and web display• Management of station configuration and metadata
Active reimbursable partnersNational Marine Sanctuary Program Kennedy Space CenterU.S. Marine Corps Goddard Space Flight CenterArmy Corps of Engineers U.S. Coast Guard
NOS/NWLON and other NOAA Obs Independent IOOS observing partners
Gulf of Maine Ocean Observing System Stevens Institute (NJ) International SeaKeepers Society Long Island Ferry Boat University of South Florida Louisiana State University Texas General Land Office University of Connecticut Louisiana Universities Marine Consortium UNC and UNC-W Skidaway Institute of Oceanography Caro-COOPS Scripps Institution of Oceanography Forrest Oil Chesapeake Bay Observing System Shell Oil Monterey Bay Aquarium Research Inst Oregon State University University of Southern Mississippi
Joint WMO-IOC Technical Commission for Oceanography and Marine Meteorology (JCOMM) Regional Marine Instrument Center (RMIC) Training Workshop on April 13-15, 2010 for WMO Regional Association IV (RA-IV) at NDBC. Representatives from ten countries participated in this first Workshop. Countries represented were: Bahamas, Costa Rica, NL Antilles, Guatemala, Belize, Canada, France, Barbados, Morocco, and China.
Observations in any format – may or may not be quality controlled
IFremer
GD
AC
Provides access to data, checks formats
EuroSITES
Formats observations and provides QC
NDBC
•Maintains specific OceanSITES platforms,•Determines what observations are released to GTS, •Assures that the platform is available and provides reliable information, •Provides the DAC with the observations in any format the DAC is willing to take, and the metadata necessary to serve as an OceanSITES platform, and•QC post-recovery data according to OceanSITES agreed procedures.
•Sets up the OceanSITES server according to the approved specifications,•Guarantees data availability from the PI,•Translates the data to the OceanSITES format,•Quality Controls real-time data according to the minimum OceanSITES agreed procedures,•Provides the observations via the GTS (if requested by the PI),•Provides the data on a FTP server for access by the GDACs
•Provides centralized access to the DAC data•Ensures no data are excluded at the GDAC level, and full high-frequency data sets are available, •Keeps only the best version of the data. Additional products like interpolated data are separate optional sets,•Check all files daily using the “File Checker” software, •Maintains the OceanSITES catalogue, and •Synchronizes the catalogues with the second GDAC periodically ( at least daily).
The worldwide tsunami observation network also requires a real-time, data assembly center to provide continual monitoring and quality control of Deep-ocean Assessment and Reporting of Tsunamis (DART®) water pressure/height observations. The Data Assembly Center monitors the various real-time transmission of DART® messages depending on the operating mode of the bottom pressure recorder.
Transmission of real-time water level heights occurs when the tsunami detection algorithm is triggered by a seismic event, when interrogated by the NOAA Tsunami Warning Centers (TWCs) or NDBC, or at pre-scheduled intervals.
•Processing of Automated Distribution Service messages from Service Argos. It uses both the TAO calibration database and calibration files to convert raw data to engineering units and also calculates buoy positions. An automated real-time QC is performed for gross error checking and then the TAO database is updated with the corrected data.
•The TAO Real-time Data Monitoring Subsystem supports daily, weekly, and monthly QA/QC activities by providing on-demand data checking functionality to the DAC. In addition to the automated gross error checking, the real-time data monitoring subsystem provides on-demand reports for once-daily, thorough examination of all current buoy data and detailed review of the real-time data.
U.S. Voluntary Observing Ship (VOS)The United States Voluntary Observing Ship Project MissionThe mission of the Voluntary Observing Ship (VOS) project is two-fold: (1) to collect and disseminate critical real-time maritime weather observations through the recruitment and support of ships to fulfill National needs and International agreements supporting commerce, forecasts and warning programs, and the Safety Of Life At Sea (SOLAS) worldwide, and (2) to define the global climate and help measure extreme weather events, climate variability, and long-term climate changes. VOS operates at no cost to the vessel, with communication charges, observing equipment and reporting supplies furnished by the National Weather Service.
Port Meteorological Officers
Port Meteorological Officers (PMOs) support observing programs aboard Voluntary Observing
Ships. They are responsible for recruitment of new vessels as observers, and also for ensuring the
quality of observations from vessels actively participating in the program.
• 10 Gigabit Ethernet technology that delivers multi-gigabit bandwidth to all resources while maintaining a 100% uptime status. • Designed around high-end enterprise class systems. Minimum standards include Dual-Core 1.8Ghz systems, 4GB RAM, redundant Gigabit network interface cards, and redundant power.
• Started with NDBC in 1970s• Focus on “Top Five” Wind Direction, Wind Speed, Atmospheric Surface Pressure, Air Temperature and Waves
• Algorithms perform check at NWS Gateway• Quality Assurance Group at NDBC provided daily check ofmarine observations – usually8 hours to two days afterdissemination• Approx. 110 platforms
• T Transmission parity error• M Missing sensor data• W Wave message is short, checksum or parity errors.• E Spectral Density are exceeded or are in error • D Delete measurement (“permanent failure”)• S Invalid statistical parameter (e.g., mean > max)• V Failed time continuity. • L Failed range (climatological) limits• R Related measurement has failed a hard QC check
(e.g., WVHGT fails → Periods failed).
Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
• a: Measurement is above monthly, regional limit.• b: Measurement is below monthly, regional limit.• c: Measurement has been adjusted, or corrected. • f: Measurement failed hourly time continuity. • m: High frequency spikes detected in the wave spectrum.• p: Failed wave height to wave period comparison test.• q: Swell direction is from an improbable direction. • w: Failed wind direction verses wave direction check.• x: Wind wave energy is too high for prevailing wind
speed.• y: Wind wave energy is too low for prevailing wind speed.
Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
• Dependent on time since last valid observation.• Max. allowed value = 0.58*Std.dev.*SQRT(TimeDiff)• Works well for normally distributed measurements• Std. dev. Chosen 50% higher climatic standard dev.
Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
• Maximum allowable values in one hour:– Sea level pressure 12.2 hPa– Air Temperature 6.4 deg. C– Water Temperature 5.0 deg. C – Wind Speed 14.5 m/s– Wave Height 3.5 m
Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010