Development of Quality Monitoring System on satellite Sea Surface Salinity Products Yongsheng Zhang NESDIS/NODC – UMD College Park/ESSIC/CICS.

Development of Quality Monitoring System on satellite Sea Surface

Salinity Products

Yongsheng Zhang NESDIS/NODC – UMD College Park/ESSIC/CICS

• In the past four years, NODC has developed a quality monitoring system known as the data Rich Inventory (RI) for Jason-2 level-2 products.

• The concept of RI is originally developed by NGDC/NESDIS for data management in CLASS. The basic idea is to extract existing metadata and calculate the QA statistics in a granule, store them in a database, and make them available as part of a data discovery system.

• NODC’s Jason-2 data quality monitoring system provides:

1) real-time original data process and visualization, statistical value calculation and QA information packing, and conversion of the outputs in CF compliant NetCDF format.

2) Provided data achiever and user the data quality information in numerical presentation via various data access tools (http, ftp and OPeNDAP) and graphical one via web interface.

Development of Quality Monitoring System at NODC

http://www.nodc.noaa.gov/SatelliteData/Jason2/qa.html

1. To develop automation processing and visualizing tools for near real-time level-2 SMOS data in swath, daily, 3-day and monthly mean time frames;

2. To develop tools for calculation of QA statistical values in a SMOS level-2 swath data file and visualization (valid observation number, mean, standard deviation, minimum & maximum) ;

3. To perform and develop tools and methods for histogram analyses for comparing the monthly mean SSS among SMOS, Aquarius and NODC objective analysis from In-situ observations;

4. To Conduct investigation on SMOS QA flags choices

Current tasks and goals of QA monitoring on SMOS/Aquarius Sea surface salinity products

• 1.0x1.0 longitude/latitude box average;

• Applied Quality Flags: 1. Dg_quality_sss_# (model #):

Quality index for sss, lower=better, values over 300 are filtered

2. Control_Flags_# (model #):

Total = 32, but only contains 27 meaningful control flags, all applied except

mask(tag # 1), roughness correction (tag # 16), availability of ECMWF(tag # 19)

and grid point measurement discrimination test (tag # 19).

• Software: FORTRAN,C-shell, GrADS and Linux image

convert;

• Outputs: EPS and PNG format.

Development visualizations of level-2 SMOS data in swath, daily, 3-day and monthly mean

Development of tools for calculation of quality statistics a level-2 granule

• Six QA statistics:– Valid number of observations– Number of observations over 3-sigma– Mean– Standard deviation– Minimum & Maximum

• Provide visualizations (EPS and PNG);

• Convert the data into CF compliant NetCDF format and provide data access via LAS/OPeNDAP/THREDDS servers (future work)

Data filters: SSS1=-999; Dg_quality_SSS_1=999, Control flags not applied

Zero Obs NO

Data Sets and Histogram Analysis

Data sets: monthly 1x1 degree, April 20120• Aquarius: JPL PO.DDAC Level-3 mapped and smoothed version 1.3.• SMOS: gridded by NODC from level-2 swath data with Dq_quality and control flag filters.• NODC monthly objective analysis.

Histogram analysisReferences: “The SST quality Monitor (SQUAM)”, P. Dash et al. 2010. Median and Robust Standard Deviation (RSD) are used to construct thresholds to remove extreme values, compared to conventional mean and Standard Deviation (SD) :

SD= RSD=IOR/S

IOR is the interquartile range: difference of the values of 75th and 25th percentile in an ordered dataset.S=1.348 for an ideal normal distribution

Outliers: median ± 4xRSD

Difference of monthly SSS between SMOS and Aquarius

Histograms of SSS difference between SMOS and AquariusApril 2012

Num

ber D

ensi

ty (%

) N

umbe

r Den

sity

(%)

SMOS Model 1 SMOS Model 2 SMOS Model 3

Left outlier: Value < median-4xRSD; Right outlier: Value > median+4xRSD

Histograms of SSS difference between NODC OA and SMOSApril 2012

SMOS Model 1 SMOS Model 2 SMOS Model 3

Num

ber D

ensi

ty (%

) N

umbe

r Den

sity

(%)

Num

ber D

ensi

ty (%

)

Histogram of SSS difference between NODC OA and Aquarius April 2012

Histogram statistics for monthly difference among NODC OA, SMOS, and Aquarius SSS, April 2012

Observation N

mean medianStandard Deviation

Robust Standard Deviation

N: left outlier

N: right outlier

SMOS(M1)-Aqu 32856 -0.09 -0.01 1.42 0.42 2018 863

SMOS(M2)-Aqu 31981 -0.19 0.03 1.38 0.42 2219 730

SMOS(M3)-Aqu 32889 -0.12 0.0 1.47 0.45 2159 805

NODC OA-SMOS (M1) 25850 0.31 0.27 1.64 1.20 138 182

NODC OA-SMOS (M2) 24941 0.36 0.25 1.72 1.26 69 277

NODC OA-SMOS (M3) 25927 0.34 0.16 1.85 1.43 72 324

NODC OA-Aqu 25883 0.34 0.31 3.70 1.15 209 358

SMOS level-2 data quality flags choices and impact in monthly mean results

Test and Applied Quality Flags:

• Dg_quality_sss_#

– Quality index for sss, lower=better, values over 300 are filtered

• Control_Flags_#

– Total 32, but only contains 27 meaningful control flags, all

applied except:

– Mask (tag # 1)

– Roughness correction (tag # 16)

– Availability of ECMWF(tag # 19)

– Grid point measurement discrimination test (tag # 19).

Distributions of number density of Dg_quality_sss_# in April 2012 (849 swath files)

num

ber d

ensi

ty (1

0**6

)

Values of Dg_quality_sss_# flags

Dg_quality_SSS_1

Good Bad

Data filtered: Dg_quality_SSS_1 <= 300

Data filtered: Dg_quality_SSS_1 >= 300

SMOS Filter: Dg_quality_SSS_1 =999

SMOS Filter: Dg_quality_SSS_1 <300

SMOS Filters: Dg_quality_SSS_1 =999; Control_flags_1, except tag# 1,6,16,18,19

SMOS Filters: Dg_quality_SSS_1 <300; Control_flags_1, except tag# 1,6,16,18,19

Future Work• Application of more flags in data quality analysis in SMOS level-2:

Control_Flags_#, Science_Flags_#;

• Compare difference among SMOS Models 1, 2 and 3.

• Develop NetCDF format for the QA statistics and data visualization tools;

• Histogram analysis and development NetCDF format for the histogram statistical data and visualizations;

• Development of software for automation processing and user service interfaces