GOES-R AWG Product Validation Tool Development

GOES-R AWG Product Validation Tool Development

Downward SW Radiation at Surface and Reflected SW Radiation at TOA

Hongqing Liu (Dell)Istvan Laszlo (STAR)Hye-Yun Kim (IMSG)Rachel Pinker (UMD)

Ells Dutton & John Augustine (ESRL)

1

2

OUTLINE

• Products

• Validation Strategies

• Examples

• Ideas for Further Enhancement and Utility of Validation Tools

• Summary

Products

• Shortwave Radiation Products:– Downward Shortwave Radiation at

Surface (DSR)• CONUS: 25km/60min

• Full Disk: 50km/60min

• Mesoscale: 5km/60min

– Reflected Shortwave Radiation at TOA (RSR)

• CONUS: 25km/60min

• Full Disk: 25km/60min

Only daytime

3

Monitoring & ValidationBackground

• Functions of tools:– routine monitoring (may not need reference data)

– routine validation (reference data, matchup procedure)

– deep-dive validation (reference data, other correlative data, matchup)

• Basic elements:– data acquisition (ABI, ground, other sat products) (Fortran 90)

– spatial and temporal matching (lots of possibilities) (Fortran 90)

– analysis (computing statistics) (IDL)• Metadata• Accuracy/Precision• RMSE• Minimum/Maximum Error

– present results (display maps, scatter plots, tables) (IDL)

4

Validation Strategies

5

Satellite Measurements– Clouds and the Earth’s Radiant Energy

System (CERES) Cloud and Radiative Swath (CRS) dataset: (1) measured TOA upward SW flux, (2) calculated Surface and Atmospheric Radiation Budget (SARB). http://eosweb.larc.nasa.gov/PRODOCS/ceres/level2_crs_table.html

Reference (“truth”) data • Collocation of ABI retrievals and reference data is

performed at the instantaneous time scale. • Matching: ABI retrievals averaged spatially;

ground measurements averaged temporally. Averaging window size is flexible.

Independent satellite retrieval (CERES)• Collocation: CERES data are averaged to the ABI

retrieval grid on a daily basis.• Matching: current retrievals use MODIS data as

input; CERES is on same platform; no need for temporal matching.

Reference Dataset

Collocation/Match-up

Ground Measurements– High-quality routine ground radiation

measurements over Western Hemisphere from 20 stations from SURFRAD (ftp://ftp.srrb.noaa.gov/pub/data/surfrad/) and BSRN (ftp://ftp.bsrn.awi.de/) networks.

Routine Validation ToolsInstantaneous Monitoring

• Present retrieval results– Specify date & load data– Selection from ‘Variable’ menu

• Primary Outputs (image)– DSR– RSR

• Diagnostic Outputs (image)– Surface diffuse flux– Surface albedo– Clear-sky composite albedo– Clear-sky aerosol optical depth– Water cloud optical depth– Ice cloud optical depth

• Quality Flags (image)– 66 flags (inputs, retrieval, diagnostics)

• Metadata (ascii file output)

• Independent of validation truth; can be executed automatically by scripts once retrievals are available.

6

Routine Validation ToolsValidation with Ground “Truth”

• Validates DSR&RSR for a period of time– Specify time period & load data

– ‘Validation’ menu Generate scatter plot of retrievals against measurements

Generate validation statistics and output to ascii file

– ‘TimeSeries’ menuGenerate time series plots of retrieval and measurements

over ground stations

7

8

”Deep-Dive” Validation ToolsValidation with CERES

• An expansion of routine validation with CERES including cross validation against NASA SARB satellite products– Options:

• Scene types – all; snow ; clear ; water cloud; ice cloud

• Retrieval path– Hybrid path– Direct path only– Indirect path only

– TOA matching (all; succeed; failed)– Surface albedo (all; succeed; failed)

– Specify date & load data– Selection from ‘Validation’ menu

• Reflected SW Radiation at TOA (RSR) • Retrieval; Retrieval-CERES; Retrieval-SARB Tuned; Retrieval-SARB Untuned; • Statistics (Scatter plot; Statistics in ascii file)

• Downward SW Radiation at Surface (DSR)• Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics

• Absorbed SW Radiation at Surface (ASR)• Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics

• Absorbed SW Radiation in Atmosphere (ABS)• Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics

• Surface SW Albedo (ALB) • Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics

• Calculate and display– additional statistics (histograms)– temporal averages on different scales (daily, weekly, monthly)

• Identify signatures by which even non-experts can identify potential problems – needed for routine operational monitoring

• Implement automatic detection of possible systematic drift or continuous abnormal retrieval in routine validation.– establish “reference” (expected) statistics from good data

– compare time series of actual statistics with reference stats

– trigger action (e.g., sending warning email) when actual stats exceed reference stats + x std.

• Combine SW validation with LW radiation retrievals– check consistency

• e.g., high RSR low OLR is expected for cloudy scenes

– additional diagnostic information for deep-dive validation (LW radiation)

• Current tool uses retrievals from MODIS proxy data. Adjustment to tools for retrievals from geostationary orbit will be needed (data preparation).

9

Ideas for Further Enhancementand Utility of Validation Tools

10

Summary

• Current tools perform three functions:– routine monitoring of product

– routine validation with reference data

– deep-dive validation with reference and intermediate data

• Validation truth data have been identified and processed

• Planned enhancements include:

– more stats

– automatic detection of problems

– checking consistency with LW

GOES-R AWG Product Validation Tool Development

Upward LW Radiation at TOAUpward and Downward LW Radiation at

Surface

Hai-Tien Lee (CICS/UMD)Istvan Laszlo (STAR/NESDIS)

Ells Dutton & John Augustine (ESRL)

11

Acknowledgments: NOAA SURFRAD, NASA CERES, BSRN, DOE ARM, Eumetsat GERB & LSA-SAF

GOESR AWG Annual Meeting, June 14-16, 2011, Fort Collins, CO

Products

• Longwave Radiation Products:– Upward LW Radiation at TOA (OLR)

• CONUS: 25km/60min

• Full Disk: 25km/60min

– Downward LW Radiation at Surface (DLR): Clear sky only

• CONUS: 25km/60min

• Full Disk: 25km/60min

– Upward LW Radiation at Surface (ULR): Clear sky only

• CONUS: 25km/60min

• Full Disk: 25km/60min

12

GOES12 Imager OLR

Validation StrategiesReference Dataset (Ground)

• Ground Measurements– High-quality routine ground radiation measurements over Western Hemisphere used for validating ABI Longwave Radiation retrievals are collected from 7 stations from SURFRAD network.

– Selected stations of BSRN and Eumetsat LSA SAF that provide surface upward and downward longwave radiation measurements can be used for offline/framework algorithm evaluation.

13

Station Network Longitude Latitude Elevation[m] Measurements Used

fpk

SURFRAD

-105.10 48.31 634

surface LW downward, upward fluxes; clear sky index; interpolated

meteorological profiles

sxf -96.62 43.73 473psu -77.93 40.72 376tbl -105.24 40.13 1689

bon -88.37 40.05 213dra -116.02 36.63 1007gwn -89.87 34.25 98

Validation StrategiesReference Dataset (Satellite)

• Satellite Measurements– OLR product from Clouds and the Earth’s Radiant Energy System

(CERES) Single Scanner Footprint (SSF) datasets are used as algorithm validation reference.

– Future NPP and JPSS OLR (from CERES FM5/6) can be used for routine monitoring and evaluation (possibly with lag).

– Operational HIRS OLR from NOAA and MetOp polar orbiters will be used as a backup for routine monitoring purpose.

14

• Tools:– IDL (primarily)

• Data Collocation• Instantaneous Monitoring• Validation over Ground Stations• Validation with CERES• Deep-dive Validation over Ground Stations• Deep-dive Validation with CERES

• Statistics:– Metadata (ATBD), plus Mean/StDev for Global, zonal and selected

domains of interests

– Mean, StdDev, RMS, Min and Max of Errors

• Visualization:– IDL, GrADs

– Figures rendered in PNG format

15

Validation StrategiesTools , Statistics & Visualization

16

Validation StrategiesExample of Deep-Dive

FM1 FM2 FM3 FM4

OLR Error vs LZA

OLR Error vs SEVIRI Ch 5

radiance (UTH)

OLR Error vs SEVIRI Ch 9

radiance (window)

OLR Error as a function of

Ch 7 and Ch 9 radiances

Extended OLR Validation

(March 2004)

17

Summary

• Validation truth data have been identified and being acquired

• Validation tools are designed to perform:– Routine monitoring of product

– Routine validation with reference data

– Deep-dive validation with reference and auxiliary data

• Planned enhancements include:

– Temporal tracking of stats

– Define level of alarms (for routine monitoring)

– Explore possible sources of more ground truth

– Clear-sky identification with auxiliary data.

END

18

Monitoring & ValidationBackground

• Functions of tools:– Routine monitoring (may not need reference data)– Routine validation (reference data, matchup procedure)– Deep-dive validation (reference data, other correlative data, matchup)

• Basic elements:– Data acquisition (ABI, ground & satellite products) (Unix Script, IDL)– Spatial and temporal matching (closed pixel vs area average) (IDL)– Analysis (computing statistics) (IDL, Datadesk)– Present results (display maps, scatter plots, tables) (IDL, GrADs,

Kaleidagraph)

• Special considerations:

– Degradation flag

19