A Neural Network PMW/IR Combined Procedure for Short Term/Small Area Rainfall Estimates

A Neural Network PMW/IR Combined Procedure

for Short Term/Small Area

Rainfall Estimates

Nal. Council of Research,Italy

University of L’Aquila,Italy

University of Birmingham,UK

Francisco J. Tapiador & Chris Kidd

University of Birmingham, UK

Vincenzo Levizzani

National Council of Research, Italy

Frank S. Marzano

University of L’Aquila, Italy

1st INTERNATIONAL PRECIPITATION WORKING GROUP WORKSHOP

Madrid, 23 – 27 September 2002

http://www.bham.ac.uk/

• Objectives of today’s presentation1. Present a methodology of data fusion of IR and PMW data at global scale:

• Short term, large coverage and high resolution rainfall estimates• Methodology to be applied to MSG (soon) and GPM products

2. Assess the quality of these estimates: • Intercomparison / Validation: HM method• Down-top approach

3. Present further research and operative products schedule

• Scheme:– Some comments on Neural Nets– Histogram matching– Validation / Intercomparison case study:

• Andalusia, Spain: 3 months of 30 minutes rain gauge data for validation

– Global research products• Global IR – derived estimates• METEOSAT - derived estimates

– Further work in this line

Outline Highlights Neural Nets Case Study Products Future work







• Highlights

– Why fuse PMW and IR?• Direct response vs indirect relationship• “Bad” spatial and temporal resolutions vs geostationary capabilities• Re-inforce the strengths and avoid the weaknesses

– Inputs processing• IR data from the Global IR database (Janowiak et al 2001) and EUMETSAT archive• PMW Rainfall retrieval based upon Kidd&Barrett SSM/I algorithm:

– V19-V85 or H19-H85 combination over ocean and over land– Polarization Corrected Temperatures (PCT) over coast

• Gauge processing: point to area estimates using maximum entropy interpolation• Histogram matching and GPI calculation for inter-comparison

– Neural nets Inputs selection Model selection Inversion procedures






Outline Neural Nets Case Study Products Future workHighlights





Neural Networks




• Neural Networks• NN works fairy well in rainfall estimation

– Operative system: PERSIANN (Sooroshian et al 2000)– Bellerby et al. 2000, etc.

• Neural Nets are not black-boxes– It is possible to make an objective NN selection (Murata et al 1994)– There are inversion procedures to investigate inside – They allow both deterministic and probabilistic approach

• Some advantages over other methods – Any function (Dirichlet’s, not pathological function) can be

approximate with an arbitrary degree of accuracy with a NN: Universal Aproximator.

– An easy method to simulate complex physical models in a quick (operative) way.








• Input selection







Correlations for some simple models











• Several NN architectures Hopfield nets

SOM (cloud characterization)+(GOES data)

Multilayer Perceptron (MLP)

Adaptative Resonance Theory Nets (Grossberg 1969, Carpenter et al 1997) ART1 and ART2

ARTMAP

Distributed ARTMAP

Fuzzy ARTMAP (including a voting procedure (ref))





• Model selection: Results








• Model selection into MLP• Calculate (not guess) the number of neurons in the hidden layer• Network information criterion (NIC) (Murata et al. 1994)




iLE log2A

ii LLE 'loglog B

n

trwL

nNIC

1

ˆlog1

BA

n Number of observations Set of parameters Gradient2 Hessianlog L Estimated maximum log likelihood

w



Outline Neural Nets Case Study Products Future work

• This allow a conscious design of the net based on Information Theory results

Highlights

• Research after training: model inversion

H

ii

I

j

ihj

hi xwgwf

1 0

, Wxy

*,min oWxfxe

ik

ik

ik x

exx

1

jikx

e

*jjjj ooo

mjOHj

jjj wo ,,







• What kind of inputs generate an output?: insight into precipitation processes at IR-focus

Highlights





Histogram Matching













Validation

(case study)




• Case study data:– Global IR (Meteosat 5)

– DMSP SSM/I – 30 min gauge validation data

• Resolutions:– Spatial: 4

Km– Temporal: 30 min

• Coverage:– Andalusia (Spain)– Oct-Dec 2001














• Methodology

Highlights


• What means “field truth” in satellite estimates validation?– Point estimates: more close to the truth AGL– Areal interpolations: encompassing errors and odd effects








n

i j

jj

i

iin

jij

nn

zzazzzf

0 02

11

10 2

1exp

2

1,,,

m

innn

m

in

dzdzzzfzzfS1

1111

,,log,,Z

1,,1

111

m

inn

m

in

dzdzzzf

rkcdzdzzzgzzf k

m

innkn

m

in

,,1,,,,1

1111

Con

stra

ints

2) Solving…

1) Maximize the entropy function (using variational methods)




Maximum Entropy InterpolationThe (theoretically) less-biased interpolation method available: an appropriate base to compare

2) Which means that we can solve the computational problems using a simple spherical kriging





Point measures (average) Maximum Entropy Interpolation

Inverse Distance Weighted




Small intercomparison of interpolation methods (Niger 2000 and Andalusia 2001)

•IDW

•Bilinear

•Kriging

•MEM





Inst

anta

neou

s In

terc

ompa

riso

n

SSM/I NN

HM NN*








Sm

all a

rea,

sho

rt-d

urat

ion

even

tsNal. Council of Research,Italy







Instantaneousestimates








• Results: Skill Scores








• Coincident data histogram comparison (October 2001)








• 0.1º Accumulated results

R2 = 0.57

0

10

20

30

40

50

0 10 20 30 40 50

Gauge accumulated at 0.1º (mm/month)

NN

est

ima

te a

t 0.1

º (m

m/m

on

th)








• 0.5º / 3 month accumulated data

R2 = 0.67

0

100

200

300

400

500

600

700

800

0 100 200 300 400 500 600 700 800

Gauge estimate at 0.5º (mm / three months)

NN

est

imat

e at

0.5

º (m

m /

thr

ee m

onth

s)







• 0.5º accumulated results

R2 = 0.73

0

20

40

60

80

100

0 20 40 60 80 100

Gauge accumulated at 0.5º (mm/month) restricted to cells with more than 4 gauge stations in

NN

est

imat

es a

t 0.

5º (

mm

/mon

th)

rest

ricte

d to

cel

ls w

ith

mor

e th

an 4

gau

ge s

tatio

ns in







• Grid size, averaging periods and correlations (Turk et. al 2002)










Global Coverage

(Reseach Products)




• Global-IR coverage (HM)







• Meteosat coverage (NN)

• Product to be validated using land-GPCC or other dataset

• Oriented to MSG: we are ready to apply this methodology













GOES-E 14:32 GOES-E 15:45Trajectories

SSM/I F14 14:30 SSM/I F15 15:44IR temperature along trajectory

•Wind (CMW?) trajectories found by 19x19 correlation matching over 19x19 region. •SSM/I rain then advected along trajectories and adjusted by dIR and tied at end points

•IR/PMW Advection Scheme

• Subscenes:

- Guinea Gulf- GIS integration







• Future operational applications

• QPE / QPF: • SSM/I estimates improve the forecasting (Hou et al 2002) • We can simulate SSM/I

• Agriculture• Hydrology• Natural Hazards

But only when the product become operative and better results will be obtained







• Future research work: MSG and GPM

• Radar data for validation/calibration• Operativity of the global coverage products: intercomparison• Integration in forecasting models: RAMS

• Use of MSG channels:• More information means more discrimination capabilities• Bidirectional reflectance model

• GPM and EGPM addressing













• Conclusions

• Accumulated areal estimates at 0.1º and 0.5º at monthly scale are similar to other works, but the down-top approach allow to know about small scale and short term estimates.• There is an almost-operative product to analyse and to improve with further research.• There are many reseach directions in NN data fusion to follow:

• Inversion• New methods (probabilistic nets)• Integration of other models

• Other physical models can be integrated into the NN methodology.• Any meteorological information can be integrated without major modifications• Complex models can be speed up simulating the result using NN







A Neural Network PMW/IR Combined Procedure for Short Term/Small Area Rainfall Estimates

Documents

italy university of

italy university of

group workshopmadrid

marzanouniversity of

pmw data

global ir database janowiak

area estimates

minutes rain gauge data