© Crown copyright 2007 Impact studies with satellite observations at the Met Office John Eyre and Steve English Met Office, UK 4th WMO Workshop on "The impact of various observing systems on NWP“; Geneva; 19-21 May 2008
Mar 27, 2015
© Crown copyright 2007
Impact studies with satellite observations at the Met Office
John Eyre and Steve English Met Office, UK
4th WMO Workshop on "The impact of various observing systems on NWP“; Geneva; 19-21 May 2008
© Crown copyright 2007
Impact studies with satellite observations at the Met Office
Focus on results with implications for design of the GOS
• ATOVS – 1, 2, 3 satellites (reminder) Steve English
• ATOVS RARS Brett Candy
• ATOVS MetOp Brett Candy
• MetOp – IASI and ASCAT Fiona Hilton, Simon Keogh
• Cloudy AIRS radiances Ed Pavelin
• Windsat Brett Candy
• GPS-RO – COSMIC and others Mike Rennie
• AMVs Mary Forsythe
• Ground-based GPS Adrian Jupp
© Crown copyright 2007
How many microwave sounders?Reduction in forecast rms error
Impact
Amount of data0% 50% 100%
0%
50%
100% 1st ATOVS
~75% of impact at 45% of data coverage 2nd ATOVS
~95% of impact at 85% of data coverage
3rd ATOVS 100% of impact at 100% of data coverage
Albach Workshop results: summary from 10 experiments
© Crown copyright 2007
How many microwave sounders?Conclusions
• First and second AMSU are very important to NWP
• Third satellite has positive impact overall, but its main role is robustness and mitigating data delays
• Complete global coverage is very important – more data improve forecasts if they fill gaps in data coverage
• Impact of 4th satellite demonstrated with 4D-var
• … but it is still most important to fill the gaps!!
© Crown copyright 2007
Regional ATOVS Retransmission System (RARS)
forecast impactForecast benefit of
timely ATOVS data
2 experiments:
• All ATOVS: data
assimilated regardless of
arrival time
• RARS: ATOVS global +
fast delivery data from 14
RARS stations
• Baseline: operations with
cut-off = 2h45
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
24 48 72 96 120 24 48 72 96 120
Forecast Time (Hours)
imp
rove
men
t in
PM
SL
fo
reca
st e
rro
r (%
)
All ATOVSG RARS
NH SH
© Crown copyright 2007
500 hPa height. RMS difference between analyses with all ATOVS and operationally-available ATOVS
ATOVS data missing cut-off would benefit N Pacific and S Hem.
Regional ATOVS Retransmission System (RARS)
forecast impact
© Crown copyright 2007
Met Office global NWP index
• Skill score:
S = 1 – rf2/rp
2
• rf = rms forecast error
• rp = rms persistence error
• Weighted as table Smean
• N = (1 – Smean)-½
• Index = 100 x N / N0
• N0 = value on 31 March 2000
June 2007 value = 130
1% reduction in r.m.s. error 1% increase in Index
WeightsForecast period
T+24 T+48 T+72 T+96 T+120
NH
PMSL 10 8 6 4 4
H500 6 4 2
W250 12
TRW850 5 3 2
W250 6
SH
PMSL 5 4 3 2 2
H500 3 2 1
W250 6
© Crown copyright 2007
MetOp ATOVS
• Switch from NOAA-15 to MetOp improved forecast skill:
• +0.6 on Met Office global NWP index
• Switch to operations was made on 17 January 2007 only 90 days after launch!
• Rapid access to new data is important to users
© Crown copyright 2007
MetOp IASI
Red – Used (Sea/Land, Clear/MWcloud)
Yellow – Used (Sea/Clear only)
Blue – Used(1D-Var preprocessor only)
Cyan – Rejected
Green / Lime – Rejected water vapour channels
Channel selection
© Crown copyright 2007
24 May – 24 June 2007
Preferred configuration • include water vapour channels• obs errors in 4D-Var: 0.5K / 1K / 4K
Met Office global NWP index• +1.21 v obs , +0.80 v analysis• +1.0 overall
Compare with AIRS for same period• +0.63 v obs, +0.12 v analysis• +0.37 overall• normally see more impact from AIRS
MetOp IASIimpact trial results
© Crown copyright 2007
H100,H50
H50 shortrange
PMSL
H500,Winds
T100,T50H500,
Winds
Down isGood!
MetOp IASIChange in rms forecast error v observations
© Crown copyright 2007
• IASI initial impact of +1.0 is top of:• 3 x ATOVS on NOAA platforms + ATOVS on MetOp• AIRS• SSMIS
• Current use of data is cautious
• cloud-free fields of view over sea
• restricted channel set
• high observation errors
• Much more impact to come …
MetOp IASIsummary
© Crown copyright 2007
Cloudy AIRS radiances
• In current assimilation of AIRS and IASI, cloud-affected obs are rejected
• only a small proportion of observations retained
• Moving towards assimilation of cloud-affected radiances
• simple cloudy RT models allow careful use of channels peaking above cloud
Cloud top
Weighting functions of channelspeaking above cloud
© Crown copyright 2007
Cloudy AIRS radiancesIncreased AIRS usage in cloudy regions
© Crown copyright 2007
Cloudy AIRS radiancesImpact of assimilating AIRS in cloudy areas
• twice as many observations assimilated
• observations assimilated in meteorologically active areas
• +1.0 points on Met Office global NWP index
• equivalent to doubling overall impact of AIRS
NWP index change with cloudy AIRS assimilation
© Crown copyright 2007
MetOp ASCAT
• C-band scatterometer
• Geophysical model function to transform wind vectors to backscatter coefficients
• Ambiguous wind vector retrievals - typically 180 degree ambiguity in wind direction
© Crown copyright 2007
Typical U component 10m wind RMS O-B values for various observation types
0
0.5
1
1.5
2
2.5
3
Ships Buoys ERS-2 AMI WindSat QuikScat MetopASCAT
RM
S(O
bserv
ed
- B
ackg
rou
nd
) m
/sMetOp ASCAT
wind speed performance
© Crown copyright 2007
MetOp ASCAT
• Tropical Cyclone Gonu, June 2007
• ASCAT + MSG IR
• C-band - far less rain-contaminated data than for Ku-band instruments
© Crown copyright 2007
MetOp ASCAT impact trial results
• 24 May – 24 June 2007
• Met Office global NWP index:
• +0.35 +0.35 v obsv obs• neutral v analysis
• in presence of QuikSCAT
and ERS-2 data
© Crown copyright 2007
Scatterometers impact on Met Office global forecasts
• ASCAT is giving approx. same impact as Seawinds• 2 global-coverage scatterometer missions provide
significantly more benefit to NWP than oneMet Office, Met R&D Technical Report 511, 2008
Trial compared with NO-SCAT control
NWP indexv obs
NWP index v analysis
All scatterometers +0.97 -0.07
ASCAT only +0.61 +0.29
QuikSCAT only +0.66 -0.08
© Crown copyright 2007
WindSat wind vectors
QuikScat WindSat
• WindSat-specific quality control developed
• In particular, low wind speeds rejected due to low information content
• Ambiguous wind vectors assimilated in similar manner to Quikscat
© Crown copyright 2007
Windsat wind vectorsanalysis increments and forecast impact
QuikScat
WindSat
PMSL improvements (%)
0
0.5
1
1.5
2
2.5
3
24 48 72 96 120 144
Forecast time (hours)
QuikScat WindSat
relative forecast impact
1-month trial, Aug 2005
© Crown copyright 2007
GPS radio occultationMet Office operational use
Sep 2006 1st assimilation of CHAMP + GRACE-A (GFZ) refractivities
Nov 2006 CHAMP and GRACE-A withdrawn – GFZ qc problems
May 2007 4 COSMIC satellites assimilated
Nov 2007 4 6 COSMIC satellites
Apr 2008 Increase vertical range: 4-27 km 0-40 km
Jul 2008? Plan to re-introduce CHAMP and GRACE-A
© Crown copyright 2007
COSMIC radio occultation dataforecast temperature v sondes
S.Hem., Dec 2006, 6 COSMIC v no GPS-RO
24h temperature forecast 200 hPa temperature
Mean error
RMS error
bias bias
rmsrms
0.8
0.4
K
0
4
2
K
0
10
100
1000 0 48 96 120h
10
100
1000 0 1 2 3K
© Crown copyright 2007
Stratospheric bias is reduced by assimilating refractivities up to 40 km.
Results shown in bending angle space
Without RO
Mean O-B
St. Dev. O-B
With RO
COSMIC radio occultation dataadding data up to 40 km
© Crown copyright 2007
Forecast rms % difference v radiosondes, June 07
better worse
4 COSMICAll GPSRO
Radio occultation: Increasing the number of occultations
23 May – 24 June 2007
© Crown copyright 2007
GPS radio occultationimpact of increased vertical range
• Increased vertical range of refractivity assimilation:
• from 4-27 km
• to 0-40 km
• Small benefit:
• non-tropical RH
• low level winds
• temperature bias in lower stratosphere
© Crown copyright 2007
GPS radio occultationoverall impact - refractivity assimilation
• Large impact in SH forecasts at all ranges for T, H and wind• > 6 % improvement in rms error v sondes, for T100, T250
• Useful improvements in Tropics in same fields:• ~3% improvement in rms error for T50, T100 and T250.
• NH impacts small but positive • Small improvements in RH
• Impact of 6 COSMIC on Met Office global NWP index:
• 1.3 v observations, 0.8 v analysis
• More impact: 4 6 8 satellites
AMV impacts
Tested in 2 seasons:
12 Dec 05 – 11 Jan 06 4D-Var N216 L50
1. Control (operational observations, March 06)
2. All AMV data removed
3. All satellite data removed
4. AMVs added on no satellite baseline
12 Dec 07 – 12 Jan 08 4D-Var N216 L50
1. Control (operational observations, Nov 07)
2. All AMV data removed
Met Office Global NWP Index• Measure of model forecasting skill• Forecasts are verified by comparison with observations and analyses• Calculated from a range of parameters (PMSL, H500, W850, W250), over different areas and forecast ranges.
AMV impact Results from Dec 2005
1.5
18.8
8.6
Operational baseline
No satellite baseline
1. AMV denial
2. No Satellite + AMV
wind at 850 hPa v sondes
Tropics
NH
SH
AMVs improve forecasts, although impact is modest compared to ATOVS radiance data.
© Crown copyright 2007
TR NH
12 Dec 05 – 11 Jan 06 12 Dec 07 – 12 Jan 08
Poor impact on TR PMSL
Poor impact on TR height fields
Overall similar pattern of impacts, but generally smaller in Dec 07 season.
Possibly due to model and observation usage improvements (e.g. IASI, GPSRO), but may be partly seasonal variation.
NWP index = -1.8 NWP index = -0.9
NH SH SHTR
Mostly positive impact from AMVs (bars above line)
AMV impactcomparing Dec 2005 with Dec 2007
Verification versus observations
© Crown copyright 2007
AMV impact MODIS polar winds
T+48 500 hPa height forecast error, Dec 07 – Jan 08:
difference between control and trial
Control - No AMVs
Improved RMS in H500 over NH polar region
also seen in original MODIS polar wind experiments
© Crown copyright 2007
zenith total delay, ZTD
62
0
10z
W
z
bpapZTD dz
T T
• Observations from E-GVAP near real-time GPS network
• very high time resolution - often several per hour - potentially useful in 4D-Var
• At the Met Office:•assimilating ZTD into regional (12 km)
and UK (4 km) models•assimilating one per hour in 4D-Var•small positive impacts on cloud,
surface temperature, visibility and precipitation
•operational since March 2007
Ground-based GPS
© Crown copyright 2007
Impact studies with satellite observations at the Met Office: Conclusions
Results with implications for design of the GOS:
• MW sounders in 3 well-space orbits are close to optimal
• ATOVS RARS – improved timeliness is beneficial for global NWP
• MetOp ATOVS – an excellent example of early availability
• MetOp IASI – substantial impact from cloud-free radiances
• … and more expected from cloudy radiances (IASI and AIRS)
• MetOp ASCAT – highest quality scatterometer; impact demonstrated
• Windsat – impact comparable to scatterometers (for global NWP)
• GPS-RO – good impact from 6 COSMIC; more impact from >6
• AMVs – useful impact; qc and error characterisation problems remain
• Ground-based GPS – small positive impact in regional/UK models
© Crown copyright 2007
Questions?
© Crown copyright 2007
-14
-12
-10
-8
-6
-4
-2
0
2
Perc
enta
ge c
hange in R
MSE
Z250
Z500
Z700
Z850
Ship
&buoy
pm
sl
Syn
op
pm
sl
3-> 0 ATOVS 3-> 1 ATOVS 3-> 2 ATOVS
How many microwave sounders?Impact on analysis accuracy:
change in fit to backgroundAlbach Workshop results
© Crown copyright 2007
IASI - Data Selection
1 pixel in 4• collocated-AVHRR “Most Homogeneous” field of
view
No data used over sea iceNo data used where IR cloud tests failed
• Cost test (English et al. 1999)
• Compare IASI with AMSU (Cheng et al. 2006)
• Threshold on SD of 4 IASI pixels (Cheng et al. 2006)
© Crown copyright 2007
IASI - Channel selection for data storage
300 channels selected with information content method (Collard 2007, submitted to QJRMS)Choose successive channels which contain most information content for atmospheric profileAvoid adjacent channels to reduce correlated error
• (only use diagonal error covariance matrix in VAR)
Avoid channels affected by trace gases we don’t modelAdd 14 extra channels for monitoring to give 314 in total(cf AIRS: 324 channels)
© Crown copyright 2007
IASI - Channel selection for data processing
Reject some problematic channels (inc highest peaking)Reduce number of water vapour channels used
• (will come back to this later)• But note we are using water vapour channels!
Left with 183 channels used in 1D-Var retrievalReject low-peaking channels
• over land• where AMSU detects cloud (by-product of surface
type test)138 used in 4D-Var where high-peaking channels are removed to avoid stratospheric ringing(cf AIRS: 63 channels)
© Crown copyright 2007
IASI - Data Processing
We use RTTOV 7• kCARTA coefficients
Observations are processed through a 1D-Var scheme before assimilation in 4D-VarObservation error (SD) of
0.5K 15μm CO2 band (c.f. O-B fit of ~0.3K)1K window channels (c.f. O-B fit of
~0.6K)4K water vapour channels (c.f. O-B fit of
~1.4K – see later!)
© Crown copyright 2007
IASI - Assimilation trials
Pre-operational testing via one-month trials 24th May to 24th June 2007Processing very similar to existing ATOVS/AIRS processingEight different configurations tested with
• Differing channel selections• Different model resolutions (N216,N320; 50L,70L)• Recalculated bias corrections• Different observation errors• It has also been tested with two different model
physics packages
© Crown copyright 2007
IASI - Trial results
Results fairly stable throughout trial period which was a difficult period for the Met Office Unified Model operationally
Results proved robust to different trial configurations
We measure trial performance using the “NWP Index”
• Combines 22 key variables of interest to our customers
• Weighted mean skill relative to persistence
• Measured using both observations and analyses as verification, and the two values averaged
All trials showed positive impact
© Crown copyright 2007
MetOp IASIChange in rms forecast error v analyses
T700, T500, T100
T250,T50
HeightsHeights
Down isGood!
© Crown copyright 2007
IASI Improvement to model fields
I wanted to show some nice plots of improvements to model fields……but the changes are minor improvements across the board adding up to a good increase in the index overall. They don’t show up in plots!Results are more robust to changes in configuration and model physics
© Crown copyright 2007
IASI – S.Hem. Height Profile T+24,Mean Forecast Error – Verification vs Sonde
© Crown copyright 2007
Tropics Relative Humidity 500hPa T+24 timeseries RMS Forecast Error – Verification vs Sonde
Control IASI
© Crown copyright 2007
Improvement in fit to other satellite dataNOAA-18 AMSU-A Channel 5 (750hPa)
VA
R R
MS
(ob-
calc
) tr
ial-c
ontr
ol [
%]
Initial Fit Final Fit
© Crown copyright 2007
Improvement in fit to other satellite dataNOAA-16 AMSU-A Channel 14 (stratosphere)
VA
R R
MS
(ob-
calc
) tr
ial-c
ontr
ol [
%]
Initial Fit Final Fit
© Crown copyright 2007
Improvement in fit to other satellite dataMetOp HIRS Channel 11 (water vapour)
VA
R R
MS
(ob-
calc
) tr
ial-c
ontr
ol [
%]
Initial Fit Final Fit
© Crown copyright 2007
MetOp ASCAT wind speed performance
Scatterometer RMS O-B wind speeds for 30/3/2007 - 7/6/2007
1
1.05
1.1
1.15
1.2
1.25
1.3
1.35
1.4
1.45
ERS2AMI SEAWINDS ASCAT
Scatterometer
RM
S O
-B (
m/s
)
after Met Office quality control applied
© Crown copyright 2007
COSMIC-1 Global model biases
Monitoring bending angle (O-B) statistics (using 6 hour forecast) shows a distinct ‘S’ shape bias in the global model at higher than 50 hPa (~22km, around model level 35).Note ECMWF mean and standard deviation also shown for comparison.
© Crown copyright 2007
40 km upper range
‘S’ shape bias can be reduced by refractivity assimilation up to 40 km. Note result on right in bending angle space
Without RO
Mean O-B
St. Dev. O-B
With RO