Study on space-time constrained Parameter Estimation from Geostationary data Second Progress Report (DRAFT version)

Study on space-time constrained ParameterEstimation from Geostationary data

Work supported by EUMETSAT, Darmstadt, Germanycontract # EUM/CO/11/4600000996/PDW

Final Report, January 2013

Work performed by

G. Masiello, C. Serio (P.I.), M. Amoroso, S. Venafra, I. De Feis∗

School of Engineering, University of Basilicata, Via dell’Ateneo Lucano 10, 85110Potenza, Italy∗IAC/CNR, Napoli, ItalyCorresponding author e-mail: [email protected]

CONTENTS 2

Contents

0 Summary 10

1 Introduction 14

2 The forward model equation 18

2.1 The case of a Lambertian Surface . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2 The case of Specularly Reflecting Surface . . . . . . . . . . . . . . . . . . . . . 20

2.3 σ-SEVIRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.1 Averaging and down-sampling the look-up-table: from σ-IASI to σ-SEVIRI 22

2.3.2 Check of the quality of the analytical Jacobian scheme . . . . . . . . . . 22

3 The general retrieval framework: static a-priori background 26

3.1 Introducing 3-D spatial constraints . . . . . . . . . . . . . . . . . . . . . . . . 27

3.1.1 Valid isotropic model for ρ(r;L) . . . . . . . . . . . . . . . . . . . . . . 28

3.1.2 Covariance for the multivariate case . . . . . . . . . . . . . . . . . . . . 29

3.2 Introducing time constraint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.2.1 Optimal estimation with times accumulation . . . . . . . . . . . . . . . 29

3.2.2 A formulation of the emissivity-temperature retrieval with the strategyof time accumulation of the observations . . . . . . . . . . . . . . . . . 31

4 The Kalman filter 34

4.1 The KF update step or analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.2 The KF forecast step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.3 A formulation of the emissivity-temperature retrieval with KF . . . . . . . . . 36

4.4 Sequential Updating: a numerical algorithm to calculate the analysis, x and itscovariance, S, without explicit inversion of matrices . . . . . . . . . . . . . . . 36

5 The ensemble Kalman filter 39

6 Optimal time-space interpolation schemes 40

7 The SEVIRI case study for emissivity-temperature retrieval 41

7.1 The target area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7.2 SEVIRI data and ancillary information . . . . . . . . . . . . . . . . . . . . . . . 42

8 Optimal estimation: implementation and retrieval exercises in simulation 47

8.1 The case of 1 hour-width time slot . . . . . . . . . . . . . . . . . . . . . . . . . 52

8.2 The case of 21 hour-width time slot . . . . . . . . . . . . . . . . . . . . . . . . 55

8.3 Implementation with the whole emissivity state-vector . . . . . . . . . . . . . . 57

8.4 An example with a dependent skin temperature vector. . . . . . . . . . . . . . 60

CONTENTS 3

9 Kalman filter: implementation and retrieval exercises in simulation 67

9.1 Kalman filter with a 2-order autoregressive process to model the evolution equa-tion of skin temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

9.2 Properties of the AR(2) model . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

9.3 Forward propagation of the analysis and the related covariance matrix . . . . 70

9.4 A retrieval exercise in simulation . . . . . . . . . . . . . . . . . . . . . . . . . . 72

9.5 Kalman filter with a persistence model equation for skin temperature . . . . . 77

10 Application to real SEVIRI observations 80

10.1 Results over land surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

10.1.1 Running the Kalman filter over a long time span . . . . . . . . . . . . . 90

10.1.2 The effect of clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

10.2 Results over sea surface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

10.2.1 Running over a long time span with the presence of clouds. . . . . . . . 108

10.2.2 Updating the Kalman filter with ECMWF analysis . . . . . . . . . . . . 114

11 Kalman filter including atmospheric parameters,[T,Q,O] 116

12 Kalman filter ([Ts, ε] case) with localized spatial constraints 124

13 Monthly Ts − ε maps over the whole target area 127

14 Concluding remarks 132

LIST OF FIGURES 4

List of Figures

1 SEVIRI channel spectral response over-imposed to a typical IASI spectrum. . 21

2 Consistency between σ-IASI and σ-SEVIRI. The upper panel shows the σ-SEVIRI radiance for the Infra-red channels 2 to 8 (6.2 to 13.4 µm). The calcu-lations refers to a black body surface emissivity and Lambertian diffusive andreflective surfaces with constant emissivity equal to 0.9. The Bottom panel showsthe percentage differences between σ-IASI and σ-SEVIRI calculations. . . . . . 23

3 Unperturbed state vector, vo, a) temperature profile, b) water vapour profile. 24

4 Consistency check of the temperature Jacobian for a perturbation of the tem-perature profile of −0.1K at each layer (panel a)), and +0.1K (panel b)). . . . 25

5 As Fig. 4, but now for the water vapour profile, panel a) considers a perturbationof -1% of the profile at each layer, the case of +1% perturbation is shown in panelb). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6 Sensitivity of the SEVIRI channels to emissivity. The computation refers to atypical tropical state vector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

7 Target area showing a map of the SEVIRI channel at 12 µm. Observation refersto the date 9 July 2010, hour 12:00. . . . . . . . . . . . . . . . . . . . . . . . . 43

8 Surface temperature (ECMWF model) map over the target area of the case study(July, the 13th 2010, 12:00 GMT). . . . . . . . . . . . . . . . . . . . . . . . . . 43

9 Example of overlapping between the SEVIRI fine mesh and that coarse corre-sponding to the ECMWF analysis. . . . . . . . . . . . . . . . . . . . . . . . . 45

10 From IREMIS to SEVIRI emissivity, passing trough IASI. . . . . . . . . . . . 45

11 Example of IREMIS-SEVIRI emissivity mapped onto the target area, for thechannel at 1149.4 cm−1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

12 Typical temperature daily cycle for desert sand. . . . . . . . . . . . . . . . . . 47

13 Background matrix for emissivity concerning the retrieval exercise. . . . . . . 49

14 Emissivity variability of the background computed from the covariance matrixshown in Fig. 13. The variability is the square root of the diagonal elements). 49

15 Variability of the daily cycle shown in Fig. 12 as a function of the time slot width. 50

16 State vector of the atmosphere (not retrieved) for the major parameters, T, H2Oand O3, used within the retrieval exercise in simulation. . . . . . . . . . . . . . 50

17 SEVIRI radiometric noise (upper panel) used to build up the observatinal co-variance matrix. For the benefit of reader accustomed to NEDT, this is shownin the bottom panel for a scene temperature of 280 K. . . . . . . . . . . . . . . 51

18 Results of the retrieval exercise for temperature with the optimal estimationscheme. The case shown applies to a time slot of 1 hour. For this exercise theFG has been set to the background (BG) . . . . . . . . . . . . . . . . . . . . . 53

19 Difference FG-True, Retrieval-True for the case shown in Fig. 18. The accuracyof the retrieval (square root of the diagonal of the covariance matrix) is shownlikewise a ±1σ tolerance interval. . . . . . . . . . . . . . . . . . . . . . . . . . 53

20 Results of the retrieval exercise for emissivity with the optimal estimation scheme.The case shown applies to a time slot of 1 hour. For this exercise the FG hasbeen set to the background (BG) . . . . . . . . . . . . . . . . . . . . . . . . . . 54

21 Difference FG-True, Retrieval-True for the case shown in Fig. 20. The accuracyof the retrieval (square root of the diagonal of the covariance matrix) is shownlikewise a ±1σ tolerance interval. . . . . . . . . . . . . . . . . . . . . . . . . . 54

LIST OF FIGURES 5

22 Same as Fig. 19, but now the time slot width is 21 hours. . . . . . . . . . . . 55

23 Same as Fig. 21, but now the time slot width is 21 hours. . . . . . . . . . . . 56

24 Background covariance matrix corresponding to a 28-size emissivity vector witha perfect correlation among channels corresponding to the same wave number. 57

25 Same as Fig. 19, but now with the whole emissivity state vector of size 28. . . 58

26 Same as Fig. 21, but now with the whole emissivity state vector of size 28. . . 59

27 Skin temperature generated with a Markovian model. . . . . . . . . . . . . . . 60

28 Background matrix for the skin temperature. . . . . . . . . . . . . . . . . . . . 61

29 Results of the retrieval exercise with the temperature signal driven by a stochasticdrift. a) time slot of one hour; b) time slot of 24 hour. For this exercise FG andBG coincide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

30 As Fig. 29, but now the two differences FG-True and Retrieval-True are shownalong with the error bars shown as ±1σ tolerance interval. a) time slot of onehour; b) time slot of 24 hour. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

31 As Fig. 30, but now the emissivity is shown as ±1σ tolerance interval. The caseshown refers to the time slot of one hour. . . . . . . . . . . . . . . . . . . . . . 63

32 Results of the retrieval exercise with the temperature signal driven by a stochasticdrift. The case applies to time slot of one hour; a) temperature retrieval; b);difference form the true value of skin temperature. For this exercise FG and BGcoincide. The emissivity is that shown in Tab. 8 . . . . . . . . . . . . . . . . . 65

33 Error analysis for the emissivity retrieval corresponding to FG and true emissivityshown in Tab. 8. The case shown refers to the time slot of one hour. . . . . . 65

34 As Fig. 32 but now a time slot of 4 hours is used. . . . . . . . . . . . . . . . . 66

35 As Fig. 33 but now a time slot of 4 hours is used. . . . . . . . . . . . . . . . . 66

36 Simulation of SEVIRI channel 7 (833.3 cm−1) response (panel b) to the forcingof the temperature daily cycle shown in panel a). Panel c) shows the effect ofthe SEVIRI radiometric noise over the response of panel b). . . . . . . . . . . 68

37 Autoregressive representation of the temperature daily cycle with the coefficientsshown in Tab. 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

38 Results of the retrieval exercise for temperature with the Kalman filter scheme. 74

39 Difference Retrieval-True for the case shown in Fig. 38. The accuracy of theretrieval (square root of the diagonal of the covariance matrix) is shown in formof a ±1σ tolerance interval. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

40 Results of the retrieval exercise for emissivity with the Kalman filter scheme. . 75


42 Difference Retrieval-True for the case shown in Fig. 40, but now only the resultfor the channel 5 is shown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

43 Results of the retrieval exercise for temperature with the Kalman filter scheme.The case shown uses a persistence model for the state equation of both emissivityand skin temperature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78


LIST OF FIGURES 6

45 Results of the retrieval exercise for temperature with the Kalman filter scheme.The case shown uses a persistence model for the state equation of both emissivityand skin temperature. The variance of the stochastic noise term for the skintemperature has been set to zero. . . . . . . . . . . . . . . . . . . . . . . . . . 79


47 Target areas in Spain, Sahara desert and Mediterranean basin used to check theretrieval algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

48 SEVIRI channel 7 observation for one clear sky day and for the the three testsites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

49 Number of iterations needed to converge as a function of the scaling factor, f .Note that we put an upper limit of 10 to the number of iterations, thereforewhen this limit is reached, the procedure has not converged. . . . . . . . . . . . 83

50 Example of emissivity retrieval and related accuracy (error bars) at the level ofone single SEVIRI pixel. The retrieval refers to the Sahara desert location. . . 86

51 As Fig. 50, but for temperature. . . . . . . . . . . . . . . . . . . . . . . . . . . 87

52 Emissivity for the Sahara desert averaged over the 219 adjacent pixels. . . . . 87

53 Skin temperature for the Spanish location averaged over the 183 adjacent pixels. 88

54 Skin temperature for the Sahara desert location averaged over the 219 adjacentpixels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

55 Retrieved skin temperature (bottom panel) for a site in the Sahara desert. Theretrieval has been obtained with the Kalman filter for ten consecutive days.In the legend, ECMWF Ts analysis refers to the skin temperature analysis atthe canonical hours within a day, whereas ECMWF Ts is the ECMWF skintemperature linearly extrapolated to the SEVIRI time steps. The upper panelin the figure also shows the quality of the reconstructed radiance (channel at833.33 cm−1). 1 r.u.=1 W m−2 sr−1 (cm−1)−1 . . . . . . . . . . . . . . . . . . 90

56 Calculated and observed radiances along with the related spectral residual forthe the three SEVIRI window channels used for the retrieval analysis shownin Fig.55. The error confidence interval shown in figure refers to the SEVIRINEDN. 1 r.u.=1 W m−2 sr−1 (cm−1)−1 . . . . . . . . . . . . . . . . . . . . . . 91

57 Cost function (left) and number of iterations as a function of the time stepcorresponding to the retrieval analysis shown in Fig. 55. The two upper panelsallow the reader to identify which radiances causes a non-convergence of thescheme. 1 r.u.=1 W m−2 sr−1 (cm−1)−1 . . . . . . . . . . . . . . . . . . . . . . 92

58 Ts − ε time evolution for the retrieval exercise shown in Fig. 55. . . . . . . . . 92

59 Exemplifying the retrieval precision for the emissivity at 1149.40 cm−1. Thefigure shows the ±σ tolerance interval (square root of the diagonal of the a-posteriori covariance matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

60 Scatter plot of Ts from the retrieval analysis and ECMWF model at the fourcanonical hours of the ECMWF analysis. . . . . . . . . . . . . . . . . . . . . . 93

61 Plot of SEVIRI radiances for the channel at 833.33 cm−1 showing the presenceof clouds according to the operational SEVIRI cloud mask. The zoom in b) andd) shows that there are both false negative (clear sky flagged cloudy) and falsepositive (cloudy sky flagged clear). 1 r.u.=1 W m−2 sr−1 (cm−1)−1 . . . . . . . 95

LIST OF FIGURES 7

62 Retrieved skin temperature for the two sites, Sahara desert and Seville. Resultsare shown only for one SEVIRI pixel. The retrieval has been obtained with theKalman filter for the whole month of July. In the legend, ECMWF Ts analysisrefers to the skin temperature analysis at the canonical hours within a day,whereas ECMWF Ts is the ECMWF skin temperature linearly extrapolated tothe SEVIRI time steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

63 As figure 62, but now the retrieval is shown which corresponds to the χ2 formminimized below the thresholds (bottom panels). . . . . . . . . . . . . . . . . . 97

64 The figure show a time sequence of SEVIRI radiances corresponding to the chan-nel at 833.33 cm−1. The figure exemplifies the presence of possibly undetectedcloudy radiances, which are removed because of lack of minimization of the costfunction below the χ2-threshold. 1 r.u.=1 W m−2 sr−1 (cm−1)−1 . . . . . . . . 98

65 As figure 62, but now the retrieval has been spatially averaged over adjacentpixels (219 for the Sahara desert target area and 183 for the Seville site). . . . 99

66 Spectral residual time series for the channel at 833.33 cm−1. Left, Sahara desert;right, Seville site. The spectral residual refer to the SEVIRI pixels whose retrievalanalysis for skin temperature as been shown in Fig. 62. . . . . . . . . . . . . . 100

67 Emissivity retrieval averaged over adjacent pixels (219 for the Sahara deserttarget area and 183 for the Seville site). . . . . . . . . . . . . . . . . . . . . . . 101

68 Monthly map of the skin temperature for the Sahara desert target area and theSeville site). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

69 Emissivity covariance matrix derived from Masuda’s model for a zenith angle of46 degress. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

70 Retrieval analysis for skin temperature. a) Optimal estimation; b) Kalman filter;c) the retrieval has been spatially averaged over the grid box of size 0.05× 0.05degrees shown in Fig. 47. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

71 Retrieval analysis for emissivity. The retrieval has been spatially averaged overthe grid box of size 0.05× 0.05 degrees shown in Fig. 47. . . . . . . . . . . . . 106

72 Maps of the daily skin temperature obtained with the two retrieval scheme. . . 106

73 Kalman filter retrieval analysis for skin temperature as a function of the stochas-tic variance term for Ts. The retrieval has been spatially averaged over the gridbox of size 0.05× 0.05 degrees shown in Fig. 47. . . . . . . . . . . . . . . . . . 107

74 Example of sea surface SEVIRI observations for the whole month of July. Theobservations refers to the window channel at 833.33 cm−1. The target area isthat shown in Fig. 47. Cloudy radiances are evidenced with red circles. . . . . 108

75 Top. Retrieved skin temperature for Mediterranean site. Results are shown onlyfor one SEVIRI pixel. The retrieval has been obtained with the Kalman filter forthe whole month of July. In the legend, ECMWF Ts analysis refers to the skintemperature analysis at the canonical hours within a day, whereas ECMWF Ts isthe ECMWF skin temperature linearly extrapolated to the SEVIRI time steps.Only clear sky radiances are processed. Bottom. χ2 values corresponding tothe retrieval shown on the left. . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

76 As Fig. 75, but now only the retrieved values which correspond to a convergedχ2 are shown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

77 Example of scatter plot of the ECMWF Ts vs that retrieved. The plot consideronly the values at the ECMWF analysis canonical hour and correspond to thesample shown in Fig. 75 converged χ2 are shown. . . . . . . . . . . . . . . . . . 111

LIST OF FIGURES 8

78 Calculated and observed radiances along with the related spectral residual forthe the three SEVIRI window channels used for the retrieval analysis shownin Fig.75. The error confidence interval shown in figure refers to the SEVIRINEDN. 1 r.u.=1 W m−2 sr−1 (cm−1)−1 . . . . . . . . . . . . . . . . . . . . . . 112

79 Kalman filter. Retrieval analysis for emissivity. The retrieval has been spatiallyaveraged over the grid box of size 0.05× 0.05 degrees shown in Fig. 47. . . . . 113

80 Exemplifying the difference with two different implementation of the Kalmanfilter. The difference shown in figure corresponds to the two following implemen-tations, 1) Kalman filter initialized each six hours with the ECMWF analysisand 2) initialized only once at t = 1. The difference 1)-2) is considered. Panela) and b) refers to the skin temperature, whereas panel c) refers to emissivity. . 115

81 Retrieval of the scaling factors for the atmospheric profiles, [T,W,Q]. The caseshown in figure applies to the Sahara desert. The results are shown for one singleSEVIRI pixels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

82 Retrieval of the skin temperature with the 2-D Kalman filter and comparisonwith the equivalent 1-D scheme. Results apply to the Sahara desert site. Datahave been spatially averaged over the SEVIRI box shown in Fig. 47. . . . . . . 119

83 Emissivity retrieval with the 2-D Kalman filter (left) and comparison with theresults obtained with the 1-D scheme. Results apply to the SEVIRI box shownin Fig. 47 and have been spatially averaged . . . . . . . . . . . . . . . . . . . . 120

84 Retrieval of the scaling factors for the atmospheric profiles, [T,W,O]. The caseshown in figure applies to the Mediterranean sea. The results are shown for onesingle SEVIRI pixels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

85 Retrieval analysis for skin temperature. a) 2-D Kalman filter; b) 1-D Kalmanfilter; c) the retrieval has been spatially averaged over the grid box of size 0.05×0.05 degrees shown in Fig. 47. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

86 Retrieval exercise for the Mediterranean target area. The figure shows the spec-tral residual for the three window channels. Results have been spatially averagedover the grid box of size 0.05× 0.05 degrees shown in Fig. 47. . . . . . . . . . . 122

87 Retrieval exercise for the Mediterranean target area. The figure shows the spec-tral residual for the four non-window channels. Results have been spatiallyaveraged over the grid box of size 0.05× 0.05 degrees shown in Fig. 47. . . . . 123

88 SEVIRI pixels pattern for the Sahara desert site showing 3 × 3 boxes used toimplement the 3-D Kalman filter. . . . . . . . . . . . . . . . . . . . . . . . . . . 124

89 Emissivity space-covariance matrix corresponding to 3× 3 spatial cluster of SE-VIRI pixels. The covariance matrix applies to the Sahara desert test site. . . . 125

90 Top left, retrieval results for the skin temperature, the results apply to one singlespatial cluster of 3× 3 SEVIRI pixel. Top right, scatter plot of the retrieved Tsvs the ECMWF analysis. the data have been averaged over the 3 × 3 cluster.Bottom left, emissivity retrieval for channel at 1149.40 cm−1 corresponding tothe first observation in the 3×3 cluster and related error interval. Bottom right,emissivity retrieval averaged over 3× 3 cluster. The results have been obtainedwith a 3-D implementation of the Kalman filter and apply to Sahara desert site(e.g. see Fig. 47. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

91 Top left: skin temperature monthly map obtained with the Kalman filter method-ology; Top right: Optimal Estimation retrieval; c) map of the difference; d)histogram of the difference. Blank areas correspond to cloudy regions. . . . . . 128

92 Top left: 8.7 µ m emissivity monthly map obtained with the Kalman filtermethodology; Top right: Optimal Estimation retrieval; c) difference Kalman-IREMIS; d) difference Kalman-IREMIS. Blank areas correspond to cloudy regions.129

LIST OF TABLES 9

93 As fig. 92, but for the channel at 10.8µ m. . . . . . . . . . . . . . . . . . . . . . 130

94 As fig. 92, but for the channel at 12.0 µm. . . . . . . . . . . . . . . . . . . . . . 131

List of Tables

1 Definition of SEVIRI channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2 Summary of the dimensionality of the basic variational form 43. The table showhow the dimensionality increase when we modify state and radiance vectors cor-responding to the reference 1-D vertical spatial case. Note that the size N in thereference case includes the surface parameters (emissivity and skin temperature).Also, note that the vector-matrix size shown in the table refrers to the full spacein which vectors and matrices are embedded. This does not necessarily coincideswith the number of elements we need to store. As an example a covariance ma-

trix of size n × n is symmetric and therefore it needs only n×(n+1)2 elements to

be represented. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3 Summary of the dimensionality of the basic Kalman filter. The table show howthe dimensionality increase when we modify state and radiance vectors corre-sponding to the reference 2-D vertical time-spatial case. Note that the size N inthe reference case includes the surface parameters (emissivity and skin temper-ature). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4 ECMWF data for the target area . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5 True and Background emissivity values at the seven SEVIRI channels used inthe retrieval exercise in simulation . . . . . . . . . . . . . . . . . . . . . . . . . 48

6 Summary of the settings for the optimal estimation scheme, which applies toretrieval exercise discussed in section 8.1. . . . . . . . . . . . . . . . . . . . . . 52

7 True and Background emissivity values at the seven SEVIRI channels used inthe retrieval exercise for sea surface . . . . . . . . . . . . . . . . . . . . . . . . . 61

8 True and Background emissivity values at the seven SEVIRI channels used inthe retrieval exercise for sea surface. In this exercise we initialize the retrievalproblem with a first guess for emissivity which differs of -5% from the trueemissivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

9 Estimation of φ1 and φ2 using either cause and effect. . . . . . . . . . . . . . . 69

10 Summary of the Kalman filter settings concerning the exercise discussed in sec-tion 9.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

11 Summary of the settings for the optimal estimation scheme, which applies tothe July 2010 case study shown in section 10. . . . . . . . . . . . . . . . . . . . 82

12 Summary of the settings for the Kalman Filter scheme, which applies to theJuly 2010 case study shown in section 10. . . . . . . . . . . . . . . . . . . . . . 82

13 Comparison of the retrieved and IREMIS emissivity for the Spanish location. . 84

14 Comparison of the retrieved and IREMIS emissivity for the Sahara desert location. 85

15 Summary of the settings for the optimal estimation scheme, which applies tothe July 2010 case study for sea surface shown in section 10.2. . . . . . . . . . . 103

16 Summary of the settings for the Kalman Filter scheme, which applies to theJuly 2010 case study for sea surface shown in section 10.2. . . . . . . . . . . . . 103

17 Summary of the settings for the 2D-Kalman Filter scheme, which includesatmospheric parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

0 SUMMARY 10

0 Summary

This report describes the activity we have performed within the project EUM/CO/11/4600000996/PDW whose main objective has been

• to study and formulate a general strategy to apply spatial and temporal constraints to theestimation of geophysical parameters from radiance measurements made from geostation-ary platforms, to apply the strategy to a particular example problem and to recommenda way forward a more general application to Meteosat Third Generation infrared data.

To date, the retrieval of geophysical parameters from satellite observations largely relies ona-priori information, which is derived from climatology and/or models for Numerical WeatherPrediction. Thus, the retrieval problem can be efficiently analyzed within the broad contextof data assimilation, which is indeed the paradigm of the many seemingly different methods,which have been developed over the past years. The Bayes theorem and its formalism offer aunifying framework for all these techniques, indeed Variational methods, Optimal Estimation,Kalman Filter, Kriging or optimal spatio-temporal interpolation end up with the same formalsolution as far as the estimate of geophysical parameters is concerned. This unified pictureallowed us to pick up the most general methodologies in order to better take into account forthe inclusion of spatio-temporal constraints.

As in any general setting of data assimilation, we have the data and the model. Within thisproject emphasis is given to the data, much more than the model. In our context, the model isnot necessarily a physically-based, deterministic set of equations of motion. It is thought of asa way to accommodate within the retrieval process, e.g., different time-space dynamics ratherthan their precise evolution. As an example, in a problem of emissivity-surface temperatureretrieval we know that temperature evolves on a daily time scale, whereas emissivity (at leastin clear sky) remains constant on time scales shorter than a day. What we ask to our model isjust to take into account for this different dynamics. The idea is that we can rely on the datato make up for our, possibly inadequate, prior dynamical model.

Concerning the problem of which algorithm is better suited to accommodate and exploitspatio-temporal data, we have found that while the Kalman filter and optimal estimation withthe strategy of accumulating the observation during the day are the most natural tools toconsider time-evolution, spatial variability is still to be externally included through a suitabledefinition of the background covariance matrix, which inevitably increases the dimension of thestate vector. Thus, in the end the problem is fundamentally one of data-dimension reduction.Nowadays, this problem is mostly dealt with suitable orthogonal transforms, which can allow usto compress much of the data and state vector information in few coefficients of the expansion.

The activity of the project has been divided into three phases.

The first phase has been mostly dedicated to perform a review of state of art methodologiesand tools concerning the exploitation of spatial and temporal variability to improve retrievalproducts from satellite observations of infrared radiances. The study has been mostly aimedat defining a suitable retrieval methodology for the infrared channels of the Meteosat SecondGeneration (MSG) SEVIRI (Spinning Enhanced Visible and Infrared Imager) and in perspectivefor the METEOSAT Third Generation (MTG) Infrared Sounder. Finally, a retrieval case studyfor surface emissivity and temperature has been identified, to exemplify the strategy of how toconsider the spatio-temporal variability of the data.

The second phase has been mostly dedicated to develop the baseline retrieval methodologyand related tools to estimate surface temperature and emissivity (Ts, ε) from SEVIRI observa-tions in the infrared. Two tools have been developed and implemented, one based on Optimalestimation with the strategy of accumulating the observation during the day and the other onebased on Kalman filter. The performance of the two tools have been assessed with case studiesin simulation and with SEVIRI real observations.

0 SUMMARY 11

The third and final phase of the project has been dedicated to complete the elaborationregarding emissivity/surface temperature retrieval for the North Africa/Spain target area andto extend the implementation of the software to include

• Kalman filter for sea surface (persistence state equation)

• Kalman filter (implementation for the case [Ts, ε, T,Q,O])

• Kalman filter [Ts, ε] with the introduction of localized spatial constraint

More precisely within the project we have

• defined and implemented a radiative transfer model for SEVIRI capable of taking intoaccount for the back reflected radiation to the satellite in the case of Lambetian andspecular reflection;

• reviewed state-of-art methodology concerning the exploitation of spatial and temporalvariability to improve retrieval products from geostationary satellite observations of in-frared radiances. The review has included the critical analysis of

– optimal estimation,

– likelihood Maximum estimation,

– variational methods,

– optimal spatial interpolation,

– kriging,

– Kalman filter,

– Ensemble Kalman filter,

– kriged Kalman filter,

– fixed rank kriging,

– fixed rank filtering;

• developed two methodologies to include temporal constraint in the retrieval process ofsurface temperature and emissivity from SEVIRI radiances

– one method (hereafter OE) exploits accumulation of the SEVIRI data during a cyclewhose size (day, several hours) may be decided by the user (skin temperature andemissivity are simultaneously retrieved for all the accumulated times),

– the other method is based on the Kalman filter (hereafter KF), which allows us toreduce the dimensionality of the problem because of the sequential use of the data,

• designed and developed a case study and acquired/developed/implemented all the relatedingredients to run it: SEVIRI data, software, computing resources. These include

– SEVIRI observations for July 2010 over a target area covering Spain, part of theNorth-West Africa and a portion of the Western Mediterranean basin,

– retrieval target area: that defined in Fig. 7

– retrieval methodology: that described in section 3.2.2 and section 4.3;

• built up a suitable background matrix of the emissivity vector using the University ofWisconsin IREMIS data base (e.g. see [2]),

• assessed the performance of the two tools, OE and KF both in simulation and with realobservations

0 SUMMARY 12

– for the Optimal Estimation methodology we have exemplified various strategies toperform the retrieval, these include

∗ different time slot widths on which consider the accumulation of observations,

∗ static background covariance matrices with fully dpendent/independent or cor-related state vectors, either for temperature and emissivity,

– for the Kalman filter, we have considered two different models for the dynamics ofsurface temperature

∗ an autoregressive model of the second order;

∗ a simple persistence model.

In the first case we have added to the scheme the complexity of the estimation of theautoregressive coefficients, but we have shown that this estimation can be obtaineddirectly from the SEVIRI observations. In the second case, we have used a simplepersistence model since, at least for the case of skin temperature and emissivity, theinformation content of the observations is very high, which lead us to conclude thatwe can use a simplistic state equation and rely mostly on the data.

• included a test example for sea surface;

• extended the Kalman algorithm to include [T (p), O(p), Q(p)];

• introduced localized spatial constraint in the Kalman filter methodology;

• performed the run for the whole month of July for both the methodologies, OE and KF.

On overall we can say that the Kalman filter couples Optimal Estimation with the possi-bility to sequentially process the observations, which reduces the dimensionality of the retrievalsystem. Another advantage of the Kalman filter is its capability to deal with unequally spacedtimes, that is with time sampling interval, ∆t which has not to be necessarily a constant. Thisallows us to jump over cloudy time periods and process only clear sky observations, without theneed to re-initialize the filter. However, in this case information coming from time-continuityis partly lost.

Concerning the performance of the two methodologies, we have that, as far as, the retrievalof skin temperature is concerned, OE and KF are almost equivalent, although a slight bias(of order of 1 K) between the two still persist even on monthly averages. OE and KF fairlycompare with ECMWF analysis for sea surface. ECMWF analysis for sea surface shows a slightwarm bias, which by the way is below 0.5 K. For land surface, OE and KF agree fairly goodwith ECMWF for nighttime observations, but at midday ECMWF shows a cold bias, whichcan achieve 10 K and more. This is the case for Spain and Sahara desert.

For emissivity, we have that the comparison with the IREMIS data base for the same dateand location is fairly good. OE stays closer to IREMIS, whereas KF seems to add independentinformation of that contained in the IREMIS data base.

Coming to the problem of dimensionality of the two methodologies, KF is superior overOE because of the aforementioned KF capability of sequential processing of the data. However,for both methodologies spatial constraints can be consistently introduced through a suitabledefinition of the state vector and related background. For an instrument such as SEVIRI, whichhas only eight spectral channels, dimensionality is strongly affected from the way we modifythe state vector to accommodate spatial information. As an example, if we introduce localizedhorizontal-spatial constraints by considering SEVIRI pixels clusters of size n × n, both stateand data vectors will grow by n2 and the related covariance matrices by n4.

If we look in perspective at Meteosat Third Generation (MTG) infrared observations, inprinciple, we have that the size of the radiance vector will put severe constraints even tothe inclusion of localized spatial constraints. However, if we consider that the observational

0 SUMMARY 13

covariance matrix is expected to be nearly diagonal, the use of the sequential updating numericalapproach will make it possible to use the 2-D Kalman filter (time dimension×vertical spatialdimension) and therefore to fully exploit time continuity of the observations.

As far as spatial constraint are considered, for MTG, a valid option could be to use ECMWFanalysis (atmosphere and surface) for the state or model equation. In this way spatial infor-mation would be conveyed into the scheme by ECMWF analysis.

An alternative could be to convey spatial information in a statistical form by consideringsuitable background covariance matrices of atmospheric spatial fields, derived by climatologyor ensemble of profiles. However, even for this case the size of the data vector would put heavylimitations to the real possibility of nearly real time processing of the data. At this sage, basedon the many studies performed for the retrieval of future MTG infrared observations, we cansay that an effective strategy to reduce the dimensionality of the retrieval process is to resort tosuitable orthogonal transforms, which in principle could be extended to the data and parameterspace as shown in [33].

1 INTRODUCTION 14

1 Introduction

Infrared instrumentation on geostationary satellites is now rapidly approaching the spectralquality and accuracy of modern sensors flying on polar platforms. Currently at the core of EU-METSAT geostationary meteorological programme is the Meteosat Second Generation (MSG).However EUMETSAT is preparing for Meteosat Third Generation (MTG). The capability ofgeostationary satellites to resolve the diurnal cycle and hence to provide time-resolved se-quences or times series of observations is a source of information which could suitably constrainthe derivation of geophysical parameters. Nowadays, also because of lack of time-continuity,when dealing with observations from polar platforms, the problem of deriving geophysical pa-rameters is normally accomplished out by considering each single observation as independentfrom past and future events. For historical reasons, the same approach is currently pursuedwith geostationary observations, which are still now dealt with as they were polar observations.

In contrast, our study aims at studying, formulating and implementing a general strategyto apply spatial and temporal constraints to the estimation of geophysical parameters fromradiance measurements made from geostationary platforms.

The fact that time continuity of the observations brings much information about atmo-spheric processes is normally evidenced by the pronounced short-term variability of geophysicalparameters. This lead us to conclude that atmospheric quantities can be characterized by anintrinsic dynamical correlation, which in many instances can be modeled with Markov chainsor Markov stochastic processes (e.g., [12, 44] and references therein).

Of course, dynamical correlation might be introduced by using a full dynamical NumericalWeather Prediction (NWP) system. However, apart from the fact that this in not withinthe EUMETSAT mandate, the objective of this study has been to get better insight intounderstanding how we can have a better exploitation of satellite data per se, in other wordsthe analysis moves within a context, which envisages an almost entirely data-driven system.

The issue of using dynamical correlation or evolution of the observations call for stateor evolutionary equations, which can be modeled with simply stochastic recursive equations.This brings us inevitably to the Kalman filter (e.g., [24, 25, 40]). With the Kalman filterwe can introduce time constraint via a suitable dynamical system, which describes how thegiven atmospheric state vector evolves over time. It is important to stress that the dynamicalsystem is mostly used to convey within the estimation process time-space information, which,within the Gaussian assumption, is normally done in terms of first and second order statistics:mean and covariance matrices. It is well known that the Kalman filter has optimal theoreticalproperties since it yields the best linear solution to the estimation of evolution of equations.

Since the Kalman filter has the Markov property [40], the current update or estimation ofstate vector depends only on the current state (a-priori information) and the observations. Inother words, unless we want to make predictions (that is to make forecasts much ahead than thecurrent state), the precise form of the evolutionary equation is not important for the estimationproblem, but the current state and its statistics. When we focus to the current update orestimation of the state vector, the Kalman filter is formally equivalent to the Rodgers’ OptimalEstimation [41], where the current state plays the role, as said, of a-priori information (althoughit has to be stressed that a substantial difference is that in Rodgers’ Optimal Estimation thea-priori covariance is not propagated based on the dynamics). Furthermore, it will be shown inthis document that a formal equivalence exists also with variational analysis and the MaximumLikelihood Estimation approach thanks to the Bayes theorem and its formalism [53, 40]. Indeedit offers a unifying framework for all the aforementioned techniques that end up with the sameformal solution as far as the the estimate of geophysical parameters is concerned and differencesconcern mostly the way they deal with spatial and temporal information.

In a more general setting in which the state vector is a given atmospheric field, we have thatthe Kalman filter recovers as a particular case the ordinary Kriging [8] for optimal interpolation

1 INTRODUCTION 15

on a 2-D surface or 3-D spatial domain. In other words, Kalman filter generalizes OptimalEstimation and Ordinary Kriging to the four dimensions [53].

There are two major difficulties in implementing the standard Kalman Filter in geophysicaldata assimilation problems. They are handling large matrices and accounting for nonlineardynamics. Typically global or high resolution problems have large state vectors, which can be aslarge as to count millions of elements. Dimensions of this size prohibit computing or storing theelements of the covariance matrices from the update and forecast steps and so it is not possibleto implement the Kalman Filter exactly for large problems. Possible alternatives pursued untilnow consist either in simplifying the state vector covariance matrix in a convenient form (e.g.this is the case of the Ensemble Kalman filter [18]), or to perform a suitable dimensionalityreduction of the state vector space (e.g., this is the approach of the Kriged Kalman filter andFixed Rank Filtering [10, 11, 26]).

The basic idea of an ensemble Kalman filter is to use a sample of states to approximatethe mean vectors and covariance matrices. Each ensemble member is updated by an approx-imation to Bayes theorem and is propagated forward in time and giving a new ensemble forapproximating the forecast distribution. This idea was proposed by Evensen [18] but has beendeveloped by many subsequent researchers. It is a form of particle filter [53] with the ensemblemembers being ”particles”. One departure from standard particle filtering is that the ensemblemembers are modified at every update step rather than just being reweighted.

The kriged Kalman filter has been introduced by [31], whereas the fixed rank filteringhas been developed and proposed quite recently by [10, 11, 26]. The basic idea in these lastapproaches is to reduce the dimensionality of the parameters space by introducing a suitablestochastic form for the state vector.

In the past year the idea of the ensemble Kalman filter has evolved mainly between theNWP community and therefore it has been mostly developed for data assimilation problemswhere we consider to use a suitable NWP model for the evolution of the state vector. Incontrast, the kriged Kalman filter and the fixed rank filtering have both evolved in the contextof geostatistics and they tend to substitute the complex NWP model with a suitable stochasticmodel. Thus, the idea underlying these last two methods is most likely more attractive for thepurpose of our objective to develop a data-driven system also if it is not directly applicable toproblems that involve the use of level 1 data for the retrieval of geophysical parameters.

According to aforementioned considerations, the activity of the project has been dividedinto three phases. In the first phase we have been mostly concerned in pursuing the followingtasks

• Task 1: Develop a conceptual method or methods which explore the full spatial and tem-poral dimensions of geostationary satellite observations in the extraction of geophysicalparameters from these observations under the consideration that ultimately a practicalimplementation in EUMETSAT operational product extraction chain should be feasible.

• Task 2 Determine how and to what extent the method(s) will affect information uniformly,account for errors correlated horizontally, vertically and temporally and account for softand hard boundaries. In other words, fully define the retrieval covariance matrix

• Task 3 Provide a qualitative comparison between the concept developed under Task 1and alternative potential solutions.

In order to establish some guidelines for the exploitation of spatio-temporal models for theanalysis of geostationary data we first have explored the introduction of temporal constraintfocusing our attention on the development of the two following strategies

1. Optimal estimation with the strategy of accumulating the observation during the day;

1 INTRODUCTION 16

2. Kalman filter with different dynamics for emissivity and temperature and, of course,exploiting the time-sequential characteristic of the tool.

In particular the Kalman filter has been developed considering two different models for thedynamics of skin temperature: a simple persistence scheme, and an autoregressive process ofthe second order. In the first case, also if the model equation is not correct since it cannotreproduce the daily cyclic behavior expected in the clear sky for land surface, we know thatskin temperature is strongly driven by the data and the observations provide a tidy constraintfor it. In the second case, the physics underlying the process says that the dynamics has to befitted with an autoregressive process of the second order, whose parameters can be estimatedagain from the observations. In both cases the use of dynamics is somewhat of a practicaldecision, a way to accommodate time-continuity within the retrieval process, and not a forecastmaking use of precise physical evolution equations, also because the observations permits tobuild a completely data-driven system. Indeed in many instances it is easier to specify realisticconditional models rather than the full joint structure, particularly when one is trying to accountfor specific type of process behavior, as pointed out in [54].

The implementation and testing of the proposed techniques on a suitable test case has beencarried out in the second phase of the project, that has been mostly dealing with the followingtasks

• Task 4 Develop suitable methods to apply temporal, and potentially spectral, constraintsto the problem of land surface emissivity. Temporal constraints have been considered forthe basic parameters (Ts, ε) and for the case where all parameters, Ts, ε, T (p), O(p), Q(p)with T (p), O(p) and Q(p) being Temperature, Ozone and Humidity profiles, respectively,are considered.

• Task 5 Apply the methods to MSG SEVIRI infrared channels over a suitable time periodand area.

• Task 6 Evaluate the results: theoretical error estimates, heuristic analysis and, wherepossible, validation / comparison to other measurements

The inclusion of possible spatial constraints have been developed in the third phase of theproject, that has been concentrated on the following tasks

• Task 7 Finish the elaboration regarding emissivity/surface temperature retrieval for theNorth Africa/Spain target area with optimal estimation endowed with the strategy ofaccumulating the observation during the day and the Kalman filter.

• Task 8 Implement the Kalman filter for sea surface (persistence state equation).

• Task 9 Implement the Kalman filter (implementation for the case [Ts, ε, T,O,Q].

• Task 10 Implement the Kalman filter [Ts, ε] with the introduction of localized spatialconstraint.

The report is organized as follows.

First, in section 2 we review the infrared radiative transfer in clear sky, in order to develop aforward model for the SEVIRI infrared channels, which can take into account for the radiationterm back-reflected to the satellite from the surface. We explicitly take into account Lambertianand specular reflection at the surface.

Sections 3, 3.2, 6 review the state of art concerning Variational methods, the Kalmanfilter, the Ensemble Kalman filter, the spatio-temporal optimal interpolation, respectively. Thereview is not intended to be necessarily exhaustive of the many contributions to the area ofdata assimilation. For this we recommend the work by Wikle and Berliner [53]. Some details

1 INTRODUCTION 17

are intentionally left out of this review in favour of the general framework and setting of thetheory and the unifying role played by the Bayesian methodology.

To exemplify the strategy of how to consider the spatio-temporal variability of the datawe have defined a suitable case study, which exploits SEVIRI observations in the infrared.The case study has been implemented both in the context of variational analysis (presented insection 3.2.2) and in the context of KF (presented in section 4.3). The implementation is quitegeneral and can be applied to a generic geostationary imager or sounder and in perspective forthe METEOSAT Third Generation (MTG) Infrared Sounder.

Details about, the target area for the case study and the ancillary information used withinthe retrieval exercises are given in section 7.

Section 8 describes the implementation of the Optimal Estimation approach and its appli-cation to a case in simulation, whereas section 9 present two possible implementations of theKalman filter. In section 9.1 we deals with a state equation for skin temperature, which isbased on suitable autoregressive processes, whereas in section 9.5 a simple persistence modelfor the state equation of skin temperature is discussed.

In all implementations of both Optimal Estimation and Kalman filter, we consider a simplepersistence model for emissivity.

Application to real SEVIRI observations is shown in section 10. Results concerning dailymaps of emissivity and temperature are shown in section 13.

Concluding remarks and recommendations towards a more general application to MeteosatThird Generation infrared data are shown in section 14.

2 THE FORWARD MODEL EQUATION 18

2 The forward model equation

In a plane parallel atmosphere, the radiative transfer equation, which is suitable for the presentproblem of retrieving surface and atmospheric parameters from nadir-looking satellite thermalinfrared observations can be written according to (e.g. [21])

R(θr, φr, σ) = ε(θr, φr, σ)τo(θr, φr, σ)B(Ts) +Ru(θr, φr, σ) +Rr(θr, φr, σ) (1)

where

(θr, φr) : is the viewing (zenith, azimuth) coupleσ : is the wave numberTs : is the surface temperatureε is the surface emissivityB : is the Planck functionτo : is the total transmittanceRu : is the upwelling atmopsheric radianceRr : is the downwelling-back-reflected to the satellite atmopsheric radiance

(2)

In Eq. (1) the dependence of B over the wave number is implicit.

The upwelling radiance term, Ru has the form

Ru(θr, φr, σ) =

∫ +∞

0

B(T )∂τ

∂hdh (3)

with h the vertical spatial coordinate and τ the transmittance from h to +∞.

The down welling term, Rr deserves more care, because it strongly depends on the optical(reflectance) properties of the surface. These properties are normally accounted for throughthe bidirectional reflectance distribution function (BRDF). An analytical expression of Rr canbe obtained based on somewhat simplistic assumptions for the BRDF. Two cases, specular anddiffuse (Lambertian) reflection, are normally dealt with in simplified analytical derivation forRr. These two cases are quite often confused in much science literature, therefore appropriateexpressions for Rr for specular and Lambertian reflection will be explicitly derived in theremaining of this section.

The most general expression for the down welling, back-reflected, atmospheric emissionterm may be written as follows (e.g., [48])

Rr(θr, φr, σ) = τo(θr, φr, σ)

∫ 2π

0

dφi

∫ π2

0

f(θr, φr, θi, φi, σ)Ri(θi, φi, σ) cos(θi) sin(θi)dθi (4)

with

(θi, φi) : incident direction to the surface, zenith and azimuth angles, respectivelyf : the bidirectional reflectance distribution function (BRDF)Ri : the spectral radiance incident at the surface from the direction (θi, φi)

(5)

2.1 The case of a Lambertian Surface

For a Lambertian surface f is equal to 1/π times the directional-hemispherical spectral re-flectivity; the latter is equal, because of the reciprocity law to the hemispherical-directionalspectral reflectivity, that is (e.g. [48])

f(θr, φr, θi, φi, σ) =1

πρd−h(θi, φi, σ) =

1

πρh−d(θr, φr, σ) (6)


withρd−h : the directional-to-emipsherical reflectivityρh−d : the emipsherical-to-directional reflectivity

(7)

where we have tacitly assumed that Lambertian reflectivity’s are independent of the angle ofincidence (ρh−d), and angle of reflection (ρd−h).

Using Eq. (6), Eq. (4) can be written according to

Rr(θr, φr, σ) =τo(θr, φr, σ)ρh−d(θr, φr, σ)

π

∫ 2π

0

dφi

∫ π2

0

Ri(θi, φi, σ) cos(θi) sin(θi)dθi (8)

which using Kirchoff’s law, ρh−d(θr, φr, σ) = 1 − ε(θr, φr, σ), with ε the spectral emissivity(note that for a pure diffuse surface the hemispherical-directional reflectivity and, hence, theemissivity does not depend on the reflection direction, therefore the dependence on the angles(θr, φr) of the emissivity will be dropped in the following formulas). With this in mind thereflected spectral radiance can be written further as

Rr(θr, φr, σ) = τo(θr,φr,σ)(1−ε)π

∫ 2π

0dφi

∫ π2

0Ri(θi, φi, σ) cos(θi) sin(θi)dθi

= 2τo(θr, σ)(1− ε)∫ π

2

0Ri(θi, φi, σ) cos(θi) sin(θi)dθi

(9)

where, in deriving the last line of the above equation, we have assumed independence over theazimuth, both for incidence and reflection directions. Furthermore, defining µ = cos(θi), wehave

Rr(θr, σ) = 2τo(θr, σ)(1− ε)∫ 0

1

Ri(µ, σ)µ(−dµ) (10)

The spectral radiance Ri can be explicitly written as

Ri(µ, σ) =

∫ 0

+∞B(T )

∂τ∗(µ, h)

∂hdh (11)

with, τ∗ the transmittance from the altitude level h to h = 0 along the slant path in thedirection, µ. The top-to-bottom transmittance, τ∗ should not be confused with the bottom-to-top transmittance, τ . In case the two refer to the same slant path, we have ττ∗ = τo. InsertingEq. (11) into Eq. (10), we have

Rr(θr, σ) = 2τo(θr, σ)(1− ε)∫ 1

0

∫ 0

+∞B(T )∂τ∗(µ,h)∂h dhµdµ

= τo(θr, σ)(1− ε)∫ 0

+∞B(T ) ∂∂h

(2∫ 1

0τ∗(µ, h)µdµ

)dh

(12)

which defining the slab or diffuse transmittance (e.g. [30])

τf∗ (h) = 2

∫ 1

0

τ∗(µ, h)µdµ (13)

becomes

Rr(θr, σ) = τo(θr, σ)(1− ε(σ))

∫ 0

+∞B(T )

∂τf∗∂h

dh (14)

According to [17] it is postulated that

τf∗ (µ, h) = τ∗(µ, h) (15)

That is the diffuse transmittance can be calculated as the transmittance function at a suit-able cosine angle; the term 1/µ is referred to as the diffusivity factor. For practical calculations,the value 1/µ =1.66 yields accurate results. This corresponds to an effective zenith angle of52.96 degrees.

Omitting the dependence on angles and wave number, the complete upwelling radianceterm reaching the satellite can be written for a Lambertian model of surface as

R(σ) = ετoB(Ts) +

∫ +∞

0

B(T )∂τ

∂hdh+ (1− ε)τo

∫ 0

+∞B(T )

∂τf∗∂h

dh (16)


2.2 The case of Specularly Reflecting Surface

Taking into account that for specular reflection we have (θr, φr) = (−θi,−φi), we have fromEq. (4)

Rr(θr, φr, σ) =1

πρh−d(θr, φr, σ)τo(θr, φr, σ)Ri(−θr, φr, σ)

∫ 2π

0

dφi

∫ π2

0

cos(θi) sin(θi)dθi (17)

with 1πρh−d(θr, φr, σ) the hemispherical-directional reflectivity for uniform irradiation (here

appropriate because of the highly diffuse nature of the thermal radiation field). The integrationover the hemisphere gives π and, considering Kirchoff’s law, we have

Rr(θr, φr, σ) = ρh−d(θr, φr, σ)τo(θr, φr, σ)Ri(−θr, φr, σ)

= τo(θr, φr, σ)(1− ε(σ))∫ 0

+∞B(T )∂τ∗∂h dh(18)

Once again, omitting the dependence on angles and wave number, the upwelling spectral radi-ance reaching the satellite can be written for a specular model of surface as


∫ +∞

0

B(T )∂τ

∂hdh+ (1− ε)τo

∫ 0

+∞B(T )

∂τ∗∂h

dh (19)

In contrast with the Lambertian surface, it should be noted that for specular reflection theupward and downward integrals have to be evaluated along the same path, therefore, consideringthat ττ∗ = τo, we have that Eq. (19) can be also put in the from,


∫ +∞

0

B(T )∂τ

∂hdh+ (1− ε)τ2o

∫ +∞

0

B(T )1

τ2∂τ

∂hdh (20)

2.3 σ-SEVIRI

SEVIRI infrared channels range from 12 µm to 3.9 µm. Their conventional definition in termsof channel number is given in the table below, whereas their spectral response is shown in Fig.1. The figure also provides a comparison with a typical IASI spectrum at a spectral samplingof 0.5 cm−1.

We see from Fig. 1 that the first channel takes contribution form a wide coverage of thespectrum. This coverage include the CO2 head band at 4.3 µm and the part of the atmosphericwindow in the shortwave.

Table 1: Definition of SEVIRI channels

Channel Number wave number (cm−1) wave number (µm1 2564.10 3.92 1612.90 6.23 1369.90 7.34 1149.40 8.75 1030.9 9.76 925.90 10.87 833.30 12.08 746.30 13.4

There are various reason against the use of the SEVIRI channel 1 in a retrieval scheme:1) it is too much broad; 2) it is contaminated from solar emission in daytime; 3) it is affectedfrom non-LTE effects during daytime; 4) the CO2 line mixing at the 4.3 µm CO2 band head


Figure 1: SEVIRI channel spectral response over-imposed to a typical IASI spectrum.

is poorly modeled in state-of-art radiative transfer, which can add a potential large bias. Forthis reason we have not developed a radiative transfer model for channel 1, which will be notused for all the applications and case studies developed in this project.

Regarding the channels from to 2 to 8, the forward model has been derived from theσ−IASI monochromatic radiative transfer designed for fast computation of spectral radianceand its derivatives (Jacobian) with respect to a given set of geophysical parameters. We willrefer to the SEVIRI forward model as σ-SEVIRI. As σ-IASI, σ-SEVIRI adopts a grid of 60pressure layers [1013.00-0.005 hPa] and is based on suitable look-up table of optical depth.

The radiative transfer equation, which σ-SEVIRI integrates has been largely explained inthe first progress report [46]. In this document we will limit ourselves to describe some technicalaspects relevant to the computational algorithm embedded in σ-SEVIRI.

As for σ-IASI the look-up table for σ-SEVIRI has been developed from one of the mostpopular line-by-line forward model, that is LBLRTM [6]. Actually, the σ-SEVIRI look-up tableis obtained by down-scaling the wave number sampling rate of the look-up table for σ-IASI.For this reason we need to give some details about σ-IASI in order to describe how σ-SEVIRIworks.

The σ-IASI model [1] parameterizes the monochromatic optical depths with a second orderpolynomial. At a given pressure, the Optical depth for the a generic i-th molecule, is computedaccording to

χσ,i = qi(cσ,0,i + cσ,1,iT + cσ,2,iT

2)

(21)

where T is the temperature, qi the molecule concentration and cσ,j,i with j = 0, 1, 2, are fittedcoefficients, which are actually stored in the Optical depth look-up-table.

Unlike other gases, for water vapour, in order to take into account effects depending on thegas concentration, such as self-broadening, a bi-dimensional look-up-table is used [32]. Thus,for water vapour, identified with i = 1, the optical depths is calculated according to

χσ,1 = q1(cσ,0,1 + cσ,1,1T + cσ,2,1T

2 + cσ,3,1q1)

(22)


The subscript σ indicates the monochromatic quantities. In the case of hyperspectral instru-ment, such as IASI, the monochromatic optical depths are computed and parameterized at thespectral sampling of 10−3 cm−1.

This spectral sampling is too much fine for band instrument such as SEVIRI. In the caseof SEVIRI the spectral sampling can be averaged and sampled at a rate of 10−1 cm−1 withoutsacrificing accuracy. Also in this case the optical depth can be parameterized with a low orderpolynomial and its coefficients are obtained as explained in the next section.

2.3.1 Averaging and down-sampling the look-up-table: from σ-IASI to σ-SEVIRI

For each species i we can define an equivalent Optical depth χ〈σ〉,i, which can be parameterizedwith respect to temperature in the same way we do for monochromatic quantities (Eq. 21,22).

In the following of this section the angular brackets, 〈·〉 will be used to indicate the averageoperation over the spectral range identified by the SEVIRI-coarse sampling.

The equivalent Optical depth is

χ〈σ〉,i = qi(c〈σ〉,0,i + c〈σ〉,1,iT + c〈σ〉,2,iT

2)

(23)

where the equivalent coefficients c〈σ〉,j,i with j = 0, 1, 2, are obtained by fitting the layer trans-mittance averaged over the coarse resolution.

qi(c〈σ〉,0,i + c〈σ〉,1,iT + c〈σ〉,2,iT

2)

= − log (〈τσ,i〉) = − log [〈exp (−χσ,i)〉] (24)

In order to test the effectiveness of this strategy applied to the Infra-Red Channels ofSEVIRI instrument, we computed and compared SEVIRI radiances with the fine mesh look-uptable and that with a coarse mesh.

The results are shown in Fig. 2, which shows that the coarse-mesh look-up table performsas good as the fine-mesh look up table. The difference is contained below 0.8% and is nearlyzero in atmospheric window channels.

Because of this down-sampling σ-SEVIRI based on the coarse look up table runs 1000 timesfaster that the σ-SEVIRI working with the fine-mesh look-up table.

2.3.2 Check of the quality of the analytical Jacobian scheme

The forward module has an analytical Jacobian scheme for the atmospheric parameters. Inthe present version analytical Jacobian matrices or vectors are provided for emissivity, skintemperature, temperature, water vapour, ozone, carbon dioxide. A check of the quality andsanity of the scheme has been performed by considering calculations of linearized radiancesunder a small perturbation of the given parameter.

As usual let R and v be the radiance vector and the state vector, respectively. Let F bethe forward model, so that we have R = F (v). Let us consider a small perturbation, ∆v. Theradiance corresponding to this perturbation is Rp = F (v + ∆v) and its linear approximation,Rlin can be computed according to

Rlin = F (vo) +∂F (v

∂v|v=vo ∆v (25)

where vo is the unperturbed d state vector.

The consistency of the Jacobian scheme is exemplified for the temperature and water vapourprofiles form Fig. 3 to Fig 5. Figure 3 shows the unperturbed state vector for temperature andwater vapour. Figure 4 shows Rp, Rlin and Rp − Rlin for a temperature perturbation at eachlayer of ±0.1 K, whereas Fig. 5 corresponds to a perturbation of water vapour of ±1% at each


Figure 2: Consistency between σ-IASI and σ-SEVIRI. The upper panel shows the σ-SEVIRIradiance for the Infra-red channels 2 to 8 (6.2 to 13.4 µm). The calculations refers to a blackbody surface emissivity and Lambertian diffusive and reflective surfaces with constant emissivityequal to 0.9. The Bottom panel shows the percentage differences between σ-IASI and σ-SEVIRIcalculations.


Figure 3: Unperturbed state vector, vo, a) temperature profile, b) water vapour profile.

layer. It is seen that the difference Rp − Rlin is zero within machine accuracy. Similar resultshold for the other surface and atmospheric parameters.

Of course this does mean that we are sensitive to changes of ±0.1 K in temperature and±1% variations in water vapour. In a real situations, the effect of noise would swamp out thistiny signals.


Figure 4: Consistency check of the temperature Jacobian for a perturbation of the temperatureprofile of −0.1K at each layer (panel a)), and +0.1K (panel b)).

Figure 5: As Fig. 4, but now for the water vapour profile, panel a) considers a perturbation of-1% of the profile at each layer, the case of +1% perturbation is shown in panel b).

3 THE GENERAL RETRIEVAL FRAMEWORK: STATIC A-PRIORI BACKGROUND 26

3 The general retrieval framework: static a-priori back-ground

In practice the problem we want to address is that of deriving the thermodynamical stateof the atmosphere, at a given time t given a set of independent observations of the spectralradiance, Rt(σ), where the underscript t indicates that the radiance can depend itself on time.If the spectral radiance is observed at different wave numbers, σi, i = 1 . . .M , then the radiancevector, Rt is defined according to

Rt = (Rt(σ1), . . . , Rt(σM ))T

(26)

where the superscript T means transpose.

It is now well established that data assimilation [53] provides a good paradigm for the manymethods we are going to review. In particular the methods that consider the a-priori covarianceas a static application data, that is Rodgers’s Optimal Estimation [41], 1D to 3D variationalanalysis [7, 50], and the Maximum Likelihood Estimation approach are formally equivalent[53]. Therefore, in reviewing the basic aspects of the retrieval problem with a static a-priori,we approach the issue from the variational perspective, since it has a more straightforwardgeneralization to the fourth time-dimension.

Under the assumption of multivariate normality our retrieval problem can be seen as one ofvariational analysis in which a suitable estimation of the state vector is obtained by minimizingthe form (see e.g. [5, 7, 50, 51]

minv

1

2(Rt − F (v))

TS−1ε (Rt − F (v)) +

1

2(v − va))

TS−1a (v − va)) (27)

Of course the factor 1/2 is not essential for the minimization, it is just a reminder for theGaussian assumptions about the probability density functions of observations and parameters.In Eq. (27), we have

F : is the forward model functionv : is the atmospheric state vector, of size Nva : is the atmospheric background state vector, of size NSε : is the observational covariance matrix, of size M ×MSa : is the background covariance matrix, of size N ×N

(28)

For practical purposes, Eq. (27) has to be linearized in the forward model, so that its minimumcan be sought through a GaussNewton iterative sequence. Linearization is obtained throughTaylor’s series expansion of F (v) around a first guess state vector, vo

Rt = Rot +∂F (v)

∂v|v=vo(v − vo) + higher order terms (29)

where Rot = F (vo). The introduction of the Jacobian,

K =∂F (v)

∂v|v=vo (30)

allows us to replace (27) with the quadratic form

minx

1

2(yt −Kx)

TS−1ε (yt −Kx) +

1

2(x− xa)

TS−1a (x− xa) (31)

whereyt = Rt −Rot

x = v − voxa = va − vo

(32)


It should be stressed that, formally, the state vector, v can be thought of as a 3-D geophysicalfield, and not necessarily of a vector in one dimension (altitude coordinate).

The formal solution of Eq. (31) is well established and can be found in many textbooks

(e.g. [51, 41]). The estimation, x of x and its covariance matrix, S read

x = xa +(KTS−1ε K + S−1a

)−1KTS−1ε (yt −Kxa)

S =(KTS−1ε K + S−1a

)−1 (33)

In the context of data assimilation, xa is normally the forecast at time, t and Sa is theerror forecast covariance matrix. The estimation, x is referred to as the analysis.

On the other hand, in the context of ensemble retrieval, also normally referred to as cli-matology inversion, xa and Sa are the mean and covariance of an ensemble of states, which arecommittal to the problem at hand: local radiosonde observations, Numerical Weather Predic-tion centres analyses and so on.

One possible problem with climatology retrieval is that Sa may provide a too loose con-straint, in which case the solution can go closer to the unconstrained least square solution andshow jackknifing. Conversely if the constraint is too tidy, the solution can just relax on xa andget no contribution from the data points. For these cases, the introduction of a global inflationparameter, γ which properly scales up and down S−1a in Eq. (33) can help. That is we substi-tute S−1a with γS−1a . It can be shown [34] that this approach is equivalent to regularization.In addition, the approach performs much better (e.g. [5]) in case we previously rotate Sa andSε in order to work with diagonal covariance matrices. In this case, the correlation structureof the ensemble is preserved and we change variances alone.

3.1 Introducing 3-D spatial constraints

Equation (31) provides the formal setting for 3-D variational problems. However, an importantassumption in deriving the solution (33) regards the independency of observations and statevector. Also in case of field variables, the observations at two different spatial points areassumed to be independent, as well. An important concept to draw from these assumptionsis that spatial information about the distribution of x can be generated in Eq. (31) from thebackground matrix, Sa.

As said in the previous section, one straightforward way to produce effective, Sa is theensemble approach. To introduce spatial constraints the ensemble of states has to be properlygeo-located, and each states has to be located on the earth sphere (latitude, longitude, alti-tude). Although appealing, this approach can be expensive in terms of storage capacity andcomputational effort. It gives rise to very large matrices, even for very limited geographic areas.As an example if we consider a 3 × 3 MTG IRS soundings, assuming the 61 levels ECMWFmodel, we have, say for temperature, a state vector of size 61× 9 = 976 ≈ 103, which yields acovariance matrix with one million of elements. These computational effort has to be summedup to those required for water vapour. The other problem is that the quality of state-of-artNWP models cannot compare to the time-space resolution of instruments, such as MTG-IRSand even SEVIRI, therefore we could have the problem to interpolate Sa from a coarse mesh,as that provided by NWP centres, to the fine mesh of the instrument, which could introducelinear dependency, hence, non-legitimate covariance matrices. A legitimate covariance matrixhas to be positive-semidefinite (positive eigenvalues).

A possible alternative is to use a suitable model for Sa, with few adjustable parameters.Let us focus for now on the case of a univariate model

For spatially distributed x, the background covariance matrix Sa can be expressed in termsof the simple (univariate) model given by (e.g. [14])

[Sa]mnij = σ(m)σ(n)ν(mn)ρ(rij ;L

(m), L(n)) (34)


The notation means the covariance of the given variate (e.g. temperature) at horizontal gridpoint i, altitude level m and that at grid-point j and altitude n. The parameter σ(m) is thestandard deviation of the variate at altitude/pressure level m, ν(mn) is the vertical correlationbetween variate values at levels n and m, rij is the horizontal distance between nodes i and j,L(m), L(n) are horizontal decorrelation length scales. Finally, the function ρ represents spatialcorrelations. Such a model may be called quasi-separable because the vertical correlations areinvariant with respect to translation along levels of constant pressure. In fact, if we keepm = n, the function ρ represents the horizontal isotropic correlations between variates at anytwo locations on a fixed pressure level. Isotropy is assumed by considering that the function ρdoes not depend on ri and rj but only on the Euclidean distance r =|| ri−rj ||. It is importantto stress that only a special class of ρ(r) gives rise to a legitimate (i.e positive-semidefinitefixed-level covariance model on a spherical surface).

In practice, we specify the function ρ(r;L(m)) at a given pressure level m, the problem isthen posed of how to estimate correlation at any two levels m and n. What is commonly done(e.g. [14]) is to consider the average:

ρ(r;L(m), L(n)) =1

2

[ρ(r;L(m)) + ρ(r;L(n))

](35)

Normally this is accurate enough, although it may produce negative eigenvalues in Eq. (34)when either the vertical correlations are large or when decorrelation length scales vary greatlybetween levels.

With the approximation (35), model (34) is completely determined by the parametersσ(m), L(m), ν(mn) and by the choice of the analytical function, ρ(r, L(m)). If we compare thedata requirements to the example for the non-parametric covariance matrix for MTG-IRS alikeinstrument, we see that now we would need 61×61+61+61 = 3843 parameters to store againstone million.

Examples of legitimate models for ρ are discussed in the following section.

3.1.1 Valid isotropic model for ρ(r;L)

The general univariate isotropic covariance model is of the form

Cov(X(x), X(y)) = (var(X(x))(var(Y (x))ρ(|| x− y ||) (36)

where X(·) is an arbitrary scalar parameter and r =|| x − y || is the Euclidean (horizontal)distance between location x and y.

A valid form for ρ(r) is the so called power law (e.g. [14]),

ρ(r) =1

1 +(rL

)2 (37)

where L is the decorrelation length scale. This can be generalized to a fractal model by includinga non integral exponent,

ρ(r) =1

1 +(rL

)α (38)

This form of correlation is ubiquitous in nature and characterizes atmospheric processes drivenby turbulence, e.g. [39].

In a previous EUMETSAT project (EUM/CO/07/4600000398/SAT) [16, 45], we haveshown that a mixture of fractal models is much more flexible and can be used to fit corre-lation structures of temperature and water vapour, as well. The model is

ρ(r) =a

1 +(rL1

)α1+

1− a

1 +(rL2

)α2(39)


Of course, the disadvantage of this model is that it has more parameters to be fitted to thedata.

Another commonly used isotropic covariance function is the exponential function

ρ(r) = exp(−r/L) (40)

The isotropic assumption is clearly not valid for actual atmospheric fields, which generallydepend on local properties of the flow. However, the use of the isotropic univariate covariancemodels is widespread in atmospheric data assimilation [19]. The fact is that covariances re-quire an appropriate ensemble of states to be estimated and this is traditionally built up byconsidering state vectors over a long period of time, for example a season or the year. The longperiod of time is considered to ensure a sufficient statistics and, therefore, a good accuracy ofthe final estimate.

In the past years, the problem of estimating the parameters in a covariance model hasbeen dealt with the principle of Maximum Likelihood estimation (MLE) by many authors(e.g. [14] and reference therein). However, the straightforward use of MLE requires order n3

operations for a data set of size n, making these methods computationally intractable for largen. Recently, a new algorithm has been introduced by [4], which is particularly suited for largespatial datasets. In this approach, covariance matrices are tapered, or multiplied element-wiseby a sparse correlation matrix. The resulting matrices can be manipulated using efficient sparsematrix algorithms.

Another algorithm, which has found large applications in the framework of the geostatisticscommunity is the the Expectation Maximization (EM) algorithm [15, 27]. The EM algorithmis an iterative procedure that attempts to find the value of the parameters that maximize thelikelihood function. To be more precise, the EM algorithm finds a solution to the likelihoodestimating equations. The likelihood estimating equation are obtained by setting the firstderivative of the likelihood function with respect to the parameters to zero.

3.1.2 Covariance for the multivariate case

So far we have dealt with the univariate case of estimating covariance matrices. However, themost general case is multivariate, since it involves atmospheric parameters such as temper-ature, water vapour, and surface parameters such as temperature and emissivity. Normally,cross-correlation among different parameters is not taken into account, which means that thecovariance model can be fitted to one parameter at a time, and finally the diverse covariance

matrices concatenated in a single matrix. In case we have n parameters, let S(i)a , i = 1 . . . n be

the a-priori matrix for the i-th parameter. The multivariate Sa is then obtained by consideringthe block-diagonal matrix,

Sa =

S(1)a , 0, . . . 0

0, S(2)a , . . . 0

. . . . . . . . . . . .

0, 0, . . . S(n)a

(41)

3.2 Introducing time constraint

3.2.1 Optimal estimation with times accumulation

With geostationary satellites, spectral observations can be made with a high repeat time (15min in the case of SEVIRI) and these observations can be accumulated, e.g., during one dayand used simultaneously in a suitable retrieval scheme. The strategy of daily accumulation


of the 15 minutes repeat cycle of SEVIRI has been used, e.g, in [23] for the joint retrieval ofsurface reflectance and aerosol optical depth.

To streamline the exposition of how time accumulated observations can be simultaneouslyused in the variational scheme (27) let us assume that the times are indexed by integers t =1, 2, . . ., although handling unequally spaced times does not add any fundamental difficulty.Let vt denote the state vector at time t and Rt the spectral observations available at time t,then a simultaneous solution for the sequence, v1, . . . ,vt can be obtained by minimizing thecost function starting at t = 1 and going through t = NT , with NT the length of the cycle.

minv1,...,vNT

NT∑t=1

[1

2(Rt − F (vt))

TS−1ε (Rt − F (vt)) +

1

2(vt − va)

TS−1a (vt − va)

](42)

The form of this problem is inherited by 4DVAR schemes [28, 29], although in the presentproblem we minimize with respect to the whole sequence v1, . . . ,vNT and not necessarily withrespect v1 alone. What we exactly mean will be clarified through the discussion in this section.

We assume that the observational covariance stays constant with time, the same as for Sa.This last assumption is correct for short times (e.g. one day or less). In this case we can assumethat spatial properties of the field change little.

Unless we assume some dependence among the state vectors, v1, . . . ,vNT , Eq. (42) givesthe simultaneous solution of NT independent problems. The data at time ti does not contributeinformation at time tj , in case i 6= j. Thus, without a suitable constraint on the sequence ofstate vectors, Eq. (42) is just an useless formal complication.

Constraints over the state vectors sequence, can be put in many modes. As an example, if weassume, e.g., that the first i = 1, ..., n values out of the N elements (n < N) in each single statevector do not change with time, than our retrieval problem is one of estimating n+(N−n)∗NTparameters with M ∗NT spectral radiances. In this case, the information from different timesis used to solve for the n time-constant unknowns. To make a concrete example, let us considerthe seven SEVIRI channels, therefore M = 7. During the day we can assume (at least in clearsky) that the surface emissivity remains a constant (thus n = 7). With surface temperature, wehave N = n+ 1 = 8 surface parameters against M = 7 SEVIRI channels. At a given time, t wehave that the problem of retrieving the seven emissivity values at the seven SEVIRI channelsplus the temperature is under-determined: less data points (seven) than parameters (eight).Conversely, according to our time-accumulation strategy, if we accumulate observations for acycle of length, say NT = 5, we have a problem with n+ (N − n) ∗NT = 7 + (8− 7) ∗ 5 = 12parameters and M ∗NT = 7 ∗ 5 = 35 data points, which makes the problem over-determined.

In practice, the problem (42) needs to be linearized and, eventually, iterated to take intoaccount the non linearity of the forward model. The linearization around a first guess sequencevo1, . . . ,voNT lead us to the quadratic form

minx1,...,xNT

NT∑t=1

[1

2(yt −Ktxt)

TS−1ε (yt −Ktxt) +

1

2(xt − xat)

TS−1a (xt − xat)

](43)

where, as beforeyt = Rt −Rot

xt = vt − votxat = vat − vot

(44)

Equation (43) is only formal and needs some suitable relationship among the sequenceof state vectors before we seek for a minimum. However, we stress that the formal solutioncontinue to be that shown in Eq. (33). An explicit form for the solution will be worked out inthe following section.


3.2.2 A formulation of the emissivity-temperature retrieval with the strategy oftime accumulation of the observations

In this section we will provide the details of the implementation of the methodology shown inthe previous section for the retrieval of surface emissivity and temperature. The implementationis performed in order to be applied to the seven SEVIRI channels at wave numbers (cm−1):

[746.3, 833.3, 925.9, 1030.9, 1149.4, 1369.9, 1612.9]

At the present stage of the development we will assume to deal with one SEVIRI pixel ata time. The inclusion of spatial constraint will be dealt with later in this study.

We will assume that the variability of the emissivity is much slower than that of the surfacetempertaure. Thus, we will assume that on the time span t = 1, . . . , NT , the emissivity is aconstant. No restriction is posed on Ts, which can vary free with time.

Before showing how we built up the retrieval problem, we introduce a transform for theemissivity, which allows us to constrain the retrieval to the physical emissivity range of 0-1.Let ε be the emissivity at any of the SEVIRI channels, we consider the transform

e = logε

1− ε(45)

which has inverse,

ε =exp(e)

1 + exp(e)(46)

The transform maps 0-1 into the interval [−∞,+∞] and vice versa, therefore if we work withthe variable e, retrieval positiveness for ε is ensured. In order to work with the parameter e wehave to properly transform the Jacobian. It easily follows from Eq. (45) that

∂R

∂e=∂R

∂εε(1− ε) (47)

where R is radiance at a generic SEVIRI channel.

If we linearize the forward model, at time t, with respect e and Ts, we obtain

yt = Atδe + BtδTst (48)

with δe = e − eo, δTst = Tst − Tost. The matrix At is the emissivity Jacobian. This is adiagonal matrix of size n × n with n = 7, as for the case of the vector, δe. Note that thisvector is not indexed by t, since we assume that it does not vary with time. Conversely, At

can vary with time, since it also depends on the skin temperature. For the present problem Bt

is a vector of dimension M × 1. We briefly recall that

• M = 7 number of SEVIRI channels;

• NT length of the cycle;

• n = 7 dimension of emissivity vector, constant with time.

• N = n+ 1 = 8 dimension of retrieval vector for each time slot, n=7 emissivity values notvarying with time and 1 value for surface temperature depending on time.

To build up the linear problem to be inverted, we define the data vector y,

y =

y1

y2

. . .yNT

(49)


with dim(y) = M ×NT , and the parameter vector,

x =

δe1δe2. . .δenδTs1δTs2. . .

δTsNT

(50)

with dim(x) = n+(N−n)×NT (in the case at hand n = 7, N = 8). The matrices, At’s need tobe concatenated vertically, whereas the BT ’s have to be put together to form a block-diagonalmatrix,

A =

A1

A2

. . .ANT

(51)

with dim(A) = [M ×NT ]× [n],

B =

B1, 0, . . . 00, B2, . . . 0. . . . . . . . . . . .0, 0, . . . BNT

(52)

with dim(B) = [M ×NT ]× [(N − n)×NT ]. Next form the horizontal concatenation of A andB, to form the composite Jacobian,

K = (A,B) (53)

with dim(K) = [M ×NT ] × [n + (N − n) ×NT ]. Furthermore, let Sae be the a-priori matrixcorresponding to the the constant parameter vector e, and let Sat the a-priori matrix, at timet, corresponding to the parameter, Tst. The final Sa is obtained by building up the block-diagonal matrix with the sequence, Sae,Sa1, . . . ,SaNT . The same block-diagonal operation hasto be done with Sε in order to have an observational covariance matrix, which spans the fulldimension of the vector, y.

At this point we can writey = Kx (54)

and the regularized, optimal, solution is provided by Eq. (33).

It has to be stressed that the example provided in this section is only illustrative. Here weare dealing with a retrieval problem for which we assume a totally dependent state vectors foremissivity and a totally independent state vector for the skin temperature. We could think,e.g., to introduce some kind of correlation for the emissivity state vector trough the backgroundcovariance matrix. Different implementations of the basic form 43 will be shown directly whendealing with specific applications to SEVIRI (see section 7.

Before concluding this section 3, Tab. 2 summarize the basic ingredients and dimensionalityaspects of the general variational form 43. Once again we stress that Tab. 2 applies to the mostgeneral case. For a particular application there could well be, e.g., a suitable representation ofthe state or data vector, which could reduce the dimensionality. As an example the case justdiscussed in this section achieves a reduction dimensionality by a factor NT for the emissivitystate vector, because we assume that this vector is totally dependent along the time span, T .


Table 2: Summary of the dimensionality of the basic variational form 43. The table show howthe dimensionality increase when we modify state and radiance vectors corresponding to thereference 1-D vertical spatial case. Note that the size N in the reference case includes thesurface parameters (emissivity and skin temperature). Also, note that the vector-matrix sizeshown in the table refrers to the full space in which vectors and matrices are embedded. Thisdoes not necessarily coincides with the number of elements we need to store. As an example a

covariance matrix of size n × n is symmetric and therefore it needs only n×(n+1)2 elements to

be represented.

Dimension Parameter and/or data Size Remark

1-D vertical spatial State vectors, v,va,v0 N(Reference case) Background covariance matrix, Sa N ×N The size is essen-

tially determinedby the layering ofthe atmosphereused in the for-ward model andby the numberof atmosphericparameters

Data vector, R MObservational covariance matrix, Sε M ×M The size is essen-

tially determinedby the number ofspectral radiancesor channels

2-D: 1-D vertical spatialwith 1-D time State vectors, v,va,v0 NT ×N

Background covariance matrix, Sa (N2T ) ×N ×N

Data vector, R NT ×MObservational covariance matrix, Sε (N2

T ) ×M ×M NT is determinedby the ratio T/∆t,with T the time slotwidth and ∆t thetime sampling in-terval

3-D: 1-D vertical spatialwith 2-D horizontal spatial State vectors, v,va,v0 n2 ×N

Background covariance matrix, Sa (n4) ×N ×NData vector, R n2 ×MObservational covariance matrix, Sε (n4) ×M ×M n × n is the size of

the horizontal boxof field of viewsof the instrument,in principle n canbe as large asto cover the fullglobe. Suitableparameterization ofSa to reduce its di-mensionality havebeen discussedin section 3.1.1.In most cases Sεcan be considereddiagonal, there-fore its effectivedimensionality isn2 ×M

4-D: 1-D vertical spatial1-D time2-D horizontal spatial State vectors, v,va,v0 NT × n2 ×N

Background covariance matrix, Sa (N2T ) × n4 ×N ×N

Data vector, R NT × n2 ×MObservational covariance matrix, Sε (N2

T ) × n2 ×M ×M See remarks for the3-D case.

4 THE KALMAN FILTER 34

4 The Kalman filter

The Kalman filter (KF) was first developed by Kalman and Bucy [24, 25] in an engineeringcontext and as a linear filter. Its derivation within the Bayes formalism has been shown bymany authors (e.g. see the review [53]).

With our notation, the formal filter can be summarized with the couple of equations below,which are often referred to as the observation equation (or data model) and the state equation(or dynamic model or system model), respectively{

Rt = F (vt) + εtvt+1 = Hvt + ηt

(55)

where H is a linear operator and the noise model term has covariance, Sη. The remainingparameters appearing in Eq. (55) have the same meaning as those introduced in section 3. KFis intrinsically linear, therefore the observation equation has to be linearized in order to writedown the optimal estimation for the state vector. With the same notation we have used untilnow, we have the linear form of KF,{

yt = Ktxt + εtvt+1 = Hvt + ηt

(56)

where we use the notation Kt for the Jacobian to stress that it depends on time, t.

It should be noted that we assume that both the noise terms, εt and ηt are independentof the state vector.

4.1 The KF update step or analysis

Under the same assumption of multivariate normal statistics as that used in section 3, we havethat the optimal KF estimate, xt at time t is given by (e.g. [53]),{

xt = xa +[SaK

Tt

(KtSaK

Tt + Sε

)−1](yt −Ktxa)

St = Sa − SaKTt

(KtSaK

Tt + Sε

)−1KtSa

(57)

which, if we consider that (e.g. [51])

SaKTt

(KtSaK

Tt + Sε

)−1=(KTt S−1ε Kt + S−1a

)−1KTt S−1ε (58)

and that, based on the Sherman-Morrison-Woodbury identity,

Sa − SaKTt

(KtSaK

Tt + Sε

)−1KtSa =

(KTt S−1ε Kt + S−1a

)−1(59)

we have,

xt = xa +(KTt S−1ε Kt + S−1a

)−1KTt S−1ε (yt −Ktxa)

St =(KTt S−1ε Kt + S−1a

)−1 (60)

which shows that the optimal KF estimate for xt is formally equivalent to that obtained by thevariational approach in section 3. We recall, once again, that in the context of data assimilation,xa is normally the forecast at time, t and Sa is the error forecast covariance matrix. Theestimation, xt is referred to as the analysis at time t, which has covariance matrix given by St.

One important aspect of the formal solution is that the analysis depends only on the dataat time t and not on that at previous times. This property is referred to as Markov property.In fact, the formal solution for the analysis does not depend on the dynamical system directly.We can see that the expression in Eq. (60) does not contain the operator, H.


The above property is also referred to as the regularization property of KF. New data comesin at t the KF updated state estimate is the minimizer of the quadratic form

minx

1

2(yt −Ktxt)

TS−1ε (yt −Ktxt) +

1

2(x− xa)

TS−1a (xt − xa) (61)

However, an important distinction with data assimilation is that Sa is potentially generatedfrom the process and not from an external spatial model. In fact Sa is iterated with the processas it will become clear in examining the forecast step for the linear KF. Before moving to thenext section, it is important here to stress that the minimization of the form (61) needs aniterative approach because of the non linearity of the forward model.

4.2 The KF forecast step

In our notation, x = v − vo and xa = va − vo, so that the formal KF estimate for the statevector is

vt = va +(KTt S−1ε Kt + S−1a

)−1KTt S−1ε (yt −Ktxa) (62)

For the forecast step the KF assumes that the process evolves in a linear way, according to theoperator H, therefore we can obtain an estimate of the forecast at time t+ 1, standing at timet, through the linear transform,

v(f)t+1 = Hvt (63)

which has uncertainty given byStf = HStH

T + Sη (64)

where Sη is the covariance matrix of the noise term ηt (see Eq. (56)).

As soon as new data comes in at time t+ 1, the forecast becomes the background,

va = v(f)t+1, Sa = Stf

and we are ready to obtain the new analysis, vt+1

An important concept to draw from this sequential updating is that spatial informationabout the distribution of vt can be generated from the dynamics of the process. In fact,analyzing the forecast covariance matrix (64), it is seen that it is based on the previous forecastcovariance matrix and also inherits the dynamical relationship from the previous time. Thus, inthe situation of assimilation for a space-time process the spatial covariance for inference is builtup sequentially based on past updates with observations and propagating the posterior forwardin time as a forecast distribution. We stress that this spatial information is the difference orerror between the conditional mean and the true field and is not the covariance of the processitself.

However, the goodness of this spatial information mostly relies on the quality of the physicswe model with the operator H. Typically, the forecast step is completed by a deterministic,physically based model. In this case, the spatial information has value. However, in case wherewe want the problem driven from the data, the model can be very simplistic and inherentlyinadequate to describe the real-world process. In this case, spatial information has to beprovided externally through a proper definition of Sη.

Indeed, in general, it is fairly common that the measurement matrix Kt and the measurement-error covariance matrix Sε are known, as in our case, but it is much less likely that the pa-rameters associated with the evolution equation distribution are known, as in our data-drivenmodel. In this case we can estimate them starting from a proper definition of Sa and using theBayesian hierarchical modeling framework in the context of linear models. The use of the Gibbssampler algorithm ensures convergence to the truth distribution of the unknown parameters,preserving the spatial characteristics in the propagation. (See [9], pages 449-460 ).


4.3 A formulation of the emissivity-temperature retrieval with KF

The problem of emissivity-temperature retrieval we have formulated in section 3.2.2 in thecontext of optimal estimation becomes straightforward once analyzed with the formalism ofKF. In fact, if we want to design a filter in which emissivity does not change with time, incontrast to surface temperature which is supposed to have a much faster dynamics, we havejust to define the operator H in such a way that the part of the state vector which representsemissivity does not change with time, whereas the part of the state vector, which correspondsto surface temperature does not depend on the model. Using the same notation used in section3.2.2, we have

yt = Atδe +BtδTst (65)

where the size of the observation vector, yt is M = 7, and the sate vector,

x =

δe1δe2. . .δenδTs

(66)

has dimension n+ 1 = 8, with n = 7. For the state equation we have xt+1 = Hxt, with

H =

1, 0, . . . 00, 1, . . . 0. . . . . . . . . . . .0, 0, . . . 0

(67)

that is H is a diagonal matrix, with ones corresponding to emissivity values and zero the surfacetemperature. In this way we sequentially input the optimal estimation block of the KF with astate vector which is modeled as having a time-constant dynamics in its emissivity component,while its component corresponding to the surface temperature depends only on the data andnot on the model equation.

With the KF approach, the dimension of the problem remains limited to 7×8. In contrast,if we consider the optimal approach with time accumulation of the observations on a cycle oflength NT , the dimensionality of the problem increases of a factor N2

T , as it has been shown insection 3.2.2.

4.4 Sequential Updating: a numerical algorithm to calculate the anal-ysis, x and its covariance, S, without explicit inversion of matrices

Either we formalize our problem with the variational approach or KF, we are faced with thesolution of a system of linear equations (e.g. Eq. (33), or Eq. (57)), which involves inversionof large matrices.

In case we want to include spatial constraints, the size of the matrix Sa may becomeprohibitive for numerically stable inversion operations. However, in many cases we have thatthe data points are uncorrelated, so that the observational covariance matrix becomes diagonal.

In this case we can use a recursive formula in which we use one datum at a time. If Mis the size of the data vector, than the recursive updating can be organized in a loop of Miterations.


The formula is better established for the system of equations provided by Eq. (57) and itreads,

x0 = xaS0 = S1

for i=1:M do:

xi = xa +Si−1ki(yi−kTi xa)kTi Si−1ki+σ2

i

Si = Si−1 − Si−1kikTi Si−1

kTi Si−1ki+σ2i

nextx = xMS = SM

(68)

where yi is the i-th element of the vector y, σ2i is the variance error of yi and ki is the i-th row

of the Jacobian K.

In this scheme the components of y are considered sequentially and there is no matrixinversion involved. In case, the matrix Sa can be represented on the basis of some analyticalmodel, such as those presented in section 3.1.1, the formula above can allow us to deal withlarge Sa.

Before concluding section 4, as done for the variational form 43, Tab. 3 summarizes thebasic ingredients and dimensionality aspects related to the Kalman filter. Comparing to Tab. 2we see that the reference case has dimension 2-D, since it is the product of the time dimensionby the vertical spatial dimension. In addition, the dimensionality of the 2-D Kalman filter isthe same as that of the 1-D reference scheme for Optimal Estimation, because of the basictime-sequential strategy of the Kalman filter. Actually, in case in we deal only with surfaceparameters, the reference Kalman filter has dimension 1, since in this case the vertical scalecollapses to zero. This has been clarified in the remark in Tab. 3.

We have not included the case corresponding to 2-D and 3-D spatial dimensions, since theKalman filter needs a relation order to distinguish present from future events. Actually, theKalman filter can be generalized to a pure spatial settings. However, these cases are betterdealt with the tool of Optimal interpolation or spatial Kriging, which are briefly discussed insection 6.

Moreover, we could even consider a 2-D truly spatial Kalman filter by introducing an ad-hoc space evolution equation along a given spatial dimension. However, this approach cannottake into account all the correlation products in a 2-D n× n spatial field, which are of order ofn4. By running along rows and columns we can at most consider 2n2 correlation products outof the total n4. For this reason we have not investigated the use of spatial evolution equation.Spatial correlation has been introduced in its correct form by considering the covariance matrixof the 2-D field.


Table 3: Summary of the dimensionality of the basic Kalman filter. The table show howthe dimensionality increase when we modify state and radiance vectors corresponding to thereference 2-D vertical time-spatial case. Note that the size N in the reference case includes thesurface parameters (emissivity and skin temperature).

Dimension Parameter and/or data Size Remark

2-D: 1-D vertical spatial1-D temporal(Reference case) State vectors, v,va,v0 vf N

Analysis covariance matrix, Sa N ×NStochastic term covariance matrix, Sη N ×NEvolution operator, H N ×N The size is essen-

tially determinedby the layering ofthe atmosphereused in the forwardmodel and by thenumber of atmo-spheric parameters.In case the verticalscale collapses tozero (no atmo-spheric parameters,the dimension ofthe filter becomes1-D

Data vector, R MObservational covariance matrix, Sε M ×M The size is essen-

tially determinedby the number ofspectral radiancesor channels

4-D: 1-D temporal1-D vertical spatial2-D horizontal spatial State vectors, v,va,v0, vf n2 ×N

Analysis covariance matrix, Sa (n4) ×N ×NStochastic term covariance matrix, Sη (n4) ×N ×NEvolution operator, H (n4) ×N ×NData vector, R n2 ×MObservational covariance matrix, Sε (n4) ×M ×M n × n is the size of

the horizontal boxof field of viewsof the instrument,in principle n canbe as large asto cover the fullglobe. Suitableparameterization ofSa to reduce its di-mensionality havebeen discussedin section 3.1.1.In most cases Sεcan be considereddiagonal, there-fore its effectivedimensionality isn2 ×M . As for thereference case, ifthe vertical scalecollapses to zero,the filter becomes3-D

5 THE ENSEMBLE KALMAN FILTER 39

5 The ensemble Kalman filter

The ensemble Kalman filter (EKF) [18] tries to avoid the complication of working with highdimensional state vectors and related covariance matrices by using a sample of states, whichare representative of the geophysical flow at hand. Each ensemble member is updated byan optimal step (corresponding to the analysis)and is propagated forward in time using thedynamical model giving a new ensemble for approximating the forecast distribution. Theensemble update step holds the main statistical details of EKF. Conversely, the forecast stepfor ensembles is both simply and explicit, since it is based on a deterministic, physically basedmodel.

Let{

x(j)a

}j=1,...,Na

an ensemble of Na members, which provides a discrete representation

of the background, or forecast, or a-priori pdf. Let xa the sample average of the ensemble, andlet

Ba =(x(1)a − xa,x

(2)a − xa, . . . ,x

(Na)a − xa

)(69)

be a matrix of the centered ensemble members. The sample forecast, or background covariancematrix is obtained by

Sa =1

Na − 1BaB

Ta (70)

In case the size of the state vector is N with N > Na, this approach allows us to work withdimensionality of Na ×N rather than with the full dimensionality N ×N . In fact, Sa definedabove has a effective rank Na−1 and when used in the update equations (57) the linear algebracan exploit this reduced rank.

We see that the EKF is equivalent in its philosophy to the climatology retrieval or inversion.If we focus on the analysis step, it is equivalent to a 3D variational assimilation scheme, inwhich we try to introduce spatial constraints (see section 3.1). The main difference is that in3D variational schemes the Sa is fixed, that is the ensemble of states is fixed, whereas in EKFthe ensemble evolves with the dynamics.

6 OPTIMAL TIME-SPACE INTERPOLATION SCHEMES 40

6 Optimal time-space interpolation schemes

Spatial optimal interpolation also known as kriging in geostatistics [8] is much more suitedfor level 2 data, that is scalar geophysical parameters defined over a spatial domain. Themethodology is in fact developed within a simplistic observation equation of the type signal-noise model, that is the scalar data is the scalar signal additively corrupted with a noise term,

yt = xt + ε (71)

In this simplistic data model or equation the jacobian K is replaced with a simple indicatormatrix, KI , which identifies the subset of space locations for which we have both data andsignal. To exemplify the matter let us suppose we have xt = (xt(s1), xt(s2), xt(s3)), that is thesignal is defined for three space locations, s1, s2, s3. Let us suppose that the data vector yt isdefined only for the two locations, s1 and s3, but not for s2, that is yt = (yt(s1), yt(s3)). Thus,KI is defined as

KI =

(1 0 00 0 1

)(72)

In vector form the data equation can be written as

yt = KIxt + ε (73)

The solution to this optimal interpolation problem is still that shown in Eq. (33), provided Kis substituted with the appropriate indicator matrix KI .

In case, we introduce a time evolution model for the signal, the problem becomes one ofoptimal time-interpolation. In this case we can use the formal setting of KF and the solutionis obtained as outlined in the two sections 4.1 and 4.2.

In a series of recent papers [10, 11, 26] Cressie and coworkers have developed efficientschemes to deal with the problem of spatial and spatio-temporal optimal interpolation. Theschemes are referred to as Fixed Rank Kriging (FRK) and Fixed Rank Filtering (FRF) for thecase of spatial and spatio-temporal optimal interpolation, respectively. The schemes have beenapplied to level 2 data such as CO2 and aerosol optical depth derived from satellite observations.

Both FRK and FRF provide efficient algorithms for the numerical calculation of the optimalestimate, xt given by Eq. (33). These algorithms avoid the inversion of large matrices, such asSa, but normally they cannot provide the cross-correlation terms in the a-posteriori covariancematrix; only the variance of elements of xt is computed.

The algorithms have been applied to very large and massive data sets, however they arenot directly applicable to problems that involve the use of level 1 data for the retrieval ofgeophysical parameters.

However, the idea, which makes the tools so powerful in dealing with massive data sets,is interesting and could be explored also with our retrieval problem. The idea consists inreducing the dimensionality of the data vector, yt trough a suitable transform (not necessarilyorthogonal). The problem is fundamentally one of data-dimension reduction, to which Cressieand coworkers give a solution by introducing a representation of the signal xt through whatthey call the Spatial Random Effect (SRE) model where the unknown random variables to bepredicted or estimated are fixed in number and are coefficients of known spatial basis function.

7 THE SEVIRI CASE STUDY FOR EMISSIVITY-TEMPERATURE RETRIEVAL 41

Figure 6: Sensitivity of the SEVIRI channels to emissivity. The computation refers to a typicaltropical state vector.

7 The SEVIRI case study for emissivity-temperature re-trieval

The case study we have developed and the many related retrieval exercises we have performedare aimed at demonstrating the capability of the retrieval methodology, when time and spatialconstraints are included in the analysis. In this first stage of the analysis, the case study isdevoted to the problem of surface temperature and emissivity estimation from the MSG SEVIRIinfrared channels.

The inclusion of atmospheric parameters will be described and demonstrated in section 11,whereas the extension of the methodology with spatial constraints will be discussed in section12.

Focusing for now our attention to the [Ts, ε] case, we see by inspection of Fig. 1, whichshows the spectral response of the infrared channels, that the window channels 4, 6 and 7 areexpected to be largely sensitive to skin temperature and emissivity. Channel 2 is expected to becompletely insensitive, whereas channel 8 and 5 are in between a good and a poor sensitivity.

This qualitative analysis can be made more rigorous by a computation of the radiancederivative with respect to the emissivity. Considering also the radiometric noise, nedn(σ)affecting the channel at wave number σ, we can define the sensitivity index, SX to a genericatmopsheric parameter, X as

SX(σ) =

(1

nedn(σ)

∂R(σ)

∂X

)(74)

he case relevant to emissivity is shown in Fig. 6 for a typical tropical atmospheric state.

As expected, channels 4, 6 and 7 have a very large sensitivity to emissivity, whereas channels2 and 3 are completely insensitive. However, in developing and implementing the retrievalmethodology, we will us all the 7 channels, also in view of the generalization of the algorithmto water temperature, water vapour and ozone.


We have developed and implemented the software for the two algorithms described insections 3.2.2 and 4.3, respectively. In doing so, the software has been designed to run with theseven SEVIRI channels defined in table 1

7.1 The target area

Retrieval examples will consider the geographic area shown in Fig. 7, which includes Spain,Westward Sahara desert, and ocean areas of the Mediterranean sea and Atlantic ocean. Thearea has been selected because of the presence of both vegetation and bare soil and desertsand. In case of bare soil, emissivity variations are expected to play a secondary role, whichis appropriate to the case we want to study, that is the presence of atmospheric parameterswith a very low time variability. Also, the inclusion of desert sand will simplify the analysis ofthe quality of the retrieval, since quartz rich sand, as those in desert area, introduce a spectralfingerprint in the atmospheric window which is unique. The stability of desert sand and thisstrong fingerprint should play a role similar to that of truth data and provide a reference againstwhich we can compare the retrieval results.

In this respect, one of the SEVIRI channels, that at 1149.4 cm−1 falls within the strongestmolecular vibration band of quartz. For the silicate molecule this occurs between 830 and 1176.5cm−1. This range is generally referred to as the SiO stretching region and primarily involvesdisplacements of oxygen atoms, resulting in an asymmetric stretching mode (the ν3 asymmetricstretch). Reflectance spectra from a sample of the mineral show peaks at those wave numberscorresponding to absorption bands. This occurs because the intensity of these bands is so highthat a mirror like opacity is induced at those wave numbers. These reflectance peaks are oftenreferred to as resthralen bands and they are clearly visible from satellite observations.

7.2 SEVIRI data and ancillary information

SEVIRI observations (Meteosat 9 highrate SEVIRI level 1.5 image data) have been acquiredfor the target area. The observations refers to the whole month of July 2010.

ECMWF analyses for the same date and target area have been also acquired, which com-prise Ts, T (p), O(p), Q(p) for the canonical hours 0:00, 6:00, 12:00 and 18:00. The horizontalspatial resolution of the ECMWF analysis is 0.5× 0.5 degrees, therefore in each ECMWF gridbox there are on average ≈ 200 SEVIRI pixels. For these 200 SEVIRI pixels, we assume thatthe atmospheric state vector is the colocated ECMWF analysis (see e.g. Fig. 9). This statevector is used as a first guess for the surface and atmospheric parameters. A complete list ofthe data is given in table 4 below. Figure 8 shows the ECMWF surface temperature map for13 July 2010 (analysis at 12:00 GMT).

For the purpose of developing a suitable background for emissivity, we have also made avail-able to the project the Global Infrared Land Surface Emissivity (e.g. http://cimss.ssec.wisc.edu/iremis/)developed at CIMSS, University of Wisconsin [2]. This data base is derived by MODIS ob-servations and is available from the year 2003 till 2011. The emissivity is made available onmonthly basis, at 10 wavelength points (or hinge points) on a 0.05 × 0.05 degree grid. Thewavelengths are 3.6, 4.3, 5.0, 5.8, 7.6, 8.3, 9.3, 10.8, 12.1, and 14.3 µm.

The emissivity cannot be straightly interpolated to the SEVIRI channels, because of the dif-ferent spectral response function between MODIS and SEVIRI. This problem has been handledby developing a suitable software, which first extrapolates the low spectral resolution emissivityspectrum to the IASI wavenumber range, and second, through convolution with the SEVIRIspectral response, to the SEVIRI channels.

The software is based on a methodology developed by E. E. Borbas of University of Wis-consin, Madison [2], however the implementation of the technique and related software havebeen developed by the authors.


Figure 7: Target area showing a map of the SEVIRI channel at 12 µm. Observation refers tothe date 9 July 2010, hour 12:00.

Figure 8: Surface temperature (ECMWF model) map over the target area of the case study(July, the 13th 2010, 12:00 GMT).


Table 4: ECMWF data for the target area

Parameter TypeSurface pressure scalarSkin temperature scalarSea surface temperature scalarTotal cloud cover scalarLand sea mask scalarSoil type scalarTemperature profileSpecific humidity profileOzone mixing ratio profileCloud ice water profileCloud liquid water profile

An example of this interpolation from IREMIS to SEVIRI trough IASI is presented in Fig.10.

The interpolation is a bit more complicated because of the different grid-mesh of IREMISdata and SEVIRI. We have developed a software, which before applying the procedure illus-trated in Fig. 10, interpolates IREMIS from the original grid mesh to that of SEVIRI.

An example of IREMIS-MODIS emissivity data for the target area transformed to IREMIS-SEVIRI is shown in Fig. 11 for the case of the SEVIRI channel at 1149.4 cm−1.

These emissivity data points have been used to build up an appropriate a-priori or back-ground (mean and covariance) for the Optimal Estimation and Kalman filter methodology. Indoing so we have not used July 2010, which is left for checking/validation.


Figure 9: Example of overlapping between the SEVIRI fine mesh and that coarse correspondingto the ECMWF analysis.

Figure 10: From IREMIS to SEVIRI emissivity, passing trough IASI.


Figure 11: Example of IREMIS-SEVIRI emissivity mapped onto the target area, for the channelat 1149.4 cm−1.

8 OPTIMAL ESTIMATION: IMPLEMENTATION ANDRETRIEVAL EXERCISES IN SIMULATION 47

Figure 12: Typical temperature daily cycle for desert sand.

8 Optimal estimation: implementation and retrieval ex-ercises in simulation

The optimal estimation with a static background and time accumulated observations has beendeveloped according to the methodology outlined in section 3.

In its present implementation, the scheme can run over a time period as long as decided bythe user, and can accumulate observations on a cycle or time slot, which is decided by the user,as well. The minimum time slot is 1 hour, which for SEVIRI means 4 different observations fora total of 7*4=28 spectral radiances per hour (as discussed in section 2.3, in our scheme we havenot included the channel at 3.9 µm because of spectroscopy issues and solar contamination).

The baseline scheme considers the retrieval of (Ts, ε). For the most resolved time slot ofone hour, assuming spectral emissivity constant over that time, the retrieval scheme considersa total of 11 parameters to be estimated for each hour (emissivity at seven channels plus 4 skintemperatures with a time resolution of 1/4 of hour. Of course the time slot can be increased,which would change accordingly the number of observations and parameters.

The static background, for temperature and emissivity, can be specified by the user. Asdiscussed in section 7, for the case study considered in this work, the emissivity backgroundhas been derived from the IREMIS data base. For temperature, an effective background maydepend on the problem at hand. A simple exercise in simulation will help to discuss this andother basic aspects of the methodology, as well as its expected performance.

Figure 12 shows a typical surface temperature daily cycle, which refers to a desert station.The data shown in Fig. 12 is based on the work by [22], who has developed a model which fitsto real observations within 0.6-2.5 K, depending on the hour of the day. Thus, the exampleshown in Fig.12 reflects a realistic situation as it is likely to occur in clear sky in African desertsor at higher latitudes during the summer season.

Using the temperature cycle shown in Fig. 12, we have computed SEVIRI synthetic radi-ances. The SEVIRI radiometric noise has been added to the radiances in order to simulate theobservations.


For the emissivity at each channel we have used the values shown in Tab. 5, which againcorrespond to a typical emissivity spectrum for desert soil.

Table 5: True and Background emissivity values at the seven SEVIRI channels used in theretrieval exercise in simulation

Channel Number wave number (cm−1) Emissivity (True) Emissivity (FG)2 1612.90 0.9711 0.97223 1369.90 0.9827 0.98214 1149.40 0.7195 0.74615 1030.9 0.8807 0.88236 925.90 0.9404 0.96867 833.30 0.9621 0.94548 746.30 0.9504 0.9599

The values in table 5 have been extracted from the IREMIS data base for the location at30.49 degrees of latitude and 5.52 degrees of longitude. This location is a flat, sandy, site in themiddle of the Algerian desert. The true emissivity corresponds to that for the month of July2010, whereas the background is the mean value of the July emissivity over the years 2003 to2011, but 2010.

The background covariance matrix has been computed, for the same location, again basedon the data accumulated for July in the IREMIS data base for the years 2003 to 2011, excluding2010.

The background covariance matrix is shown in Fig. 13. As expected the covariance matrixshows a variability peak at channel 4, which corresponds to the wave number 1149.40 cm−1.This channel is in between the strong reststrahlen bands of quartz. Figure 14 shows the stan-dard deviation (square root of the diagonal elements) of emissivity as function of the SEVIRIchannels. Apart from channel 4, it is seen that the variability of emissivity is a few partsper thousand, which is bit less than expected considering that it should apply to year-to-yearvariation. We think that this is the combined effect of the site (desert sand emissivity is poorlyaffected by seasonal variations) and smoothing applied to generate the data base over a meshgrid of 0.05o × 0.05o.

Concerning the background state or vector for the skin temperature, we have assumed itto be equal to the true cycle minus 4 K. For its variability, we have have to consider that inclear sky the skin temperature is strongly driven by the daily cycle and, hence, has a strongdeterministic behaviour. The process is not stationary and its variability increases with time.This is shown in Fig. 15 where we consider the standard deviation of the skin temperaturedaily cycle shown in Fig. 12 averaged over time slots of 1 hour to 21 hours. It is seen that thevariability grows from about 1 K to 15 K according to the time slot.

Figure 15 is important because it says that, for a typical desert and/or summer surfacetemperature daily cycle, we have a variability even over a slot of 1 hour. This is importantwhen consider its potential effect over the radiative transfer equation in order to separate thecouple Ts − ε. It means that the cycle, within one our, can develop significant variability toyield linearly independent information to separate Ts from ε.

To complete the simulation, we need to specify the atmospheric state vector. We haveused a typical tropical set of profiles for (T (p), O(p), Q(p)). The atmospheric state vector isassumed to be perfectly known within the simulation. The set of profiles for (T (p), O(p), Q(p))is shown in Fig. 16. For trace gases we use climatology.

The retrieval scheme has been initialized with the emissivity first guess values set equalto the background shown in Tab. 5. The same choice has been done for skin temperature,therefore the skin temperature first guess is the test cycle shown in Fig. 12 minus 4 K.


Figure 13: Background matrix for emissivity concerning the retrieval exercise.

Figure 14: Emissivity variability of the background computed from the covariance matrix shownin Fig. 13. The variability is the square root of the diagonal elements).


Figure 15: Variability of the daily cycle shown in Fig. 12 as a function of the time slot width.

Figure 16: State vector of the atmosphere (not retrieved) for the major parameters, T, H2Oand O3, used within the retrieval exercise in simulation.


Figure 17: SEVIRI radiometric noise (upper panel) used to build up the observatinal covariancematrix. For the benefit of reader accustomed to NEDT, this is shown in the bottom panel fora scene temperature of 280 K.

Finally, for the observational covariance matrix we have used the SEVIRI radiometric noise(in units of NEDN), which is shown in Fig. 17.


8.1 The case of 1 hour-width time slot

We first discuss the results we obtain by accumulating the data for one hour, which for SEVIRI,as already said, means a cycle of 4 observations.

In view of the strong deterministic characteristic of the daily cycle, we consider the back-ground covariance matrix to be diagonal. According to Fig. 15 the diagonal elements are equalto 1.332K2.

It is important here to remark that we are just discussing a retrieval exercise to exemplifythe methodology. It is not that our methodology can deal with deterministic daily cycle alone.A case in which we consider a stochastic daily cycle with a non-diagonal covariance matrix willbe discussed in section 8.4.

Before showing the results, we summarize the settings of the retrieval scheme in Table 6

Table 6: Summary of the settings for the optimal estimation scheme, which applies toretrieval exercise discussed in section 8.1.

Element Setting/Reference

Time slot width 1 hourEmissivity true values from IREMIS data base, see third column in Tab. 5

Emissivity FG and BG vectors (FG=BG) from IREMIS data base see fourth column in Tab. 5Emissivity static background from IREMIS data base, see Fig. 13

Skin Temp. true values values shown in Fig. 12Skin Temp. FG and BG vectors (FG=BG) True-4 K

Skin Temp. static background errors diagonal with elements set to 1.332 K2

Observational Covariance matrix diagonal, from SEVIRI radiometric noise (see Fig. 17)Atmospheric profiles assumed known, see Fig. 16Convergence criterion cost function

Coming back to our exercise, the results for temperature are shown in Fig. 18 and Fig.19, from which we see that the temperature cycle is recovered almost perfectly, despite the 4K difference between test and FG.

The results for emissivity is shown in Fig. 20 and Fig. 21. It is seen that the retrievalmove correctly towards the true solution at each channel. The retrieval tends to be slightlydownward biased. However, this bias interests the third-fourth decimal digits, and it has noimportant effect over the retrieval of the skin temperature.


Figure 18: Results of the retrieval exercise for temperature with the optimal estimation scheme.The case shown applies to a time slot of 1 hour. For this exercise the FG has been set to thebackground (BG)

Figure 19: Difference FG-True, Retrieval-True for the case shown in Fig. 18. The accuracyof the retrieval (square root of the diagonal of the covariance matrix) is shown likewise a ±1σtolerance interval.


Figure 20: Results of the retrieval exercise for emissivity with the optimal estimation scheme.The case shown applies to a time slot of 1 hour. For this exercise the FG has been set to thebackground (BG)

Figure 21: Difference FG-True, Retrieval-True for the case shown in Fig. 20. The accuracyof the retrieval (square root of the diagonal of the covariance matrix) is shown likewise a ±1σtolerance interval.


Figure 22: Same as Fig. 19, but now the time slot width is 21 hours.

8.2 The case of 21 hour-width time slot

What happen in case do we increase the width of the time slot? In principle one might expect tohave a drastic improvement in the accuracy of the results in comparison to the case of 1 hour.However, one should consider that the problem is not linear, at least for skin temperature.Moreover, if we increase the time slot width we explore new components of the daily cycle,so that we need more data to resolve these new components of the cycle. The situation iswell represented in Fig. 15, which shows that the cycle-variability grows up with the time slotwidth.

If we want to use the largest width of the time slot, which in our case corresponds to 21hours, we have to consider a diagonal covariance matrix with elements on the diagonal equalto 152 K2. In other word, the increase in the number of data points is compensated by theincrease in the variability of the daily cycle.

During a day the scene cannot be considered isothermal, so we cannot expect any improve-ment in the emissivity retrieval because of the increase in the number of observations.

The results for temperature are shown in Fig. 22, which can be compared to those of Fig.19 (time slot width of 1 hour). It is seen that the accuracy and quality of the retrieval arealmost unchanged either we accumulate data for 1 hour or 21 hours.

The retrieval of emissivity is shown in Fig. 23, which may be compared to Fig. 21. Alsofor the case of emissivity we have almost the same results as those shown for a time slot of1 hour. The error goes slightly down, but not linearly with the number of data points. Theanalysis is consistent with that shown in Fig. 21 also for what concern the bias. The retrievalis slightly downward biased.


Figure 23: Same as Fig. 21, but now the time slot width is 21 hours.


Figure 24: Background covariance matrix corresponding to a 28-size emissivity vector with aperfect correlation among channels corresponding to the same wave number.

8.3 Implementation with the whole emissivity state-vector

In the two sections above, we have assumed the emissivity to be constant within a time slot, andthe corresponding state vector has been collapsed to a vector whose dimension is the numberof SEVIRI channels considered in the retrieval analysis. This approach has the advantage ofreducing the dimensionality of the problem, which otherwise would linearly increase with thewidth of the time slot.

However, the user could also want to consider a not totally dependent emissivity vector.This can be easily accommodate within the present scheme by considering the full-size emissivityvector.

As an example, in case of a time slot of one hour, the size of the emissivity vector would be28 and the corresponding background covariance matrix would have a size 28× 28. Note thatin this case the number of observations would be 28 against 32 parameters to be estimated. Inpractice this is not a problem provided the emissivity vector is not totally independent.

To exemplify how we can deal with the full-size emissivity vector, let us consider the sameexercise as that shown in section 8.1. in the following, we will describe how this exercise canbe implemented with an emissivity vector of size 28. To reproduce the same exercise as that insection 8.1, we have to introduce a background covariance, which models a perfect correlationbetween channels at the same wave number, but at different time. As an example the channel,say, at wave number 833.3 cm−1 at time t1 has to be perfectly correlated (correlation equal to1) with the same channel at any given later time tn.

The 28× 28 emissivity background which simulate a total dependent emissivity (constantemissivity at each wave number) is shown in Fig. 24. This matrix can be obtained by the matrixshown in Fig. 13 by properly expanding its size to account for the inter-channel correlation.

Using this background matrix we can run the same exercise as that shown in section 8.1,but now with an emissivity state vector of size 28. The correlation is accounted for by thecovariance matrix, rather than by collapsing the 28-size vector to a 7-size state vector.


Figure 25: Same as Fig. 19, but now with the whole emissivity state vector of size 28.

The retrieval for temperature is shown in Fig. 25, while that for emissivity in Fig. 26. Itseen that we get exactly the results we obtained in section 8.1.

The implementation with a full-size emissivity vector adds flexibility to the scheme in casewe want to introduce some kind of dependence in this parameter. This dependence, whichhas not to be necessarily a perfect correlation, can be modeled directly within the emissivitybackground matrix.


Figure 26: Same as Fig. 21, but now with the whole emissivity state vector of size 28.


Figure 27: Skin temperature generated with a Markovian model.

8.4 An example with a dependent skin temperature vector.

In the previous sections we have been dealing with a totally independent skin temperaturevector. This is a good choice in case of clear sky and land surface where the skin temperatureevolution is governed by the daily cycle. However, there could be conditions and situations forwhich the skin temperature could well show a stochastic behaviour. In this case, the stochasticcorrelation has to be modeled in the background covariance matrix and our scheme has beendeveloped in order to accommodate any user-defined covariance matrix.

As an example, let is us consider the following stochastic model for the skin temperature,which simulates a persistence with a Markovian drift{

T (t) = To + τ(t)τ(t) = 0.8τ(t− 1) + η(t)

(75)

with To = 25 oC, where η is a Gaussian variate with zero mean and variance (1− 0.82). Withthis choice the variance of the Markovian drift, τ is unity.

A realization for one day of the process above is shown in Fig. 27. To make reference tothe SEVIRI repeat cycle, the Markov process in Fig. 27 is sampled with a sampling rate of 15min over a cycle of 24 hours.

For emissivity we consider the Masuda’s emissivity model for sea surface with a wind speedof 5 m/s. The true and background vectors are summarized in table 7

The atmospheric state vector is exactly the same as that used in section 8.1. For thebackground matrix for emissivity we have used this time a diagonal matrix with elements overthe diagonal equal to 10−6 (which means a variability over the third digit).

The background matrix of the skin temperature can be easily obtained by the autocorre-lation function, which is given at any time t by 0.8t. This background matrix is shown in Fig.28 for the time-range t = 1, ..., 96.

The background vector for the skin temperature has been set to the constant value of 21◦C.The First Guess has been considered to be the same as the background.


Table 7: True and Background emissivity values at the seven SEVIRI channels used in theretrieval exercise for sea surface


Figure 28: Background matrix for the skin temperature.

The retrieval exercise has been performed for two different time slot: one hour and 24 hour.For temperature the results are compared in Fig. 29 and Fig. 30. It is seen that the retrievalcan follow the stochastic drift of the signal with an accuracy of ±0.1K. There is no substantialimprovement if we change the time slot from one hour to 24 hour. The results show that in casewe can correctly model the correlation structure of the cycle over any time slot, the retrieval ispretty independent of the time slot we use.

The retrieval for emissivity is shown in Fig. 31. Because of the tiny difference betweentrue and background, we have that both differences, FG-True and Retrieval-True are withinthe noise bar. In other words we cannot resolve differences, which occur on the third decimaldigit. The case shown in Fig. 31 refers to the time slot of one hour. We got equivalent resultsfor the case of a time slot of 24 hour.


Figure 29: Results of the retrieval exercise with the temperature signal driven by a stochasticdrift. a) time slot of one hour; b) time slot of 24 hour. For this exercise FG and BG coincide.

Figure 30: As Fig. 29, but now the two differences FG-True and Retrieval-True are shownalong with the error bars shown as ±1σ tolerance interval. a) time slot of one hour; b) timeslot of 24 hour.


Figure 31: As Fig. 30, but now the emissivity is shown as ±1σ tolerance interval. The caseshown refers to the time slot of one hour.


In the above exercise we have assumed that the first guess for emissivity differs from thetrue emissivity on the third decimal digit. This is quite realistic if we consider the quality ofMasuda’s model for sea surface. However, the same exercise can be also used to check thelinearity of the retrieval problem to emissivity. To this end, we have performed a new exercisein which the first guess emissivity is set, at each channel, equal to the true value minus 0.05.The new setting for emissivity is summarized in Tab. 8. However, we have also to balance thebackground emissivity covariance to allow for a variability of 5%. This is achieved by scalingup the covariance matrix of factor 104. In other words, the emissivity covariance matrix isdiagonal with elements equal to 0.01. All the other ingredients of the retrieval exercise are leftunchanged with respect to the case just shown.

Table 8: True and Background emissivity values at the seven SEVIRI channels used in theretrieval exercise for sea surface. In this exercise we initialize the retrieval problem with a firstguess for emissivity which differs of -5% from the true emissivity.


Figures 32 and 33 summarize the new results in case we use a time slot of 1 hour. It isseen that the precision of the retrieval is largely affected because of the increased uncertaintyfor the emissivity. However, with respect to the previous results in this section, the bias ofthe retrieval is almost unaffected, either we consider the case of skin temperature or emissivity.Again, we see that for the window channels the emissivity bias is almost zero.

The precision of the retrieval can be improved by considering a larger time slot width. Thisis exemplified in Fig. 34 and Fig. 35, which summarize the retrieval for a case in which a timeslot of 4 hours is used.

To summarize, these last two exercises show that a good retrieval is most the result of agood balance between background and data information. The time slot width can be tuned inorder to improve precision (that is variance of the retrieval), however it has no important effecton the bias of the final solution.


Figure 32: Results of the retrieval exercise with the temperature signal driven by a stochasticdrift. The case applies to time slot of one hour; a) temperature retrieval; b); difference formthe true value of skin temperature. For this exercise FG and BG coincide. The emissivity isthat shown in Tab. 8

Figure 33: Error analysis for the emissivity retrieval corresponding to FG and true emissivityshown in Tab. 8. The case shown refers to the time slot of one hour.


Figure 34: As Fig. 32 but now a time slot of 4 hours is used.

Figure 35: As Fig. 33 but now a time slot of 4 hours is used.

9 KALMAN FILTER: IMPLEMENTATION ANDRETRIEVAL EXERCISES IN SIMULATION 67

9 Kalman filter: implementation and retrieval exercisesin simulation

9.1 Kalman filter with a 2-order autoregressive process to model theevolution equation of skin temperature

For the Kalman filter the state or model equation is one of the most important ingredients forthe implementation of the scheme.

For emissivity an evolution equation is straightforward if we consider the low variability ofthis parameter with time. Let e = (e1, . . . , en)T be the emissivity vector, a suitable dynamicalequation is then a simple persistence1

e(t+ 1) = e(t) + ηe(t+ 1) (76)

where eη(t+ 1) is a noise term with covariance, Sηe. This persistence model can be mathemat-ically formulated in terms of the identity propagation operator, He,

e(t+ 1) = Hee(t) + ηe(t+ 1) (77)

where He is the identity matrix. Because of this simply persistence model, the second orderstatistics properties of the vector e and the noise term ηe are the same, that is Se = Sηe.

The problem is a bit more complicated for surface temperature, since this parameter isstrongly influenced, hence, governed by the solar daily cycle.

In this respect, it is well known (see e.g. [37, 38, 22]) that the diurnal cycle of the surfacetemperature can be modeled with simply analytical functions of time, which are composed bya periodic component over-imposed to an exponential (Lorentzian) damping, which takes intoaccount the nighttime decrease of temperature. An example has been provided in Fig. 12,which, as said, shows the daily cycle of temperature for a desert station. What is importantfor us is not the analytical function, which generates the data shown in Fig. 12, but thedifferential equation, in this case the finite-difference equation, whose solution is the function,which originated the data. In fact, within the formalism of Kalman filter we need the stateequation, not its solution. This is an inverse problem, which had a certain popularity in the earlystage of chaos theory: how to infer the dynamic from a time series or realization of a process.One possible solution was given by [42], who proposed to fit the dynamic with autoregressiveprocesses. For the case at hand, that is the daily cycle of surface temperature, there are physicalarguments which says that the dynamics has to be fitted with an autoregressive process of secondorder. The fit for this case is the second order difference equation,

Ts(t+ 1) = ϕ1Ts(t) + ϕ2Ts(t− 1) + ηT (t+ 1); t = 1, . . . , n, . . . (78)

where Ts is the surface temperature and ηT is a noise term.

Given a realization of the process, the autoregressive coefficients ϕ1, ϕ2 can be fitted inany of several ways. We used the forward-backward approach, which minimizes the sum ofa least-squares criterion for a forward model, and the analogous criterion for a time-reversedmodel. The method is implemented in a MATLAB script and yields an estimate of ϕ1, ϕ2, oncewe have a time series of the process.

For the data shown in Fig. 12, we obtained ϕ1 = 1.992;ϕ2 = −0.992. The importantaspect is that the value Ts(t + 1) can be forecasted based on the previous values (which hereassume the role of initial conditions), Ts(t) and Ts(t− 1).

It is also important to stress that the AR representation of a certain class of analyticalfunction (this class includes harmonic waves) is exact (e.g. [42]), that is it is not an approxima-tion at discrete times of a continuous function. It gives the exact values of the function at the

1unless otherwise stated notations and nomenclature are the same as those used in sections 3 and 4


Figure 36: Simulation of SEVIRI channel 7 (833.3 cm−1) response (panel b) to the forcingof the temperature daily cycle shown in panel a). Panel c) shows the effect of the SEVIRIradiometric noise over the response of panel b).

given discrete time, t. In fact, if we apply the forecast scheme to the daily cycle shown in Fig.12 (the cycle has been generated with a continuous function belonging to the aforementionedclass), we get the exact solution. The autoregressive formalism is important because it exactlyfits to the way the Kalman filter works.

However, in the real world, which has many degree of freedom, Eq. (78) has to be intendedas a stochastic process with an intrinsically random component. However, because of the strongdeterministic character of the solar diurnal cycle, the random component may have a very lowstrength, that is variance. This is particularly the case in clear sky.

In case we have not a direct access to skin temperature observations, the problem is posedof how we can estimate the autoregressive coefficients. Towards this objective, it has to bestressed that SEVIRI window channels are mostly determined by the surface temperature dailycycle, therefore these coefficients could well be estimated from the radiance signal itself, e.g., inthe window channel at 833.3 cm−1. It is important to stress, that this channel has the highestemissivity value and the lowest variability (standard deviation) among emissivity spectra ofnatural minerals and terrains.

Figure 36 compares the temperature diurnal cycle of Fig. 12 to the calculated SEVIRIradiance for the channel at 833.3 cm−1. In the real world, the situation is a bit more complicatedbecause the radiance signal is itself corrupted with noise. The difference between signal andobservation (signal plus noise) can be appreciated in Fig. 36, where the observation has beencomputed using the SEVIRI radiometric noise for the channel at hand.

Table 9 shows the estimation of the two autoregressive parameters, ϕ1 and ϕ2 in case weuse the temperature cycle, the radiance response (signal) and the radiance observation.


Table 9: Estimation of φ1 and φ2 using either cause and effect.

Type of data ϕ1 ϕ2

Temperature Daily Cycle 1.992 -0.992Radiance response (signal) 1.993 -0.993

Radiance response (observation) 1.904 -0.904

Figure 37: Autoregressive representation of the temperature daily cycle with the coefficientsshown in Tab. 7.

Although there are differences among the threes couples of coefficients, estimated usingthe temperature cycle and the radiance response, the effect over the forecasted dynamic of theprocess is negligible, as it is shown in Fig. 37 where the original temperature cycle is reproducedwith the three different sets of coefficients shown in Table 9.

The example of Fig. 37 shows that the dynamical behaviour of the temperature daily cyclecan be easily estimated directly from the SEVIRI radiance in channel 7 (833.3 cm−1). This isone of the approaches we have taken when considering the application of the Kalman filter toSEVIRI observations.

9.2 Properties of the AR(2) model

In view of the importance of the AR(2) model for the dynamical representation of the temper-ature daily cycle, we describe in this section the basic properties of an autoregressive model oforder 2.

The model can give rise to bounded (stable) and unbounded (explosive) solutions. Thestable or unstable nature of the solution is easily described by a set of conditions over the


autoregressive coefficients themselves.

In order for the solution to be stable, the following conditions have to be satisfied

ϕ1 + ϕ2 < 1ϕ2 − ϕ1 < 1−1 < ϕ2 < 1

(79)

It is interesting to note that a pure sine wave yields a non-stable process with ϕ2 = −1. From thecoefficients shown in Table 7 it is seen that the second autoregressive coefficients is quite close tothe unstable condition, which indicates the presence of a harmonic component associated withthe solar cycle. In effect, a bit of noise in the system (in this case SEVIRI radiometric noise)help to stabilize the process. In presence of a cyclic component, the condition ϕ2 < 0 has to besatisfied, which means that the second condition above is satisfied as well. In addition, in caseof a harmonic cycle, the process tend to have ϕ1 = 1−ϕ2 and therefore the first condition aboveis not met. Because we know, that we are in presence of a cycle, once we have estimated thecoefficients, it is recommended that we should set ϕ1 = 1 + abs(ϕ2)− eps, where eps = 0.001.

Going back to table 9 it is easily checked that none of the AR(2) process estimated with thethree different data sets is stable. The stability has been recovered by using the eps-correctionfor ϕ1 suggested above.

The second order statistics of the AR(2) model can be derived from the following set ofequations (e.g. [3])

ρ1 = ϕ1(1− ϕ2)−1

ρ2 = ϕ21(1− ϕ2)−1 + ϕ2

σ2Ts

= σ2η(1− ρ1ϕ1 − ρ2ϕ2)−1

(80)

where σ2Ts

and σ2η are the variance of the process and the random component, respectively. In

view of the strong deterministic forcing of the solar daily cycle, σ2η is almost negligible when

compared to the variance of the cycle itself.

To be consistent with the formalism of the Kalman filter, the AR(2) process has to bere-written in a first-order difference equation form. This can be done by introducing the signaland noise vectors, respectively (see e.g. [43]),

Ts(t+ 1) = (Ts(t+ 1), Ts(t))T

ηT (t+ 1) = (ηT (t+ 1), 0))T (81)

whose evolution obeys the first-order difference equation,

Ts(t+ 1) = HTTs(t) + ηT (t+ 1) (82)

with,

HT =

(ϕ1, ϕ2

1, 0

)(83)

The variance of the noise term,ηT will be denoted with SηT .

9.3 Forward propagation of the analysis and the related covariancematrix

The forward propagation of the analysis or forecast is simply obtained by evolving the statevector with the model operator or propagator, H that maps the evolution of the process intime. For our baseline scheme, this is obtained by the block diagonal concatenation of He,which is the identity matrix, and HT defined in Eq. (83),

H =

(He, 00, HT

)(84)


The St error matrix is propagated according to

HStHt (85)

It should be stressed that in case we use an AR(2) model for the state equation, the matrix Sthas to be modified, before forward propagation, in order to be consistent with Eq. 81. Towardsthis objective, we need to store the a-posteriori error of the analysis for Ts(t) and Ts(t − 1).

At each time t this variance is the end-diagonal element of St. To simplify notation, let σ2Ts

(t)and σ2

Ts(t − 1) be the two corresponding variances, respectively. Then, we have to add to the

matrix, St, one more row and column, whose elements are zeros apart from the last two ones,which are given by

ρ1σTs(t)σTs(t− 1) (86)

andσ2Ts(t− 1) (87)

respectively, with

ρ1 =ϕ1

1− ϕ2(88)

Finally, we obtain the covariance of the stochastic forcing, Sη according to

Sη =

(Sηe, 0

0, SηT

)(89)

and the covariance matrix of the forecast, Sf is obtained by

Sf = HStHT + Sη (90)

It should be stressed at this point that while St is the estimation error of the state vectorv(t), the η-random terms appearing in the evolutionary equation of Eq. (76) and Eq. (78) andthe corresponding covariance matrices Sηe, SηT for emissivity and surface temperature, respec-tively represent stochastic forcing or features not resolved by the model. In other words theycannot be interpreted the same way as we do with measurement errors, that is exogenous errorterms. They are endogenous stochastic terms with, reflect the climate or weather variability ofthe process

For emissivity this means that Sηe is intrinsically the background covariance-matrix esti-mated by climatology. For the case of temperature the noise term is the intrinsic residual thatwe experience in fitting AR(2) models to the daily cycle of surface temperature. According to[37, 38, 22] this intrinsic fluctuations (standard deviation) range in between 0.5 and 2.5 K. Forthe baseline scheme we assume the default value of 1.0 K.


9.4 A retrieval exercise in simulation

The same retrieval exercise as that we have dealt with in section 8 using Optimal Estimationwith time accumulation of observations, has been implemented and performed with the Kalmanfilter methodology.

The state equation for skin temperature has been modeled with an AR(2) process and itsparameters have been estimated directly from the noisy radiances (e.g. see Table 9). Theseparameters are not the correct parameters of the signal, so that the present exercise also willallow us to check the sensitivity of the retrieval to the state model.

In the context of the Kalman filter, the stochastic noise term (see section 9.3) assumesa meaning and a role, which is much more important that the background covariance matrix.This last one is used to start the iteration cycle, the system looses its memory at a rate, which isdetermined by the stochastic noise term. The stochastic term governs the asymptotic propertiesof the retrieval covariance.

By properly tuning the stochastic noise covariance, we can have a retrieval which is eitherdominated from the data, or the state model (this effect will be further analyzed in next section).

For the present exercise, the initial background covariance matrix for temperature (notethat the skin temperature is a scalar, hence the related state vector has size 1 and the relatedcovariance matrix is of size 1× 1) has been set equal to (1.332) K2, the same as that used withthe Optimal Estimation scheme and a time slot of one hour (see section 8.1).

For the stochastic term we have considered a standard deviation of 1 K. This is in linewith our confidence about the model equation (note that the model is estimated from the data,hence, it is only an approximation of the true state equation, so that a very low variance-valuefor the stochastic term would drive the retrieval along the wrong state equation). In addition,according to [22], a deterministic AR(2) model fits to the data within a residual error rangingfrom 0.5 to 2 K.

For the emissivity, the initial background matrix has been derived from the IREMIS database and it has been presented in Fig. 13.

This matrix describes the infra-annual variability of the emissivity, so that at a first glanceits variability might be too strong to represent hour-to-hour emissivity variations in clear sky.However, from Fig. 14 we see that the variability of the background derived by the IREMISdata base is very low and quite close to what is expected for hour-to-hour variations. In otherwords, it seems that there is no need to scale down this matrix to go from year-to-year tohour-to-hour variability.

However, for the stochastic term, the covariance matrix has been scaled by a factor 100.This is somewhat arbitrary and has been here considered just to exemplify the role of thestochastic noise over the retrieval.

The down-scaling has been performed according to the procedure here illustrated. Let Bbe a generic covariance matrix of size n × n. The elements of this matrix will be denotedBi,j ; i, j = 1, . . . , n. The correlation matrix, C is defined according to

Ci,j =Bi,j√Bi,iBj,j

; i, j = 1, . . . , n (91)

The matrix B is scaled according to

B(s)i,j =

√Bi,iBj,j

fCi,j ; i, j = 1, . . . , n (92)

where B(s) is the matrix scaled by the factor f . For the present exercise we have used f = 100.The scaling operation above preserves the correlation structure.


Table 10: Summary of the Kalman filter settings concerning the exercise discussed in section9.4.


Emissivity model equation PersistenceSkin Temp. model equation Autoregressive process of order 2

Emissivity true values third column in Tab. 5Emissivity initialization (at time=1) fourth column in Tab. 5

Emissivity initial background see Fig. 13Emissivity stochastic covariance that in Fig. 13 scaled by a factor f = 100

Skin Temp. true values Fig. 12Skin Temp. initial value (at time=1) True-4 K

Skin Temp. initial background 1.332 K2

Skin Temp. stochastic variance 1 K2

Observational Covariance matrix diagonal, from SEVIRI radiometric noise (see Fig. 17)Atmospheric profiles assumed known, see Fig. 16Convergence criterion cost function

The various assumptions, first guess, background vectors and covariance matrix used in thepresent simulation exercise, are summarized in Table 10.

The results for temperature are shown in Fig. 38 and Fig. 39. If we compare the resultswith those in Fig. 18 and Fig. 19, we see that the Kalman filter achieves an accuracy of about0.1 K, which is about a factor 2 better than the accuracy shown in Fig. 19. From Fig. 39 wesee that the error bar get to its asymptotic value just after a few time-steps. The slight increasein the variability of the retrieval at the end of the day is due to the dependence of the resulton the data points. The signal-to-noise ratio of the data points is lower at the end of the timeinterval shown in Fig. 39.

The results for emissivity are shown in Fig. 40. On overall we have a very good performanceat each channel and the retrieval rapidly converges to the true value. The difference betweenRetrieval and True, depicted in Fig. 41, shows that the retrieval is a bit downwards biased,although this bias affect the third decimal digit. In other words, the retrieval coincides withthe truth within the first two digits.

The decrease of the error bar is much more evident in the case of emissivity because of thedown scaling of the stochastic noise term. This behaviour is better evidenced in Fig. 42, whichshows the retrieval for the channel 5, that is the channel sensitive to the reststrahlen bands.

Based on extensive simulations, which are here not shown for the sake of brevity, it seemsthat the advantage of the Kalman filter over the optimal estimation scheme could be twofold.

First, the system is less expensive in terms of storage and computational costs. The baselinescheme we have developed takes ≈ 14 s to run for a complete daily cycle (performance obtainedon single Intel i7 Cpu 3.33 MHz). For the same daily cycle, the Optimal Estimation takesabout 34 s.

Second the Kalman filter appears to be less sensitive to the First Guess initialization,because of the potential extra information provided by the system equation.

In principle, once we have initialized the Kalman filter this could run forever. However, ithas to be stressed that the autoregressive coefficients, that is the dynamics of the daily cycle,can change with time. Moreover, the daily cycle is also influenced by atmospheric weatherprocesses, which add day-to-day variability therefore even on the scale of a few days the dailycycle can change its dynamics.


Figure 38: Results of the retrieval exercise for temperature with the Kalman filter scheme.

Figure 39: Difference Retrieval-True for the case shown in Fig. 38. The accuracy of theretrieval (square root of the diagonal of the covariance matrix) is shown in form of a ±1σtolerance interval.


Figure 40: Results of the retrieval exercise for emissivity with the Kalman filter scheme.



Figure 42: Difference Retrieval-True for the case shown in Fig. 40, but now only the result forthe channel 5 is shown.


9.5 Kalman filter with a persistence model equation for skin temper-ature

We have shown and exemplified how the daily cycle is well represented by a Markov or autore-gressive process of second order. However, the SEVIRI observations in the atmospheric window,that is the data, have a strong dependence on skin temperature. This has been exemplified,e.g., in Fig. 37. The dependence is so strong that we can use directly the SEVIRI observationsat channel 833.3 cm−1 to estimate the dynamical behaviour of the skin temperature during theday.

The consequence of this strong dependence is that that we can relax the constraint overthe state equation for Ts provided we build up a retrieval system which is mostly driven by thedata.

This will be exemplified in this section with a the use of a simple persistence state equationfor the skin temperature,

Ts(t+ 1) = Ts(t) + ηT (t+ 1) (93)

We know that this model is not correct since it cannot reproduce the daily cyclic behaviourexpected in clear sky for land surface. It could be a fair model for sea surface, where thermalinertia of water strongly damps the effect of the solar cycle, however it cannot represent a goodmodel for land surface.

It has to be stressed that within the context of the Kalman filter we do not need the exactmodel equation of a given parameter, provided the parameters are strongly driven by thedata.

The present example also allows us to clarify the meaning of the stochastic noise covaraince,Sη (e.g. Eq. 64). This parameter makes it possible to tradeoff between a retrieval driven onlyby the data (its variance set to +∞) and one driven by the model alone (its variance just setto zero).

The exercise shown in the previous section 9.4 is now considered by replacing the modelequation (78) with that of representing a persistence (Eq. (93)). The variance of the stochasticnoise term is set to 1 K2. The only thing is changed with respect to the set up shown in Table10 is the state equation for the skin temperature, which is now set up to a persistence.

The results for temperature are shown in Fig. 43 and Fig. 44. If we compare with theresults provided in section 9.4, we see that there is not a significant difference. This exemplifiesthat for the problem at hand, the observations provide a tidy constraint for the skin temperature.

The results for emissivity are also equivalent to those shown in section 9.4. They are notshown here for the sake of brevity.

We come now to the issue of the stochastic noise term as a tradeoff parameter betweendata and state equation. We can drive the retrieval closer to the state equation by choosing alower value for the stochastic noise term. To exemplify the situation let us consider the case ofskin temperature and let us set its related stochastic noise variance equal to 0.

The temperature retrieval for this case case is exemplified in Fig. 45 and Fig. 46. We seethat at a given time the retrieval looses its dependence on the data and just follows the stateequation. At that time the error gets almost to zero as it is possible to see from Fig. 46, whichagain gives evidence that the retrieval is driven only by the model equation.


Figure 43: Results of the retrieval exercise for temperature with the Kalman filter scheme.The case shown uses a persistence model for the state equation of both emissivity and skintemperature.



Figure 45: Results of the retrieval exercise for temperature with the Kalman filter scheme.The case shown uses a persistence model for the state equation of both emissivity and skintemperature. The variance of the stochastic noise term for the skin temperature has been setto zero.


10 APPLICATION TO REAL SEVIRI OBSERVATIONS 80

Figure 47: Target areas in Spain, Sahara desert and Mediterranean basin used to check theretrieval algorithms.

10 Application to real SEVIRI observations

To check the stability, sanity and quality of the retrieval methodology, we have applied it tothree relatively small target areas, one in Spain, the second in the Sahara desert and the thirdin the ocean. These are shown in Fig. 47. The area in Spain is close to the city of Seville andthat in North-West Africa is in the middle of the Algerian desert. In both cases the altitude isclose to sea level. The ocean target area is situated to South of the Sardinia Island

The size of these target areas is 0.5 degrees latitude by 0.5 degrees longitude and correspondsto one box of the ECMWF grid mesh (see, e.g., Fig. 9).

For the Spanish location we have a total of 183 SEVIRI pixels, for the site in Africa wehave a total of 219 SEVIRI pixels and 178 for the ocean site.

The retrieval exercise, while considering real observations, is of size as large as still to allowus to keep the software package under control, which is important for our purpose of checkingthe quality of the retrieval and consistency of the various software tools.

The land cover of the Spanish site is a mosaic of cultivated areas, with green grass, foliageand bare soil, that corresponding to the Sahara desert is just a desert sand flat area, withno vegetation. The area is interesting to us because day-to-day variations are unlikely, andwe expect that emissivity does not vary on two consecutive days. The Spanish area is alsoexpected to be stable because the month of July corresponds to the dry season in the South-European Mediterranean region. The ocean target area is expected to be well representedthrough Masuda model [36] as far as emissivity is concerned, therefore it represents a case forwhich ECMWF analysis should provide a valid reference to check the accuracy of the retrievedskin temperature.

Figure 48 shows the SEVIRI channel 7 (833.3 cm−1) for a typical clear day as a functionof the location. While the forcing of the solar cycle is striking for the two land target area, forthe ocean site this is greatly attenuated because of the relatively large thermal inertia of ocean.


Figure 48: SEVIRI channel 7 observation for one clear sky day and for the the three test sites.


To avoid confusion, the analysis for sea surface is postponed to section 10.2, while in thenext section (section 10.1) we will be dealing with the analysis over land surface.

10.1 Results over land surface

For the retrieval exercise the Optimal Estimation (OE) and the Kalman filter (KF) schemeshave been initialized with the emissivity Fist Guess computed on the basis of the IREMIS database for the period 2003-2011, but the year 2010, which has been left for comparison/validation.The atmospheric state vector is the ECMWF analysis, space-time interpolated to the positionof SEVIRI observations.

Although, the ECMWF analysis provides, in general, enough information for all hoursof the day concerning atmospheric parameters, in order to avoid to add unknown variabilitysources, the present exercise consider channels 4,6 and 7 (1149.4, 925.9 and 833.3 cm−1) alone.These are window channels, for which most of the contribution comes from the surface emission.

For the Kalman filter we have used an AR(2) model for skin temperature and a simplepersistence for emissivity. The settings of the two retrieval case studies are summarized inTable 11 for the optimal estimation and Table 12 for the Kalman filter.

Table 11: Summary of the settings for the optimal estimation scheme, which applies to theJuly 2010 case study shown in section 10.


Time slot width 1 hourEmissivity true values unknown, IREMIS data base for year 2010 used for comparison

Emissivity FG and BG vectors (FG=BG) average from IREMIS data base, over the years 2003-2011, but 2010Emissivity static background from IREMIS data base, 2003-2011 years, but 2010

Skin Temp. true values unknown, ECMWF used for comparisonSkin Temp. FG and BG vectors (FG=BG) ECMWF analysis

Skin Temp. static background diagonal with elements set to 1.332 K2

Observational Covariance matrix diagonal, from SEVIRI radiometric noise (see Fig. 17)Atmospheric profiles assumed known, time-space colocated ECMWF analysisConvergence criterion cost function

Table 12: Summary of the settings for the Kalman Filter scheme, which applies to the July2010 case study shown in section 10.


Emissivity model equation PersistenceSkin Temp. model equation Autoregressive process of order 2

Emissivity true values unknown, IREMIS data base for year 2010 used for comparisonEmissivity initialization (at time=1) average from IREMIS data base, over the years 2003-2011, but 2010

Emissivity initial background from IREMIS data base, 2003-2011 years, but 2010Emissivity stochastic covariance as line above scaled down with f = 1

Skin Temp. true values unknown, ECMWF analysis used for comparisonSkin Temp. initial value (at time=1) ECMWF analysis at 00:00 hour




As said the background covariance matrix has been obtained for each site from the IREMISdata base. We have checked that these matrices shows a variability for emissivity, which is well


Figure 49: Number of iterations needed to converge as a function of the scaling factor, f .Note that we put an upper limit of 10 to the number of iterations, therefore when this limit isreached, the procedure has not converged.

below 0.01. For this reason, when considering the emissivity stochastic noise term, its relatedcovariance matrix has not been scaled down. In other word, the present result applies to astochastic noise term which is exactly equal to the initial value of the background covariance.

We have performed a sensitivity of the retrieval on the scaling factor, f introduced in Eq.(92). For f = 1 we have checked that the scheme converges (in the χ2-sense) almost at eachiteration step. This means that f = 1 provides a good balance between data and model. Forf = 100 we do not converge in many steps, and for f = 10000 we do not converge at all. Thisanalysis is shown in Fig. 49 in terms of number of iterations at each time step from t1 to t96,this set covers the time interval 00:00 to 24:00 hours. The upper limit for the iterations numberis 10, so that when this upper limit is reached, the cost function remains above the thresholdχ2 tolerance value corresponding to a confidence interval of 95%, hence it is not minimized.The conclusion is that the more we constrain the emissivity retrieval to the persistence stateequation the more we get results which are inconsistent with the observations.

For the optimal estimation, we have used a time slot width of one hour. This is in agreementwith the results found in simulation in section 8.1. In addition, it is consistent with theobservations in Fig. 48, which show a very large time gradient because of the strong dailycycle.

The retrieved emissivity, averaged over all pixels and over the day is shown in Tab. 13 forthe case of the Spanish target area.

The following aspects arise from the analysis of the results shown in Table 13.


Table 13: Comparison of the retrieved and IREMIS emissivity for the Spanish location.

09 July 2010 Optimal Estimation Kalman Filter IREMIS FGChannel at 833.3 cm−1 0.9727 0.9807 0.9509 0.9651Channel at 925.9 cm−1 0.9666 0.9730 0.9563 0.9637Channel at 1149.4 cm−1 0.9567 0.9600 0.9567 0.9628

10 July 2010Channel at 833.3 cm−1 0.9659 0.9805 0.9509 0.9651Channel at 925.9 cm−1 0.9643 0.9765 0.9563 0.9637Channel at 1149.4 cm−1 0.9527 0.9635 0.9567 0.9628


• the retrieval methodology (both Kalman filter and Optimal Estimation) is stable: for twoconsecutive days we get almost the same emissivity.

• The comparison with IREMIS is fairly good, although both Kalman filter and OptimalEstimation schemes show an emissivity, which is larger than that of IREMIS. This isespecially true for the channel at 833.3 cm−1.

• Optimal Estimation and Kalman filter are consistent, although the Optimal Estimationremains closer to the first guess emissivity.

The above aspects are further evidenced by analyzing the results in Table 14, which refersto the Sahara desert, for which we know that the day-to-day variability has to be negligible.In addition, the scene homogeneity tend to minimize the effect of different geometry of theMODIS and SEVIRI field view.

Also in this case we see a very nice consistency between Optimal Estimation and Kalmanfilter. Both schemes tend to evidence a slight downward bias in the IREMIS emissivity at 833.3cm−1.


Table 14: Comparison of the retrieved and IREMIS emissivity for the Sahara desert location.

09 July 2010 Optimal Estimation Kalman Filter IREMIS FGChannel at 833.3 cm−1 0.9720 0.9744 0.9652 0.9693Channel at 925.9 cm−1 0.9423 0.9388 0.9428 0.9457Channel at 1149.4 cm−1 0.7458 0.7419 0.7381 0.7408




Figure 50: Example of emissivity retrieval and related accuracy (error bars) at the level of onesingle SEVIRI pixel. The retrieval refers to the Sahara desert location.

Figure 50 shows the emissivity retrieval for the channel at 1049.4 cm−1 for one single dayand one SEVIRI pixel as a function time. This plot is shown to exemplify the highest timeresolution, which is possible to achieve with the Kalman filter. The time resolution coincideswith the SEVIRI repeat cycle of 15 min. In addition, the plot also allows us to show theaccuracy of the estimate at the level of the highest time-space resolution, 1/4 hour and thesingle SEVIRI pixel. The error for emissivity is normally below ±0.01. Note that this error isnot expected to go to zero because of the non-zero stochastic term we have used in the analysis.The stochastic term is at each iterate equals to the static background derived form the IREMISdata base. We could use a different strategy, however the strategy we have used does not seemto over-constrain the data towards the state equation.

The temperature retrieval, corresponding to the emissivity estimate shown in Fig. 50 canbe seen in Fig. 51, which again exemplify the variability expected for temperature. Also for thiscase, the presence of a stochastic term with variance equal to 1 K2 avoid to produce ridiculouserror bars going to zero and prevent the scheme from being biased toward the state model.

One potential interesting aspect, which arises from Fig. 50 is the temperature dependenceof emissivity at 1149.4 cm−1. This channel is very sensitive to quartz absorption, and we knowthat the refractive index of quartz is sensitive to temperature. The slight emissivity decrease,which is seen in the hottest part of the day, is in agreement with the theoretical dependence ofquartz emissivity on temperature. Thus the behavior shown in Fig. 50 could be the results ofquartz or minerals contained in the desert sand. For the desert case, this behavior is also seenat the other two window channels and even if we average over the horizontal spatial coordinate.This is shown in Fig. 52 which refers to the Sahara desert on 10 July 2010. The emissivity hasbeen averaged over the 219 adjacent SEVIRI pixels.

The most striking results we obtain for temperature is that the ECMWF analysis over landis largely downward biased either we consider the Spanish location or the desert area. The biasremains even if we consider spatial averaging as shown in Fig. 53 (Seville) and Fig. 54 (Saharadesert). The magnitude of the bias can reach even 10 K in the hottest hours of the day, andtend to largely decrease during nighttime hours.


Figure 51: As Fig. 50, but for temperature.

Figure 52: Emissivity for the Sahara desert averaged over the 219 adjacent pixels.


Figure 53: Skin temperature for the Spanish location averaged over the 183 adjacent pixels.

Figure 54: Skin temperature for the Sahara desert location averaged over the 219 adjacentpixels.


Figure 53 and Fig. 54 also provide a comparison with the statistical split window approachmethodology developed for SEVIRI by [49]. The statistical retrieval, in addition to the SEVIRIbrightness temperature at channels 6 and 7, also requires the emissivity at the same channels.The emissivity used for the calculation has been extracted from the IREMIS data base. It isinteresting to note the large sensitivity of the skin temperature on emissivity. Our emissivityretrieval differs from the IREMIS values for less than 1%, however this cause differences of upto 2-3 K again in the hottest pert of the day. In passing, the statistical method seems to givevery nice results for Spain. For the Sahara desert the temperature is largely underestimated atmidday.

To sum up, we have that both the Optimal Estimation with time accumulation of obser-vations and the Kalman filter seem to provide both very stable and convincing results. TheKalman filter is faster of a factor 2 and more because of the possibility of sequential inputtingof the observations. Also the dimensionality of the Kalman filter is lower than that of Opti-mal Estimation, although for the baseline algorithm, which is dedicated to emissivity and skintemperature, this is not really a problem, unless we consider spatial constraint.

One advantage of the Kalman filter is the possibility to parameterize within the schemethe strong dynamical information of the diurnal cycle. However, although in a non parametricform, this is also contained in the radiance itself how our analysis has shown, therefore a wellbalanced Optimal Estimation should benefit, as well.


Figure 55: Retrieved skin temperature (bottom panel) for a site in the Sahara desert. Theretrieval has been obtained with the Kalman filter for ten consecutive days. In the legend,ECMWF Ts analysis refers to the skin temperature analysis at the canonical hours within aday, whereas ECMWF Ts is the ECMWF skin temperature linearly extrapolated to the SEVIRItime steps. The upper panel in the figure also shows the quality of the reconstructed radiance(channel at 833.33 cm−1). 1 r.u.=1 W m−2 sr−1 (cm−1)−1

10.1.1 Running the Kalman filter over a long time span

In the previous examples results have been obtained for a time span of one day. In this sectionwe consider a time span of ten days characterized by clear sky.

The retrieval for skin temperature is exemplified in Fig. 55 which refers to the retrieval forten consecutive days for a location in the Sahara desert. The analysis shown in Fig. 55 hasbeen obtained with the settings shown in Tab. 12. However, we have used a persistence modelalso for the skin temperature and the scaling factor, f has been set to f = 10.

Form Fig. 55 it is possible to see that a slight cloudiness affects the observations at thebeginning of the second day. We do not skip these observations when performing the retrieval,therefore Fig. 55 shows that slight cloudiness does not bring the Kalman filter to an unstableregion. In other words, the stability of the filter is not influenced by slight cloudiness, althoughthis information is forward propagated through the forecast.

Figure 56 shows the time sequence of observed and calculated radiances, along with thecorresponding spectral residual, for the three window channels, which have been used in theretrieval analysis. It is seen that the quality of the fit is fairly good for the whole time spanof ten days. The relative misfit corresponding to the slight cloudiness at the beginning of thesecond day is evident in Fig. 56.

The same conclusion is arrived at if we focus attention to the minimization of the costfunction, that is the χ2-form expressed by Eq. 61 (multiplied by a factor of 2!). It is wellknown that this form distributes according to a χ2 variable with M degrees of freedom, withM the size of the radiance vector. A threshold, χ2

th to check the goodness of the minimization

is therefore obtained, e.g., by considering a ±3σ interval, which gives χ2th = M + 3

√(2M).

Along with the χ2 form, another useful parameter to check the quality of the retrievalsystem is the number of iterations, which at each time-step are needed to converge, that is toreduce the χ2 form below the thresholds.

For the retrieval analysis shown in Fig. 55, the number of iterations and the χ2 form areshown in Fig. 57. As already said, we allow the retrieval scheme for at most ten iterations ateach time step, therefore when the number of iterations reaches the value of ten it means thatthe scheme has not converged.


Figure 56: Calculated and observed radiances along with the related spectral residual for thethe three SEVIRI window channels used for the retrieval analysis shown in Fig.55. The errorconfidence interval shown in figure refers to the SEVIRI NEDN. 1 r.u.=1 W m−2 sr−1 (cm−1)−1

There are important aspects, which can be drawn from Fig. 57. Non-convergence is causedby light cloudiness. In general, it is confined at nighttime (supposedly when the signal-to-ratiois smaller). Non-convergence does not necessarily drive the filter in regions of instability.

The retrieval for emissivity is shown in Fig. 58 along with the time evolution of skintemperature. It is seen that the emissivity follows the day-night cycle with relative peaksat nighttime. It is interesting to see that the deep minimum in emissivity occurring at thethird day corresponds to the highest skin temperature observed in the ten days at hand. Thisbehaviour tends to confirm that there is a correlation temperature-emissivity for the desertsand. In fact, it is well know that quartz has a refractive index depending on temperature, thisdependence dictates that emissivity is anti-correlated with temperature.

Furthermore, Fig. 59 exemplifies the precision of the retrieval for the emissivity at 1149.40cm−1. It is seen that day-night variations are significant within retrieval precision. This iscomputed as the square root of the diagonal of the a-posteriori or analysis covariance matrix.

Finally, we present and discuss the correlation between the retrieved skin temperature andECMWF Ts. This analysis is summarized Fig. 60, which shows a scatter plot of Ts at thefour canonical hours of the analysis. From this figure it is seen that in general we have a goodagreement for nighttime hours (that is 0:00, 6:00), at 18:00 ECMWF model tends to be slightlydownward biased, whereas the size of bias becomes large at midday when the sun angle is atits highest value. This behaviour is also confirmed for the case of the Seville site (see Fig. 60).For the case of Seville we have more data points because the analysis refers to the whole monthof July (see next section).


Figure 57: Cost function (left) and number of iterations as a function of the time step corre-sponding to the retrieval analysis shown in Fig. 55. The two upper panels allow the readerto identify which radiances causes a non-convergence of the scheme. 1 r.u.=1 W m−2 sr−1

(cm−1)−1

Figure 58: Ts − ε time evolution for the retrieval exercise shown in Fig. 55.

10.1.2 The effect of clouds

If detected cloudy radiances can be skipped in either schemes, that is Optimal Estimationand Kalman filter. To this respect, it should be noted that within the context of OptimalEstimation, one single cloudy pixel within the given time slot, could cause rejection of theother clear sky pixels. This is not the case for the Kalman filter where we can fully exploit thetime resolution of radiances, since we can sequentially process the observations.

In case clouds are not detected, the effect over the Kalman filter is not critical unless wedeal with severe overcast conditions. Figure 61 shows SEVIRI observations for the channel at833.33 cm−1 and for the whole month of July 2010. Observations from both Sahara and Sevillesites are shown and cloudy observations are marked with circles. The cloud mask used is thatoperational for SEVIRI. It is seen that there are obvious undetected cloudy radiances in bothrecords. Cloudy pixels are easily seen if we consider the time continuity expected in the dailycycle.

Figure 62 shows the retrieval for skin temperature corresponding to whole month of July.The retrieval analysis is shown for one single SEVIRI pixel and for the two sites of Saharadesert and Seville. The analysis shown in Fig. 62 has been obtained with the settings shownin Tab. 12. However, we have used a persistence model also for the skin temperature and thescaling factor, f has been set to f = 10.


Figure 59: Exemplifying the retrieval precision for the emissivity at 1149.40 cm−1. The figureshows the ±σ tolerance interval (square root of the diagonal of the a-posteriori covariancematrix.

Figure 60: Scatter plot of Ts from the retrieval analysis and ECMWF model at the fourcanonical hours of the ECMWF analysis.

The analysis has been performed only for clear sky soundings (according to the operationalSEVIRI cloud mask). Cloudy radiances are skipped in the analysis, which means that weuse a time step which is not a constant. Missing values of the skin temperature in Fig. 62corresponds to cloudy radiances. However, undetected cloudy radiances are processed, whichcause occasional sharp gradients in the time behaviour of Ts.

These sharp gradients can be eliminated in case we consider only retrievals for which thecost function χ2 has been correctly minimized. This is shown in Fig. 63 where we consideronly the retrievals, which correspond to converged solutions.

The convergence criterion, which we stress is based on χ2-thresholding of the cost function,also attenuates the effect of undetected clouds. In presence of cloudy radiances the cost functionis normally not minimized below the prescribed threshold. It is not easy to assess this extraremoval of cloud, because undetected cloudy radiances, as those shown in Fig. 61, need to bepicked by visual inspection of the time sequences of data. This is a very long and prone toerrors procedure. What we can say is that the convergence criterion can act as a further filterto cloudy radiances. This exemplified in Fig. 64, which shows for the Seville station a series ofpossibly undetected cloudy radiances for which the convergence criterion fails.

Coming back to the results for the surface parameters, as shown in the previous section,we have that the ECMWF model compares fairly good with the retrieval at night-time hours,whereas during the day ECMWF skin temperature is largely downward biased.


Spatial averaging does not change the above conclusion as it is shown in Fig. 65, whichhas been obtained by averaging the results over adjacent pixels (219 for the Sahara desert and183 for the Seville site).

In general, the spectral residual is reduced within the SEVIRI noise. This is shown in Fig.66 for two samples, one corresponding to the Sahara desert and the other one to the Sevillesite. For the sake of brevity only the spectral residual corresponding to channel at 833.33 cm−1

is shown. Similar results apply to the other two window channels considered in the analysis.

The retrieval for emissivity are shown in Fig. 67. This has been averaged over adjacentclear sky pixels. We stress that clear sky is defined according to the SEVIRI cloud mask, whichcan still contain undetected cloudiness. The effect of cloudiness is visible for both sites, Saharadesert and Seville. However, a cyclic day-night behaviour is evident for both locations.

Monthly maps of the skin temperature are shown in Fig. 68. It is seen that even in a boxof size 0.05 × 0.05 degrees we can appreciate a temperature gradient. This is confined to 4◦Cfor the desert site and reaches 6◦C for the Seville location.


Figure 61: Plot of SEVIRI radiances for the channel at 833.33 cm−1 showing the presence ofclouds according to the operational SEVIRI cloud mask. The zoom in b) and d) shows thatthere are both false negative (clear sky flagged cloudy) and false positive (cloudy sky flaggedclear). 1 r.u.=1 W m−2 sr−1 (cm−1)−1


Figure 62: Retrieved skin temperature for the two sites, Sahara desert and Seville. Results areshown only for one SEVIRI pixel. The retrieval has been obtained with the Kalman filter for thewhole month of July. In the legend, ECMWF Ts analysis refers to the skin temperature analysisat the canonical hours within a day, whereas ECMWF Ts is the ECMWF skin temperaturelinearly extrapolated to the SEVIRI time steps.


Figure 63: As figure 62, but now the retrieval is shown which corresponds to the χ2 formminimized below the thresholds (bottom panels).


Figure 64: The figure show a time sequence of SEVIRI radiances corresponding to the channelat 833.33 cm−1. The figure exemplifies the presence of possibly undetected cloudy radiances,which are removed because of lack of minimization of the cost function below the χ2-threshold.1 r.u.=1 W m−2 sr−1 (cm−1)−1


Figure 65: As figure 62, but now the retrieval has been spatially averaged over adjacent pixels(219 for the Sahara desert target area and 183 for the Seville site).


Figure 66: Spectral residual time series for the channel at 833.33 cm−1. Left, Sahara desert;right, Seville site. The spectral residual refer to the SEVIRI pixels whose retrieval analysis forskin temperature as been shown in Fig. 62.


Figure 67: Emissivity retrieval averaged over adjacent pixels (219 for the Sahara desert targetarea and 183 for the Seville site).


Figure 68: Monthly map of the skin temperature for the Sahara desert target area and theSeville site).


10.2 Results over sea surface.

For the case of sea surface the forward model has been operated in the specular mode and thecovariance matrix for emissivity has been derived from the Masuda model [36]. For the givenangle, the emissivity has been computed for a wind speed ranging in the interval [0,15] m/swith a step of 2.5 m/s. The emissivity has been computed at a spectral resolution of 0.25 cm−1

and convolved with the SEVIRI instrumental response to produce the emissivity at the sevenSEVIRI channels. The target area used to check the performance of the retrieval methods isthat shown in Fig. 47. This area is imaged by SEVIRI at an average zenith angle of 46 degrees.The covariance matrix of emissivity derived for this angle is shown in Fig. 69.

Other settings of the retrieval methods, which are relevant to the application for sea surfaceare summarized in Tab. 15 (Optimal Estimation) and Tab. 16 (Kalman filter).

Table 15: Summary of the settings for the optimal estimation scheme, which applies to theJuly 2010 case study for sea surface shown in section 10.2.


Time slot width 1 hourEmissivity true values unknown, Masuda model used for comparison

Emissivity FG and BG vectors (FG=BG) from Masuda modelEmissivity static background from Masuda model

Skin Temp. true values unknown, ECMWF used for comparisonSkin Temp. FG and BG vectors (FG=BG) ECMWF analysis

Skin Temp. static background diagonal with elements set to 1. K2


Table 16: Summary of the settings for the Kalman Filter scheme, which applies to the July2010 case study for sea surface shown in section 10.2.


Emissivity model equation PersistenceSkin Temp. model equation Persistence

Emissivity true values unknown, Masuda model used for comparisonEmissivity initialization (at time=1) Masuda model

Emissivity initial background Masuda ModelEmissivity stochastic covariance as line above scaled down with f = 10


Skin Temp. initial background 1. K2



For the retrieval exercise shown in this section we have used the SEVIRI window channelsalone, that is the channel at 833.33 cm−1, 925.93 cm−1 and 11409.40 cm−1.

We first discuss the analysis for one single day, 31 July 2010. The retrieval will be shownfor all the 178 SEVIRI pixels considered for the target area presented in Fig. 47. Once againclear sky has been qualified according to the operational cloud mask, therefore there is thepossibility of undetected cloudiness.

The results for the skin temperature are shown in Fig. 70 for all the 178 pixels. Note thatthe ECMWF skin temperature coincides with the First Guess and the Background state vectors


Figure 69: Emissivity covariance matrix derived from Masuda’s model for a zenith angle of 46degress.

in the case of the Optimal Estimation retrieval. For the case of Kalman filter, the ECMWFmodel is used to initialize the scheme at the time t = 1, the ECMWF analysis is not usedto eventually update the filter at later times. It is important to keep in mind this differencebetween the two schemes when inter-comparing the results. The Optimal Estimation yields aresult which is closer to the ECMWF analysis, but this is not surprising when we consider theway we have used to build up First Guess and Background state vectors.

The closeness of the Optimal Estimation retrieval to ECMWF is better seen when weconsider the spatial average over the full box of size 0.05 × 0.05 degrees, which is shown inpanel c) of Fig. 70. Both Optimal Estimation and Kalman filter agree with showing a surfacetemperature, which is a bit lower than that of ECMWF. However, the Kalman filter shows alarger difference from the ECMWF model. However, it should be considered that this differenceis of the order of 0.75 K on average. Also interesting is to see that Optimal Estimation andKalman filter are almost identical for times close to t = 1, they begin to separate once we moveahead along the day and the Kalman filter looses memory on the initialization point.

It is interesting to see that the skin temperature reaches a maximum around 3 p.m. localtime, rather than at midday as shown by ECMWF model. This is in agreement with [20],which shows that during the daytime, solar heating may lead to the formation of a near-surfacediurnal warm layer, particularly in regions with low wind speeds. Analysis of TMI and AVHRRskin temperature have revealed that the onset of warming begins as early as 8 a.m. and peaksnear 3 p.m., with a magnitude of 2.8◦C during favorable conditions.

The difference in the retrieval for skin temperature between Optimal Estimation andKalman Filter is explained with the diverse retrieval for the emissivity. In fact, Optimal Es-timation remains quite close to the first guess, whereas the Kalman filter leads toward anemissivity, which is slightly higher than the first guess. This is exemplified in Fig. 71. Onceagain it is important to stress that the Optimal Estimation is initialized at each time step with


Figure 70: Retrieval analysis for skin temperature. a) Optimal estimation; b) Kalman filter; c)the retrieval has been spatially averaged over the grid box of size 0.05× 0.05 degrees shown inFig. 47.

an appropriate First Guess, whereas the Kalman filter is initialized only at t = 1. On overall,we see from Fig. 71 that Optimal Estimation is not capable to develop an emissivity estimatedifferent from the First Guess, whereas Kalman filter, which is less constrained by the FirstGuess, seems to lead toward an emissivity, which, as said, is slightly higher than the first guess.

Figure 72 shows the spatial distribution of the daily average of the skin temperature withinthe grid box of size 0.05×0.05 degrees, whose SEVIRI observations have been analyzed (see Fig.47). It is interesting to note that we see a spatial distribution with a gradient North to South(which is expected) along with a relative cold structure which stretches from the North-Eastedge of the box to that in the direction West-South.

The retrieval exercise exemplified in this section also deserves to address the problem ofthe tuning of the stochastic variance term in the Kalman filter. As shown in tab. 16, thestochastic variance term corresponding to skin temperature has been set to 1 K2. This termexpress our confidence on the state equation, which has been assumed to be a persistencemodel. This choice for sea surface is a good one also on the physical ground of the big thermalinertia of ocean. Thus, a variability of 1 K2 within 15 min could be really too large. To checkthe sensitivity of the results to this assumption, we have performed new calculations for theMediterranean target area corresponding to values of the stochastic term equal to 0.25 K2 and0.01 K2. The results for the skin temperature, averaged over the full target area, are shownin Fig. 73 and they tend to show a better agreement with the ECMWF analysis in case thestochastic term is in between 0.25 and 0.01 K2. The case corresponding to 1 K2 seems to bedownward biased of approximately 0.5 K.

This final example also stresses that our exercises are intended for illustrative purposes of


Figure 71: Retrieval analysis for emissivity. The retrieval has been spatially averaged over thegrid box of size 0.05× 0.05 degrees shown in Fig. 47.

Figure 72: Maps of the daily skin temperature obtained with the two retrieval scheme.

the various methodologies. The proper setting of the parameters need to be done with ad hoctuning/validation data sets.


Figure 73: Kalman filter retrieval analysis for skin temperature as a function of the stochasticvariance term for Ts. The retrieval has been spatially averaged over the grid box of size0.05× 0.05 degrees shown in Fig. 47.


10.2.1 Running over a long time span with the presence of clouds.

In this section we show the retrieval results obtained by running the Kalman filter for the wholemonth of July for the target area shown in Fig. 47.

In case of sea surface, the daily cycle is less pronounced and the time-continuity expectedin the SEVIRI window channels cannot be used to check a-posteriori for the presence of cloudswithin the SEVIRI field of view. Figure 74 shows a time-sequence for the whole month of Julyof the channel at 833.33 cm−1. Cloudy radiances are identified with the operational SEVIRIcloud mask.

In running the Kalman filter we just skip cloudy radiances, which means that the schemeis run with a time step or sampling interval which is not a constant. Also, it is possible thatthe record of observations contains cloudy radiances. The settings for the present run are thoseshown in Tab. 16.

Figure 74: Example of sea surface SEVIRI observations for the whole month of July. Theobservations refers to the window channel at 833.33 cm−1. The target area is that shown inFig. 47. Cloudy radiances are evidenced with red circles.

As for the land surface, we have that the scheme does not converge for all the clear skyradiances, which is likely the effect of undetected clouds. However, this does not seem to havea strong impact over the results, which show a fair agreement with the ECMWF analysis, as itis possible to see from Fig. 75. The results refer to one single SEVIRI pixel, and the agreementwith the ECMWF model can be partly improved in case we consider only the retrievals, whichcorrespond to a converged χ2 form, that is below threshold. This case is shown in Fig. 76.However, it is confirmed that the retrieval shows a downward bias with respect the ECMWFmodel. This is quantified in Fig. 77, which shows a scatter plot of the ECMWF Ts vs thatretrieved with the Kalman filter.


Figure 75: Top. Retrieved skin temperature for Mediterranean site. Results are shown only forone SEVIRI pixel. The retrieval has been obtained with the Kalman filter for the whole monthof July. In the legend, ECMWF Ts analysis refers to the skin temperature analysis at thecanonical hours within a day, whereas ECMWF Ts is the ECMWF skin temperature linearlyextrapolated to the SEVIRI time steps. Only clear sky radiances are processed. Bottom. χ2

values corresponding to the retrieval shown on the left.


Figure 76: As Fig. 75, but now only the retrieved values which correspond to a converged χ2

are shown.


Figure 77: Example of scatter plot of the ECMWF Ts vs that retrieved. The plot consider onlythe values at the ECMWF analysis canonical hour and correspond to the sample shown in Fig.75 converged χ2 are shown.

The bias between ECMWF analysis and retrieval is quantified in ≈ 0.6◦C, whereas thestandard deviation is of the order of 1◦C. In addition, contrary to the land surface analysis,from Fig. 77 we see that for sea surface there is no special pattern as a function of the hour ofthe day.

Generally, the forward model is capable to reduce the difference between observations andcalculations within the SEVIRI radiometric noise, which testifies the good consistency of theradiative transfer developed for SEVIRI. An example of spectral residual for the three windowchannels is provided in Fig. 78.

Finally, the retrieval for emissivity is shown in Fig. 79. This has been spatially averagedover the 178 SEVIRI pixels inside the grid box of size 0.05 × 0.05 degrees, shown in Fig.47. The retrieval for emissivity confirms that the Kalman filter converges towards a solutionwhich is characterized by a larger emissivity value than that of the initialization point. Inother words, we have that the retrieved emissivity seems to be larger than that predicted withMasuda’s Model [36]. Considering the large angle of view (≈ 46 degrees), which characterizesthe SEVIRI observations at hand, this is expected. In fact, as shown in [52] the Masuda modeltends to underestimate sea surface emissivity for large angles. In addition, we have to considerthat the first guess we use in the analysis is the average value of the emissivity at 7 diversewind speeds (0 to 15 m/s with a step of 2.5 m/s) and this average value could well be differentfrom the real condition of the sea during observations.


Figure 78: Calculated and observed radiances along with the related spectral residual for thethe three SEVIRI window channels used for the retrieval analysis shown in Fig.75. The errorconfidence interval shown in figure refers to the SEVIRI NEDN. 1 r.u.=1 W m−2 sr−1 (cm−1)−1


Figure 79: Kalman filter. Retrieval analysis for emissivity. The retrieval has been spatiallyaveraged over the grid box of size 0.05× 0.05 degrees shown in Fig. 47.


10.2.2 Updating the Kalman filter with ECMWF analysis

In the previous section we have discussed the implementation of the Kalman filter with apersistence model in which the scheme is initialized at the very beginning, that is at t = 1.After that the filter evolves according to the model equation and data. This has the advantagethat the filter reaches asymptotically a regime state, in which we loose memory about the waywe initialize the system.

However, in presence of clouds it is questionable if this regime state is ever reached. Fur-thermore, undetected clouds could add biases to short time evolution of the filter itself. In thiscase we could update or re-initialize the filter with new external information, e.g., the ECMWFanalysis which is available each six hours.

The ECMWF analysis is itself a model product, therefore the updating is equivalent touse ECMWF analysis for the state equation or model. This could be good for temperature,however for the case of emissivity, which is not provided from ECMWF model, the updatingjust reset the emissivity to its first guess.

To show the flexibility of the Kalman scheme, we have implemented a version of the filterin which the skin temperature is set to the ECMWF analysis, which means that each six hoursthe filter is re-initialized. At the re-initialization time the emissivity is set back to its FirstGuess value.

This implementation is here only intended for illustrative purposes. It is not said that thisis the optimal way to process SEVIRI observations.

The scheme has been run for the whole month of July and one single SEVIRI pixel belongingto the sea surface pattern shown in Fig. 47. Figure 80 shows the difference with the equivalentscheme with no re-initialization. It is seen that for the skin temperature differences are normallywell within ±0.5◦C. Occasionally we see spikes of 1− 2◦C, however this occurs close to cloudyradiances. For emissivity, as expected, the filter with re-initialization remains closer to theFirst Guess and does not seem to reach a regime state. We stress that this is shown forillustrative purposes alone. We could update the skin temperature alone, we could use ECMWFanalysis as the state equation at each time, and so on. What we can say at this stage, is thatobservations show a very high information content for skin temperature, therefore whateverthe model equation we use, retrieval results are fairly good and stable. However, when dealingwith atmospheric parameters the use of the ECMWF analysis as just the model equation couldbecome a need because satellite observations could provide a loose constraint.


Figure 80: Exemplifying the difference with two different implementation of the Kalman filter.The difference shown in figure corresponds to the two following implementations, 1) Kalmanfilter initialized each six hours with the ECMWF analysis and 2) initialized only once at t = 1.The difference 1)-2) is considered. Panel a) and b) refers to the skin temperature, whereaspanel c) refers to emissivity.

11 KALMAN FILTER INCLUDING ATMOSPHERIC PARAMETERS,[T,Q,O] 116

11 Kalman filter including atmospheric parameters,[T,Q,O]

.

In this section we discuss a possible implementation of the Kalman filter, which includesalso the retrieval of atmospheric parameters. The scheme is mostly intended for SEVIRI thathas a limited number of channels, hence the corresponding radiances have a relatively low levelof information content.

Let R(σi) be the radiance at a generic wave number, σi, with i = 1, . . . ,M and M thenumber of spectral channels at hand. To simplify notation we will write Ri for R(σi) for R(σi).

The spectral radiance can be linearized around an initial or first guess state vector accordingto

Ri = R0i(σ) +∂Ri∂Ts

∆Ts +∂Ri∂εi

∆εi +N∑j=1

∂Ri∂Tj

∆Tj +N∑j=1

∂Ri∂Qj

∆Qj +N∑j=1

∂Ri∂Oj

∆Oj (94)

where Ts is the skin tempertaure, εi the emissivity for the channel i, T = (T1, · · · , TN ) isthe temperature profile, Q = (Q1, · · · , QN ) is the water vapour mixing ratio profile and O =(O1, · · · , ON ) is the ozone mixing ratio profile. Furthermore, N is the number of layers used todiscretize the atmopsheric variables. Moreover, we have

∆Ts = Ts − T0s∆εi = εi − ε0i∆Tj = Tj − T0j∆Qj = Qj −Q0j

∆Oj = Oj −O0j

(95)

where the underscript 0 indicates initial or first guess values and R0i is the radiance for channeli corresponding to the initial state vector.

Of the seven SEVIRI channels, so long we have considered in this study, three are windowchannels (833.33 cm−1, 925.90 cm−1 and 1149.40 cm−1), one is mostly sensitive to the temper-ature profile (746.30 cm−1), another one is sensitive to the ozone profile (1030.9 cm−1)) andthe remaining two (1369.90 cm−1 1612.90 cm−1) are mostly sensitive to water vapour. If weconsider that the σ SEVIRI forward model has N = 60, we have a total of 7 channels againsta total of 180 atmospheric parameters for temperature, water vapour and ozone. The problemis therefore heavily under-determined.

We can reduce the number of atmospheric parameters with some suitable assumptions.Towards this objective, we assume that the unknown profiles are identical to the initial or firstguess state vector, apart from an unknown scale factor,

T = (1 + fT )T0; Q = (1 + fQ)Q0; O = (1 + fO)O0 (96)

With this assumption the linear form 94 can be reduced to

Ri = R0i(σ) +∂Ri∂Ts

∆Ts +∂Ri∂εi

∆εi + aifT + bifQ + cifO (97)

with

ai =N∑j=1

∂Ri∂Tj

T0j ; bi =N∑j=1

∂Ri∂Qj

Q0j ; ci =N∑j=1

∂Ri∂Oj

O0j (98)

Considering the seven SEVIRI channels, the inverse problem is then transformed to onewith 11 unknowns: one for the skin temperature, seven for emissivity and 3 for the atmospheric


parameters, namely, fT , fQ, fO. Based on the definition of Eq. 96 these last three parameterscorrespond to an adjustment of the bulk temperature of the atmosphere, the columnar amountof H2O and O3, respectively. Moreover, they can be interpreted directly in terms of fractionalchange of bulk temperature, columnar amount of water vapor and ozone.

This scheme is not the most suitable for high spectral resolution infrared observations, asthose EUMETSAT plans to measure, e.g. with the MTG-IRS instrument. For MTG-IRS thenumber of channels would be as large as to directly invert Eq. 94 for the atmospheric profiles,(T,Q,O).

To implement the Kalman filter with the data or measurement equation given by Eq. 97we need a model equation for the additional parameters, fT , fQ, fO and suitable informationfor the first guess (T0,Q0,O0) profiles. These profiles have to be available at each time step, t.One might argue that the profile at step t+ 1 is that moved forward from the previous step t.However, this is a too much crude approximation because, even on a time scale of one day, theatmospheric profiles can change shape, whereas our methodology is not sensitive to the shapeof the profile.

A better approach is to use the ECMWF analysis spatially and temporally interpolatedto the given SEVIRI field of view. In doing so we have that the shape of the profiles evolvewith time according to the ECMWF analysis, and what we adjust with the observations arethe bulk quantities of the profiles, through the scale factors, fT , fQ, fO. This is the strategy wehave used to implement the Kalman filter. For the state equation of the scaling factor we havejust assumed a simple persistence equation.

The settings of the present implementation are summarized in Tab. 17.

Table 17: Summary of the settings for the 2D-Kalman Filter scheme, which includes atmo-spheric parameters.


Emissivity model equation PersistenceSkin Temp. model equation PersistenceScale factors (fT , fQ, fO) Persistence

Emissivity true values unknownEmissivity initialization (at time=1) IREMIS data base

Emissivity initial background IREMIS data baseEmissivity stochastic covariance as line above scaled down with f = 10




Scale factors true values unknownScale factors (fT , fQ, fO) initial values (at time=1) (0,0,0)

Scale factors (fT , fQ, fO) initial background diagonal 0.22, 0.22, 0.22

Scale factors (fT , fQ, fO) stochastic variance initial background scaled down by 10Observational Covariance matrix diagonal, from SEVIRI radiometric noise (see Fig. 17)

Atmospheric profiles assumed known in shape and obtainedfrom time-space colocated ECMWF analysis

Convergence criterion cost function

Results are exemplified for one day of SEVIRI observations over the Sahara desert site.Figure 81 shows the retrieval for the scaling factor for one SEVIRI pixel. We see that forozone the scaling factor is nearly zero, we have no significant adjustment for ozone, whereasfor water vapour the adjustment reaches the level of nearly +30% at midday. For temperaturethe scaling factor is below 1-1.5%, which means a change in the bulk temperature of 3-4.5 K.Once again, the larger adjustment is seen at midday. This is consistent with the results forskin temperature and with IASI retrieval over the Sahara desert [35]. The IASI retrieval shows


that both temperature and water vapour profiles derived from the ECMWF analysis are largelyunderestimated (in the lower troposphere) at midday.

Figure 81: Retrieval of the scaling factors for the atmospheric profiles, [T,W,Q]. The caseshown in figure applies to the Sahara desert. The results are shown for one single SEVIRIpixels.

IASI results [35] also show that the ECMWF skin temperature has a large downward biasin daytime, which is directly confirmed from the results shown in Fig. 82, which compares theretrieval of the present 2-D scheme and the 1-D filter discussed in section 10.1 to ECMWFanalysis. The skin temperature has been spatially averaged over the SEVIRI box shown in Fig.47. It is seen that differences are confined below 0.5◦C at any time.

For the case emissivity (see 83 we see some more consistent variations. However, also inthis case they normally are within error bars. of particular interest is the emissivity at 1149.40cm−1 (the channel within the reststhralen band of quartz). The decrease of emissivity aroundmidday is seen also in the case of the 2-D scheme, for which we have atmospheric parameterswithin the state vector. This means that the pattern day-night seen in the emissivity is not anartifact of unresolved atmospheric variability. Also interesting is the fact that the same patternis shown by the ozone channel at 1030.90 cm−1, which is affected by reststhralen absorption,as well.

For sea surface, the extension to 2-D has been obtained by properly modifying the softwarepackage we have developed to implement the 1-D scheme shown in section 10.2. As for landsurface, the system has been developed considering a persistence model for the three scalingfactor.

Figure 84 shows the retrieval of the scaling factors for one day (31 July 2010) and oneSEVIRI pixel belonging to the SEVIRI pixel pattern shown in Fig. 47. The retrieval for thescaling factor lead us to conclude that the ECMWF analysis for sea surface provides a good


Figure 82: Retrieval of the skin temperature with the 2-D Kalman filter and comparison withthe equivalent 1-D scheme. Results apply to the Sahara desert site. Data have been spatiallyaveraged over the SEVIRI box shown in Fig. 47.

model for the atmosphere. In fact, the scaling factor, fT is well below 0.2%, which implies verylittle change in the bulk atmosphere has rendered by the ECMWF analysis. For water vapourwe can have variation up to 2.0%. Occasionally, we have observed scaling factor for watervapour as high as 12-15%. However, this rare events and normally occur when convergence isnot reached.

Figure 85 compares the 2-D filter vs that 1-D for the same location and day. The parametercompared is the skin temperature. It is possible to see that there are differences between thetwo implementations. However, on average the difference is well below 0.5◦C.

The emissivity retrieval does not show any substantial difference with respect to the re-trieval obtained with the 1-D scheme and is here not shown, since it does not add informationto what already presented and discussed in section 10.2. Much more interesting is the spectralresidual at the seven SEVIRI channels, which help us to identify possible biases within theforward model. The spectral residual for the 3 window channels is shown in Fig. 86, whereasfor the remaining four channels it is shown in 87. The results shown in these two figures havebeen averaged over 179 samples, therefore they are highly statistically significant.

We see that the the window channels are fitted quite well and the bias is not significant.Also, the spectral residual is well confined within the radiometric noise bars.

For the non-window channels we have a nice fit for the two channels in the water vapourabsorption band. For the ozone channel at 1030.90 cm−1 and the CO2 channel at 746.27 cm−1,the results are mixed, which in part reflects the fact that for these two channels, we have notenough information to resolve atmospheric variability for temperature and ozone. However,the degrees of misfit is quite close to the magnitude of the radiometric noise. The fact thatwe can have a good fit for the window channels, despite the relative misfit for CO2 and O3,shows that the window channels are nearly unsensitive to the atmospheric component, whichalso gives more confidence to the 1-D scheme for emissivity and skin temperature, where theatmospheric component plays the role of an interfering factor.


Figure 83: Emissivity retrieval with the 2-D Kalman filter (left) and comparison with the resultsobtained with the 1-D scheme. Results apply to the SEVIRI box shown in Fig. 47 and havebeen spatially averaged

Figure 84: Retrieval of the scaling factors for the atmospheric profiles, [T,W,O]. The caseshown in figure applies to the Mediterranean sea. The results are shown for one single SEVIRIpixels.


Figure 85: Retrieval analysis for skin temperature. a) 2-D Kalman filter; b) 1-D Kalman filter;c) the retrieval has been spatially averaged over the grid box of size 0.05× 0.05 degrees shownin Fig. 47.


Figure 86: Retrieval exercise for the Mediterranean target area. The figure shows the spectralresidual for the three window channels. Results have been spatially averaged over the grid boxof size 0.05× 0.05 degrees shown in Fig. 47.


Figure 87: Retrieval exercise for the Mediterranean target area. The figure shows the spectralresidual for the four non-window channels. Results have been spatially averaged over the gridbox of size 0.05× 0.05 degrees shown in Fig. 47.

12 KALMAN FILTER ([TS , ε] CASE) WITH LOCALIZED SPATIAL CONSTRAINTS 124

12 Kalman filter ([Ts, ε] case) with localized spatial con-straints

In this section we discuss the extension of the [Ts, ε] case, developed for the three SEVIRIwindow channels, to the 3-dimensions by properly extending the state vector to nearby pixels.Considering that the basic [Ts, ε] scheme is 1-D because of the time dimension, this sectionprovides a 3-D example of the Kalman filter: one time dimension and two horizontal-spatialdimensions.

The extension to the horizontal-space dimensions begins with by considering horizontalspatial boxes of n × n. If we consider that within the basic [Ts, ε] algorithm for the threeSEVIRI atmospheric window channels, the state vector has a size of four, we have that forthe 3-D implementation we have to consider state vectors of size n × n × 4. In the presentimplementation we will consider n = 3, although the scheme can be generalized to any n.

Figure 88 provides examples of 3 × 3 SEVIRI pixels for the Sahara desert test site. Thestate vector for each 3 cluster is built up by considering a simple vertical concatenation of thestate vectors corresponding to each SEVIRI pixel within the cluster. The position within each

single cluster can be individuated by a couple of coordinates (i, j), i, j = 1, . . . , 3. Let be ε(k)ij

the emissivity in channel k for the pixel at position (i, j) and let Ts;ij the skin temperature atposition (i, j). We conventionally assume k = 1 for the channel at 833.33 cm−1, k = 2 channelat 925.90 cm−1 and k = 3 channel at 1149.40 cm−1.

Figure 88: SEVIRI pixels pattern for the Sahara desert site showing 3 × 3 boxes used toimplement the 3-D Kalman filter.

We pile up emissivity and skin temperature in two different vectors. These have size 27and 9, respectively.

ε =

ε111ε211ε311ε112ε212ε312. . .ε133ε233ε333

(99)


Ts =

Ts;11Ts;12Ts;13Ts;21Ts;22Ts;23Ts;31Ts;32Ts;33

(100)

and we define the state vector, v (of size 36) according to form

v =

(ε

Ts

)(101)

In this way we can compute covariance matrices separately for the spatial emissivity vec-tor, ε and the spatial skin temperature vector, Ts. In fact, the space constraint is introducedthrough the covariance matrices, E(εεT ), E(TsT

Ts ), where the superscript T indicates trans-

pose and E(· · ·) stands for expectation values.

Using the IREMIS data base the spatial covariance matrix for emissivity has been computedfor the test site of the Sahara desert shown in Fig. 47. This space-covariance matrix is shownin Fig. 89.

Figure 89: Emissivity space-covariance matrix corresponding to 3×3 spatial cluster of SEVIRIpixels. The covariance matrix applies to the Sahara desert test site.

Once we have a proper space covariance matrix for emissivity and skin temperature, theimplementation of the Kalman filter is straightforward, provided we start form the 1-D (scalartime) implementation. We only need to properly increase the size of the data and state vectorsand corresponding matrices.

For illustrative purposes, we have developed a 3-D scheme in which the covariance matrixof the state vector has been obtained by the IREMIS data base for the emissivity elementsand is a simple diagonal matrix for the skin temperature. This is intended only for illustrativepurposes, we could use a more elaborate form for the Ts-covariance, however also in this formthe scheme we have developed allows us to understand if the concept works. And in fact theconcept does work as it is possible to see from Fig. 90, which shows the skin temperatureretrieval for ten consecutive days of July. Since we work with 3 × 3 cluster of SEVIRI pixels,at each time t, we simultaneously get the solution for the 9 pixel within the cluster. The mostserious shortcoming we can have with this approach is the presence of clouds. If just one pixelis cloudy within the cluster, it can have effect on the retrieval for all the remaining clear skypixels. This effect is seen in Fig. 90 at the beginning of the second day. Figure Fig. 90 also


provides a scatter plot of the kalman 3D Ts vs ECMWF analysis. This comparison confirmsthat the ECMWF analysis fails at midday. Furthermore, Fig. Fig. 90 also provides the resultsof the emissivity retrieval. This is consistent with what obtained with the 1-D filter.

Figure 90: Top left, retrieval results for the skin temperature, the results apply to one singlespatial cluster of 3×3 SEVIRI pixel. Top right, scatter plot of the retrieved Ts vs the ECMWFanalysis. the data have been averaged over the 3×3 cluster. Bottom left, emissivity retrieval forchannel at 1149.40 cm−1 corresponding to the first observation in the 3× 3 cluster and relatederror interval. Bottom right, emissivity retrieval averaged over 3× 3 cluster. The results havebeen obtained with a 3-D implementation of the Kalman filter and apply to Sahara desert site(e.g. see Fig. 47.

13 MONTHLY TS − ε MAPS OVER THE WHOLE TARGET AREA 127

13 Monthly Ts − ε maps over the whole target area

The Kalman filter and the Optimal Estimation schemes have been applied to the a whole monthof SEVIRI observations for the target area shown in Fig. 7.

Retrievals have been computed at the SEVIRI time (15 min) and spatial resolution andthen averaged to form daily and monthly maps. For the sake of brevity here we will show theresults for the monthly maps.

To limit computational burden we have used the temporal 1-D implementation of bothschemes, therefore the SEVIRI pixels have been inverted independent each of other. Theemissivity has been retrieved at the three atmospheric window channels: 833.33 cm−1 (12 µm),925.93 cm−1 (10.8 µm) and 1149.40 cm−1 (8.7 µm).

The Optimal Estimation scheme we have used to perform the retrievals consider a timeslot width of one hour. The settings are equivalent to those shown in Table 11. Of course,each single SEVIRI pixel has been characterized with the appropriate background vector andcovariance according to the IREMIS data base.

Once again we stress that the IREMIS year 2010 has not been used to build up the emissivitybackground, in order to use it for comparison with the retrieval products.

The Kalman filter we have used to yield daily and monthly maps is that introduced insection 9.1 with the settings shown in Tab. 12. Observations have been processed on dailybasis, that is the filter has been re-initialized each 24 hours.

Only clear sky observations have been processed. Clear sky has been qualified accordingto the SEVIRI operational cloud mask..

Figure 83 compares the Kalman filter monthly map to that obtained with the Optimalestimation. It is seen that the maps compare fairly well each other, although differences exist,which are mostly concentrated in the hottest part (Sahara desert) of the target area.

Figure 84 compares the two maps corresponding to the emissivity at 1149.4 cm−1, whichis located in the strong reststhralen absorption band of quartz. It is possible to see that bothmaps exhibits a spatial variability, which closely follows the distribution of the richer quartzsand across the desert. Figure 84 also provides the difference with the corresponding IREMISmap for July 2010. On overall, we see that the Optimal estimation maps stay closer to thatof IREMIS. This is not necessarily a positive aspect. It also reflects the fact that the Optimalestimation stays closer to the background. On this aspect the Kalman filter map seems todevelop new and independent features of the IREMIS background, which is not necessarily anegative aspect.

However, in general both methodologies (Kalman filter and Optimal estimation) agree fairlywell with the IREMIS data base map.

The results for the other two channels, 10.8 µm and 12 µm are summarized in Fig. 85 and86, respectively. Both these two figures confirm that the Kalman methodology develops moreindependent features with respect to the IREMIS data base than the Optimal Estimation.


Figure 91: Top left: skin temperature monthly map obtained with the Kalman filter method-ology; Top right: Optimal Estimation retrieval; c) map of the difference; d) histogram of thedifference. Blank areas correspond to cloudy regions.


Figure 92: Top left: 8.7 µ m emissivity monthly map obtained with the Kalman filter method-ology; Top right: Optimal Estimation retrieval; c) difference Kalman-IREMIS; d) differenceKalman-IREMIS. Blank areas correspond to cloudy regions.


Figure 93: As fig. 92, but for the channel at 10.8µ m.


Figure 94: As fig. 92, but for the channel at 12.0 µm.

14 CONCLUDING REMARKS 132

14 Concluding remarks

We have reviewed variational and sequential optimal estimation schemes with the objectiveto apply them to SEVIRI infrared observations. Specific algorithms have been developed andimplemented for the two basic parameters, skin temperature and emissivity. The algorithmshave been extended to include spatial constraints, both in the vertical and horizontal.

Extensive retrieval exercises, in simulation and with real observations, have demonstratedthat the concept of applying temporal and spatial constraints to SEVIRI observations doeswork.

On overall we can say that the Kalman filter couples Optimal Estimation with the possi-bility to sequentially process the observations, which reduces the dimensionality of the retrievalsystem. Another advantage of the Kalman filter is its capability to deal with unequally spacedtimes, that is with time sampling interval, ∆t which has not to be necessarily a constant. Thisallows us to jump over cloudy time periods and process only clear sky observations, without theneed to re-initialize the filter. However, in this case information coming from time-continuityis partly lost.

Concerning the performance of the two methodologies, we have that, as far as, the retrievalof skin temperature is concerned, OE and KF are almost equivalent, although a slight bias(of order of 1 K) between the two still persist even on monthly averages. OE and KF fairlycompare with ECMWF analysis for sea surface. ECMWF analysis for sea surface shows a slightwarm bias, which by the way is below 0.5 K. For land surface, OE and KF agree fairly goodwith ECMWF for nighttime observations, but at midday ECMWF shows a cold bias, whichcan achieve 10 K and more. This is the case for Spain and Sahara desert.

For emissivity, we have that the comparison with the IREMIS data base for the same dateand location is fairly good. OE stays closer to IREMIS, whereas KF seems to add independentinformation of that contained in the IREMIS data base.

Coming to the problem of dimensionality of the two methodologies, KF is superior overOE because of the aforementioned KF capability of sequential processing of the data. However,for both methodologies spatial constraints can be consistently introduced through a suitabledefinition of the state vector and related background. For an instrument such as SEVIRI, whichhas only eight spectral channels, dimensionality is strongly affected from the way we modifythe state vector to accommodate spatial information. As an example, if we introduce localizedhorizontal-spatial constraints by considering SEVIRI pixels clusters of size n × n, both stateand data vectors will grow by n2 and the related covariance matrices by n4.

Clouds. To avoid cloudiness we need both high repetition rate (good temporal resolution)and a very good horizontal spatial resolution. It can be shown that the density of clear skysoundings improve by considering smaller field of views [13]. Considering the need of clearsky, we can conclude that the most suitable and straightforward form of the Kalman filterfor application to the retrieval of satellite infrared observation is that 2-D, (1-D time)×(1-Dvertical). Once we have solved the retrieval for L1 observations, the problem of extending tothe full disk the corresponding clear-sky L2 products could be solved by 2-D spatial OptimalInterpolation or Kriging. State-of-art Bayesian methodology applied to satellite data tends toaddress the problem this way. That is, first one applies a 2-D (time,vertical) algorithm to clearsky L1 products, and in sequence a 2-D horizontal spatial Kriging to L2 parameters in orderto extend information to the full disk, including cloudy pixels.

Although case studies developed in this study are necessarily limited to surface parame-ters, because of limited information content of SEVIRI infrared channels as far as atmosphericparameters are concerned, the retrieval methodology has been described in its most generalframework and can, therefore, provide guidelines to extend the algorithms to future instru-ments, such as MTG-IRS. This instrument will have some 2000 spectral channels. Therefore,the data space, rather than the parameter space, will be driving the design of a L2 processor.


Even with M = 2000 a 2-D Kalman filter (time×vertical) is feasible in terms of computationalcosts. In this respect, if we consider that the observational covariance matrix for MTG-IRS isexpected to be nearly diagonal, we can use the sequential updating approach shown in section4.4. With this approach we would need to store only M = 2000 diagonal elements and use anumerical algorithm which does not involve any matrix inversion. From a computational pointof view, the dimensionality of the problem would be driven by the analysis covariance matrix,Sa, which at this point could include also suitable spatial constraints, which could make themethodology, 4-D. However, in case we consider to use, e.g., the ECMWF analysis or forecastdirectly as the state equation of the Kalman filter, we could avoid to include spatial constraintsand rely on the spatial homogeneity expected in the ECMWF analysis or forecast.

One possible problem against this feasible approach for MTG-IRS is the need of L1-datacompression for dissemination and distribution. This will have the positive effect of seeminglyreducing the data space dimensionality of a factor about 8 to 10. There are plans to use,e.g., MPC = 300 Principal Components (PC) scores. Because of inevitable differences in thePC processor installed at EUMETSAT and that used by a given user, it is likely that thecompression will lead to a projected observational diagonal matrix which is no longer diagonal.The effect is that the sequential updating approach described in section 4.4 cannot be applied,and the dimensionality of the inverse problem jumps to MPC ×MPC . Contrary to what isnormally believed PC compression will make the dimensionality of the retrieval problem largerthan that for the uncompressed case. In fact, the factor is MPC ×MPC ×M−1, which if weconsider MPC = 300 and M = 2000 gives 45. The problem becomes even more acute if weconsider reconstructed radiances. In fact, for this case the observational covariance matrix ishighly non-diagonal.

The way to exit form this vicious circle has been for long shown by the authors. MTG-IRS is a Fourier transform spectrometer, if we compress the data by considering interforogrampackets, the covariance matrix will remain diagonal, despite the end-user. With MTG-IRS, theFourier transform is the only way to ensure that the compression factor will translate exactlyto the dimensionality reduction of the retrieval problem. If we compress of a factor, say, 10,the retrieval problem will benefit of a factor dimensionality reduction equal to 10.

A last point that it is worth to mention has to do with surface emissivity. In principle, thedimensionality of this parameter is the same as that of the radiance vector. Thus, this makesthe dimensionality of the state vector to grow up to M + N . However, emissivity itself canbe represented in terms of suitable orthogonal transforms, in which only few coefficients needto be retained. These approaches have proved to be very powerful to process IASI data (e.g.,[2, 35, 56]) and can be easily extended or adapted to MTG-IRS.

Recommendations. Throughout this study we have been dealing with two general strate-gies for the retrieval of surface and atmospheric parameters: Optimal Estimation and KalmanFilter. It has been shown that optimal estimation is just a particular application of Kalmanfilter. In case we want to take advantage of time continuity and constraint, the Kalman fil-ter has to be preferred to Optimal Estimation. We can include within Optimal Estimationtime continuity, however this happens at expense of the dimensionality of the retrieval prob-lem. Conversely, the time-sequential attribute of Kalman filter makes it possible to processtime sequences of data points in a way that does not affect the dimensionality of the retrievalproblem.

The Kalman filter also needs a state equation for the parameter space or state vector. Thisstate equation governs the time evolution of the state vector and enables the sequential strategyof the methodology. The state equation can be a physical (e.g. NWP dynamical equations)or a statistical model (e.g., autoregressive process). However, the Kalman filter methodologycan also work in case the state equation does not provide the correct model for the parametersvector. In fact, one of the strengths of the Kalman approach is that it can accommodate theknowledge that the state equation or model for the process of interest is inherently inadequateto describe the real-world phenomenon. This is, e.g., the case when we consider a persistence


model for the surface temperature under clear sky conditions. In this case we know that themodel is inherently wrong, because in clear sky Ts is governed by the daily cycle, which isbetter represented by a deterministic second-order difference equation. Our confidence aboutthe state equation is inserted through a stochastic variance term, which can trade-off betweendata and model.

For this reason we recommend the use of a persistence model either for land or sea surface.We think that after a training/validation phase, the Kalman approach for the retrieval of Ts−εfrom SEVIRI window channels could become operational.

For the problem of deriving a suitable background for emissivity for land surface we defi-nitely recommend to use the UW/IREMIS data base.

Open issues. Our analysis has been largely illustrative of the many capabilities allowed bythe use of tools and concepts of data assimilation for the processing of time-sequential satellitedata. These examples have outlined possible mature applications to SEVIRI radiances for theretrieval of surface parameters. However as said before, we think that the Ts − ε retrievalfrom SEVIRI still deserves an intense phase of traning/validation for the correct settings ofthe stochastic variance term. The validation data set also should include situations of abruptchanges in surface parameters and/or radiances (e.g. rainfall, wether events, cloudiness) tocheck how the Kalman filter is, eventually, drifted away from its stationary state and to assessthe magnitude of the relaxation time needed to attain a new, possibly, stationary evolution.This validation should concern both land and ocean and should be performed with truth datahaving a spatial coverage and extend wide/large enough to check the capability of SEVIRI toresolve spatial structures.

In perspective for MTG-IRS or geostationary high spectral resolution infrared instruments,one second important issue is how to include atmospheric parameters within the retrieved statevector. The way we have described in section 11 is mostly intended for SEVIRI, which hasa poor spectral resolution. The methodology described in section 11 is not recommended forhigh spectral resolution infrared observations. In this case, a better strategy would be to use apersistence model at each layer and introduce inter-layer correlation through the definition of aproper background as it is done, e.g., with IASI in the context of one-dimensional data assimi-lation. A development and related feasibility analysis of a 2-D Kalman filter (time and verticalcoordinate) could be carried out in simulation using the forecast or analysis from a limited-area NWP model, such as COSMO. The use of a high horizontal spatial resolution NWPmodel would make it possible to yield simulations, which are consistent with the horizontalspatial sampling expected for MTG-IRS. High temporal resolution of simulated atmosphericfields could be obtained through linear interpolation or extrapolation with a suitable stochasticmodel. The stochastic model could also be used to simulate inter-layer correlations. Further-more, the Kalman filter should be designed and developed for suitable transformed data andparameters spaces in order to alleviate the curse of dimensionality.

Finally, we think that the problem of developing a full 4-D Kalman filter has to be addressedwith a dedicated study. The present study, while having touched a large variety of schemeswithin the large context of Bayesian data assimilation, is too much specific to SEVIRI whendealing with applications and too much general in its mathematical formulation when dealingwith the atmospheric component. Atmospheric parameters have to be addressed in the contextof a specific (existing or designed) instrument in order to be less generic in drawing conclusionsand defining possible strategies. Again, we think that a good exercise could be performed havingspecifically in mind an instrument such as MTG-IRS, whose observations could be simulated,as said before, starting from the analysis or forecast of limited-area NWP models. The 4-DKalman filter feasibility should be addressed along the lines presented in section 5, whereasthe specific processing algorithm should follow the sequential approach explained in section4.4. We do not think that the inclusion of spatial constraints should follow the methodologyexplained in section 3. This approach would lead to static backgrounds, which could be notsuited to properly address and exploit time continuity of atmospheric fields. We think that

REFERENCES 135

spatial constraints should be dynamically obtained by, e.g., ECMWF analysis, and conveyedin a methodological setting, which should mimic as much as possible the Ensemble Kalmanfilter approach. We say mimic because the approach would be driven from the data, andnot from a NWP dynamical model, which in fact would be substituted with a simplistic, e.g.,persistence model. Spatial dynamical information would be introduced by estimating first andsecond order statistics from an ensemble of NWP profiles. This strategy would make up forthe lack of physics, which is inherent to, e.g., a persistence or simplistic evolutionary modelfor the state vector. To these simplistic models we would leave only the mathematical roleof conveying within the scheme the property of sequential processing of the data, whereas thephysical, spatial information would be conveyed in the scheme by the ensemble of NWP profiles.This ensemble should be not static, but continuously updated based on the analysis or forecastof a suitable NWP model.

References

[1] Amato, U., Masiello, G., Serio, C., and Viggiano, M., 2002, The σ-IASI code for thecalculation of infrared atmospheric radiance and its derivatives, Environmental Modelling& Software, 17/7, 651–667.

[2] Borbas E.E. and B.C. Ruston, The RTTOV UWiremis IR land surface emissivity module,NWP SAF, EUMETSAT, 2010.

[3] Box George E. P. and Gwilym M. Jenkins, Times Series Analysis: forecasting and control,Holden-Day, 1976.

[4] Cari G. Kaufman, Mark J. Schervish, Douglas W. Nychka. Journal of theAmerican Statistical Association. December 1, 2008, 103(484): 1545-1555.doi:10.1198/016214508000000959.

[5] Carissimo A, De Feis I, Serio C.(2005). The physical retrieval methodology forIASI: the δ-IASI code. Environ. Modelling & Software, vol. 20/9; p. 1111-1126, doi:10.1016/j.envsoft.2004.07.003

[6] Clough, S. A., Shephard, M. W., Mlawer, E. J, Delamere, J. S., Iacono, M. J., Cady-Pereira, K., Boukabara, S., and Brown, P. D., 2005 Atmospheric radiative transfer mod-eling: a summary of the AER codes, Journal of Quantitative Spectroscopy & RadiativeTransfer, 91, 233–244.

[7] Courtier P., Variational methods, J. Meteorol. Soc. Jpn 75 (1997) 211178.

[8] Cressie N., Statistics for Spatial Data, Wiley, New York, 1993.

[9] Cressie N. and Wikle K., Statistics for Spatio-Temporal Data, Wiley, New York, 2011.

[10] Cressie, N, Shi, T., and Kang, E.L., Fixed Rank Filtering for Spatio-Temporal Data,Journal of Computational and Graphical Statistics, Volume 19, Number 3, Pages 724-745,2010

[11] Cressie N, Johannesson G., Fixed rank kriging for very large spatial data sets, J. R. Statist.Soc. B 70, Part 1, pp. 209-226 (2008).

[12] Cuomo V., Serio, C., Crisciani, F. and Ferraro, A. (1994), Discriminating randomness fromchaos with application to a weather time series. Tellus A, 46: 299-313. doi: 10.1034/j.1600-0870.1994.t01-2-00005.x

[13] Cuomo V., Pietrapertosa C., Serio C., Tramutoli V. (1999). Assessing the impact of cloudmorphology on infrared sounder scan geometry. Int. J. Rem. Sens., vol. 20; p. 169-181,ISSN: 0143-1161

REFERENCES 136

[14] Dee D.P., Da Silva A.M., Maximum-likelihood Estimation and Observation Error Covari-ance Parameters. Part I: Methodology, Month.Weath.Review, 127, 1822-1834, 1999,

[15] Dempster A. P., Laird N. M., and Rubin D. B. Maximum likelihood from incompletedata via the em algorithm. Journal of the Royal Statistical Society: Series B, 39(1):117,November 1977.

[16] Di Girolamo, P., Grieco G., Masiello G., Serio C. (P.I.), Summa D., Assessing the 3-Dcorrelation structure of atmospheric humidity field and evaluation of the MTG InfraredSounding Mission to resolve it (MTG3D): Final Report. (EUMCO074600000398SAT) De-cember 2007.

[17] Elsasser, W.M., Heat Transfer by Infrared Radiation in the Atmosphere, Harvard Meteor.Studies, 6, Harvard University Press, Cambridge, Mass., USA, 1942.

[18] Evensen, G. Sequential data assimilation with a nonlinear quasi- geostrophic model usingMonte Carlo methods to do forecast error statistics. Journal Geophysical Research 99(C5),10143-1016 (1994)

[19] Gasparo, G. and Cohn, S.E., Construction of correlation functions in two and three di-mensions, Quarterly Journal of Royal Meteorological Society, 125, 723-757, 1999.

[20] Gentemann, C. L., C. J. Donlon, A. Stuart-Menteth, and F. J. Wentz, Diurnal signalsin satellite sea surface temperature measurements, Geophys. Res. Lett. , 30(3), 1140,doi:10.1029/2002GL016291, 2003.

[21] Goody R.M., Atmospheric Radiation: Theoretical Basis Oxford University Press, UK,1996

[22] Gottsche F. and F. Olesen, A Simple Physically based Model of Diurnal Cycles of LandSurface Temperature, Proceedings 2009 EUMETSAT Meteorological Satellite ConferenceBath, United Kingdom, 21 - 25 September 2009, EUM P.55

[23] Govaerts Y.M., Wagner S., Lattanzio A., Watss P., Joint retrieval of surface reflectanceand aerosol optical depth from MSG/SEVIRI observations with an optimal estimationapproach: I. Theory, JGR, 115, D02203, 2010. doi:10.1029/2009JD011779.2010

[24] Kalman, R.E. A new approach to linear filtering and prediction problems. Journal of BasicEngineering 82 (1): 35-45 (1960).

[25] Kalman, R. E., R. S. Bucy. New Results in Linear Filtering and Prediction Theory. Trans-actions of the ASME - Journal of Basic Engineering 83, 95-107 (1961).

[26] Kang E.L, Cressie N., and Shi T., Using temporal variability to improve spatial mappingwith application to satellite data, The Canadian Journal of Statistics Vol. 38, No. 2, 271-289 (2010).

[27] Katzfuss, M. and Cressie, N. Maximun likelihood estimation of covariance parameters inthe spatial-random effects model, Proceedings of the Joint Statistical Meetings, Alexandria,VA, Americal Statistical Association, pp 3378-3390, 2009

[28] La Dimet F -X, Talagrand O., Variational algorithms for analysis and assimilation ofmeteorological observations: theoretical aspects. Tellus A, 38(2), 97-110, 1968.

[29] Lewis J. M., Derber J C, The use of adjoint equations to solve a variational adjustementproblem with advective constraints. Tellus A, 37(44), 309-322, 1985.

[30] Liou, K.N., Radiation and Cloud Processes in the Atmosphere, Oxford University Press,1992

REFERENCES 137

[31] Mardia K.V., C. Goodall, Redfern, E. J., Alonso, F.J., The Kriged Kalman filter, Sociedadde Estadistica e Investigacion Operativa Test (1998) Vol. 7, No.. 2, pp. 217-285

[32] Masiello, G. and Serio, C.,2003. An effective water vapor self-broadening scheme for look-up-table-based radiative transfer, in Remote Sensing of Clouds and the Atmosphere VII.Proceedings of the SPIE, Volume 4882. (K. P. Schaefer, O. Lado-Bordowsky, A. Comeron,and R. H. Picard, eds.), pp. 52–61.

[33] Masiello, G. and C. Serio (2004), Dimensionality-reduction approach to the ther-mal radiative transfer equation inverse problem, Geophys. Res. Lett., 31, L11105,doi:10.1029/2004GL019845.

[34] Masiello G, Serio C, Antonelli P. Inversion for atmospheric thermodynamical parameters ofIASI data in the principal components space. Q. J. R. Meteorol. Soc. DOI:10.1002/qj.909(2011).

[35] Masiello C., Serio C., Simultaneous physical retrieval of surface emissivity spectrum andatmospheric parameters from Infrared Atmospheric Sounder Interferometer spectral radi-ances. Appl. Opt. 2012 (submitted).

[36] K. Masuda, T. Takashima, and Y. Takayama, Emissivity of pure and sea waters for themodel sea surface in the infrared window regions,17Remote Sensing Environ. 24, 3131791988!.

[37] Menglin J. and R. E. Dickinson, Interpolation of surface radiative temperature measuredfrom polar orbiting satellites to a diurnal cycle. 1. Without clouds JOURNAL OF GEO-PHYSICAL RESEARCH, VOL. 104, NO. D2, PAGES 2105-2116, JANUARY 27, 1999

[38] Menglin J., Interpolation of surface radiative temperature measured from polar orbitingsatellites to a diurnal cycle 2. Cloudy-pixel treatment JOURNAL OF GEOPHYSICALRESEARCH, VOL. 105, NO. D3, PAGES 4061-4076, FEBRUARY 16, 2000

[39] Nino E., Serio C, Laser Doppler velocimetry analysis of transitional pipe flow. The Euro-pean Physical Journal B, Condensed Matter Physics, vol. 14, p. 191-200, (2000).

[40] Nychka, D. W. and Anderson, J.L., Data Assimilation, National Center for AtmosphericResearch, P.O. Box 3000, Boulder, CO 80307, Draft article for the Handbook on SpatialStatistics (2008).

[41] Rodgers C. D., Inverse methods for atmospheric sounding, Theory and Practice WorldScientific, Singapore, 2000.

[42] Serio, C., Discriminating low-dimensional chaos from randomness: A parametric timeseries modelling approach, Il Nuovo Cimento, 107B, 681-702, 1992.

[43] Serio, C., V. Cuomo, V. Crisciani, A. Ferraro, Discriminating randomness form chaos withapplictaion to a weather time series, Tellus 46A, 299-313, 1994.

[44] Serio C., (1994) Autoregressive Representation of Time Series as a Tool to Diagnose thePresence of Chaos, Europhys. Lett. 27 103, doi:10.1209/0295-5075/27/2/005

[45] Serio C, Masiello G., Grieco G., Carissimo A., Di Girolamo P., Summa D., Rodriuuez A.,Stuhlmann R., and Tjemekes S., Potential of the MTG-IRS mission to resolve small scalevariability of atmospheric humidity. In: Teruyuki Nakajima and Marcia Akemmi Yamasoe.Current Problems in Atmopsheric Radiation (IRS 2008). vol. 1100, p. 331-334, Melville(NY): American Institute of Phisics, (2009).

[46] Serio C., G. Masiello, M. Amoroso, S. Venafra, U. Amato, I. De Feis, Study on Space-TimeConstrained Parameter Estimation from Geostationary Data. Phase I Technical Report,EUMETSAT contract No. EUM/CO/11/4600000996/PDW, March 2012.

REFERENCES 138

[47] Serio C., G. Masiello, M. Amoroso, S. Venafra, U. Amato, I. De Feis, Study on Space-TimeConstrained Parameter Estimation from Geostationary Data. Second Progress Report,EUMETSAT contract No. EUM/CO/11/4600000996/PDW, August 2012.

[48] Siegel R., and J.R. Howell, Thermal Radiation Heat Transfer, McGraw-Hill, 1972.

[49] Sobrino J.A., Romaguera, M., Land Surface temperature retrieval from MSG1-SEVIRIdata Remote Sensing of Environment 92 (2004) 247174

[50] Talagrand O., Assimilation of observations, an introduction, J. Meteorol. Soc. Jpn 75(1997) 191179.

[51] Tarantola A., Inverse Problem Theory: Methods for Data Fitting and Model ParameterEstimation, Elsevier, New York, 1987

[52] Watts, P., M. Allen, and T. Nightingale, Sea surface emission and reflection for radiometricmeasurements made with the along-track scanning radiometer, J. Atmos. Ocean. Tech.,13, pp. 126-141, 1996.

[53] Wikle, C.K., Berliner L.M., A Bayesian Tutorial for data assimilation, Physica D, 230(1-2) pp. 1-26, 2007, doi:10.1016/j.physd.2006.09.017

[54] Wikle, C.K., Hooten M.B., A general science-based framework for dynamical spatio-temporal models, Test, 19 pp. 417-451, 2010, doi:10.1007/s11749-010-0209-z

[55] Wu X., Smith W.L., Emissivity of rough sea surface for 817 µm: modeling and verification,Appl. Opt. 36 pp. 2609-2619 (1997)

[56] Zhou, D.K., Larar, A.M., Liu, X., Smith, W. L., Strow, L. L., 879 Ping Yang, Schlussel,P.; Calbet, X. (2011). Global Land Surface 880 Emissivity Retrieved From Satellite Ul-traspectral IR Measurements, 881 IEEE Transactions on Geoscience and Remote Sensing,49, 1277-1290, 882 doi:10.1109/TGRS.2010.2051036.

Study on space-time constrained Parameter Estimation from Geostationary data Second Progress Report (DRAFT version)

Documents