Algorithm Theoretical Basis Document (ATBD) Merging_atbd_v2p1

7/26/2019 Algorithm Theoretical Basis Document (ATBD) Merging_atbd_v2p1

1/26

GlobAEROSOL Data User Element

Algorithm Theoretical Basis Document

for the creation of a

Merged Global Aerosol Product

DRAFT Version 2.1

6 February 2007

R. Siddans, C.A. Poulsen, B. J. Kerridge

Space Science and Technology Department,

CCLRC Rutherford Appleton Laboratory, U.K.

G. E. Thomas, S. M. Dean, E. Carboni, R. G. Grainger

Atmospheric, Oceanic, and Planetary Physics,

Clarendon Laboratory, University of Oxford, U.K.

CCLRC Rutherford Appleton Laboratory


2/26


3/26

Contents

1 Document version history. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 WP Document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

5 The Individual-Sensor Aerosol Products . . . . . . . . . . . . . . . . . . . . . . . 7

5.1 General characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

5.2 AATSR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

5.3 MERIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

5.4 SEVIRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

5.5 ATSR-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

6 Deliverable products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

7 Definition of the GlobAerosol grid . . . . . . . . . . . . . . . . . . . . . . . . . . 10

7.1 Gridding schemes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

7.2 Selected grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

8 The Merging algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

8.1 Temporal characteristics of the IGAPs and MGAP . . . . . . . . . . . . . 15

9 Merging co-located retrievals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159.1 General approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

9.2 Merging parameter maps . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

9.3 Treatment of bad data . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

9.4 Treatment ofAngstrm coefficient . . . . . . . . . . . . . . . . . . . . . 18

9.5 Treatment of aerosol speciation . . . . . . . . . . . . . . . . . . . . . . . 18

9.6 Determination of the covariances . . . . . . . . . . . . . . . . . . . . . . 18

9.7 Correcting sensor dependent biases . . . . . . . . . . . . . . . . . . . . . 19

10 Step-by-step implementation of the merging algorithm . . . . . . . . . . . . . . . 19

11 Outputs of the merging scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

12 Requirements on retrieval algorithms imposed by Merging scheme . . . . . . . . . 2413 MPM file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

13.1 Basic assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

13.2 File format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

13.3 MPM file type 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

13.4 MPM file type 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

13.5 MPM string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3


4/26


5/26

1. DOCUMENT VERSION HISTORY 5

1 Document version history

1.0: Original document issued 4 February 2005.

1.1: Issued 31 May 2005.

Added section13: Definition of MPM file format.

Added definition of sinusoidal grid coordinates, u, vto section7.1.

Defined grid selected for study (sinusoidalNeq = 4008).

2.0: 6 February 2007

Introduced IOAP as a delivered product and re-defined IGAP.

Updated section5.

Added figure3to illustrate nominal time of merged product.

Added section10,giving step-by-step description of the merging implementation. Sec-

tion includes quality control criteria determined during preliminary GAP validation.

Added figures4and5to illustrate merging process.

Criteria for selection of merged aerosol type changed / updated in section 9.5.

Operational settings defined during WP5200 included.

2.1: 29 Jun 2007

Minor correction to sinusoidal grid definition. Addition of algorithm to identify index

to compact array storage of gridded data.

2 Introduction

This document describes the algorithm for merging the global aerosol products (GAPs) derived

individually from the AATSR, MERIS and SEVIRI sensors into a single merged dataset. The

document describes the merging scheme and in addition details requirements which are imposed

on the individual sensor retrieval schemes to facilitate the merging process.


6/26

6

3 WP Document

This document covers the input required under work package 2330:

4 Overview

This document is structured in the following way:

A brief description of key aspects of ATSR-2, AATSR, MERIS and SEVIRI products is

provided. Only details which impact the merging algorithm are provided. For further infor-

mation see the relevant ATBD.


7/26

5. THE INDIVIDUAL-SENSOR AEROSOL PRODUCTS 7

The distinct product types which will be delivered are defined.

The spatial and temporal characteristics of the merged data-set are defined.

The approach to combine observations from each sensor is described.

Key outputs required from the merging scheme are listed.

A summary of the requirements imposed by the merging scheme on the individual sensor

retrieval schemes is given.

5 The Individual-Sensor Aerosol Products

5.1 General characteristics

Each IOAP is nominally produced at 10km spatial resolution. The inherent spatial resolu-tions of each sensor differ, but all are significantly finer than 10km. An approach to grid the

sensor data up to the 10km GlobAerosol resolution is defined in this document such that the

same grid is used for all delivered products.

The products consist of retrieved estimates of aerosol optical depth (AOD) at 0.55 and 0.865

m, the Angstrm coefficient .

For ATSR-2, AATSR and SEVIRI these products are produced using the Oxford-RAL Aerosol

and Clouds (ORAC) scheme, separately for each of 5 different aerosol models (namely con-

tinental, desert, maritime, urban and biomass). After these retrievals, the most-

likely aerosol type is identified using quality-control criteria. For MERIS no type informa-tion is available.

The AOD,, is defined by

() =

0e(z, ) dz=

0(s+a)(z, ) dz (1)

The total extinction coefficient, e, is defined as the sum of the extinction due to absorption,a, and scattering,s.zis altitude and is wavelength.

The Angstrm coefficient is defined as

d log [()]

d log [] . (2)

By assuming that varies linearly with in the visible, it is directly calculated from the tworetrieved optical depths:

A0.55,0.865 = log[0.865/0.55]

log[0.865/0.55]. (3)

5.2 AATSR

AATSR data is available from April 2002 to the present.

One product is produced per day corresponding to the time of each Envisat observation

(defined by the sun-synchronous orbit with descending node equator crossing of 10:30am

local solar time).

The AATSR swath width is 512i km centred on the sub-satellite point.


8/26

8

5.3 MERIS

MERIS data is available from April 2002 to the present.

As AATSR, One product is produced per day corresponding to time of each Envisat obser-

vation.

The MERIS swath width is 1150 km centred on the sub-satellite point.

5.4 SEVIRI

SEVIRI is on the Meteosat-8 platform (formerly named MSG-1) which is currently in geo-

stationary orbit above 0 latitude and longitude. Data is available every 15 minutes.

SEVIRI was launched on 28 August 2002. The mission was declared operational on 29

January 2004. It is assumed that data will be available to this project only after the latterdate.

The geographical coverage available is illustrated in figure1.

The SEVIRI products will be produced at 10:15 and 16:15 UT (univeral time = GreenwichMean Time, GMT). (Level 1 data is available every 15 minutes but only 2 time-slots per day

can be processed within available computational resources.)

Figure 1: The Earth disk viewed by SEVIRI. The line-of-sight zenith angle of SEVIRI from the

ground is shown by the colour scale.

5.5 ATSR-2

ATSR-2 data is available from April 1995 to the present. However (a) the pointing accu-

racy of the ERS-2 platform was degraded by gyro failure after January 2001 and (b) after

the ERS-2 tape recorders failed in June 2003 geographical coverage is restricted to regions

observed while ERS-2 has line-of-sight contact with a number of ground-stations. In any

event, comprehensive data-sets after January 2001 will are unlikely to be available to this

projects (routine production and cataloguing of UBT data at RAL was suspended after gyro

failure, pending the implementation of an adequate geolocation scheme).


9/26

6. DELIVERABLE PRODUCTS 9

The ATSR-2 swath width is 512km centred on the sub-satellite point, though due to down-

link bandwidth restrictions, various reduced coverage modes are often operated.

6 Deliverable productsThree types of product are identified. Each product contains data referenced to the defined GlobAerosol

grid:

1. Individual-sensor Orbit-based Aerosol Products (IOAP). These files contain all the output of

the individual sensor retrievals, with coverage corresponding to that of the original level 1

(radiance) data from the sensor. In particular:

For ORAC retrievals (ATSR-2, AATSR and SEVIRI) these files contain the results for

all of the five aerosol types, plus an indication of the best type, i.e. a flag indicating

which aerosol retrieval is considered most representative of the scene, based on qualitycontrol criteria which are defined following the preliminary validation of the individual

sensor aerosol products (see section 10, below). No information should be lost in

translating the original ORAC files (one of which is produced for each aerosol type

separately) into the amalgamated IOAP.

Since, in all cases, radiance data is gridded onto the selected GlobAerosol grid (see grid

selection below) before the retrievals are run, the IOAPs contain results which corre-

spond directly to GlobAerosol grid boxes. However, data is only stored for grid boxes

for which a retrieval was attempted for the given orbit (ATSR-2, AATSR, MERIS) or

time-slot (SEVIRI).

2. Individual-sensor Global Aerosol Products (IGAP). One IGAP is produced per day for each

sensor. The IGAPs contain records for all GlobAerosol grid boxes (filled with null-records

were no retrieval is available for a given box). For ORAC retrievals results are only reported

for the best aerosol type (and this type is identified).

In all cases the IGAPs contain estimates of aerosol optical depth (AOD) at 0.55 and 0.865

mand the Angstrm coefficient .

Note that for all instruments there may be more than one retrieval present in a given grid

square. For SEVIRI this is clear since 10:15 and 16:15 UT time slots are processed. For

ATSR-2, AATSR and MERIS two or more results will potentially be available where swaths

overlap towards high latitude. To resolve these ambiguities the IGAPs are produced usinga merging algorithm, defined below, which attempts to derive the aerosol property most

appropriate to a specific time which is defined for each grid box.

In addition, the reported aerosol properties in the IGAPs are also bias corrected based on

results from the preliminary validation of the IOAPs.

IGAPs therefore constitute bias-corrected daily maps of aerosol properties derived from each

sensor, at a specific time of day.

3. Merged Global Aerosol Products (MGAP). One MGAP is produce per day which combines

the results from all available sensors into a single map in the same format as the IGAPs,

valid for the same specific time of day. The same algorithm is defined below to implement

the merging steps necessary to generate both the IGAPs and MGAPs from the IOAPs. From

the processing point of view IGAPs are produced by running the merging scheme using

IOAPs from one instrument only.


10/26

10

MGAPs therefore constitute daily maps of what is considered to be the best representation of

the aerosol field for the specific time of day, based on information from all included sensors.

7 Definition of the GlobAerosol grid7.1 Gridding schemes

All individual sensors are required to be produced at 10km spatial resolution. For the individual

sensors, it would be possible to satisfy this requirement by defining a grid based on the inherent

sampling characteristics of the instrument (e.g. averaging radiances or retrievals from 10x10 1km

pixels in order of acquisition by the instrument). However such grids would differ between

the IOAPs. Spatial interpolation of the IOAPs onto a common grid defined for the IGAP/MGAP

is not desirable, since this would imply a degradation of spatial resolution. It is preferable for

the IGAP/MGAP grid to be adopted at the earliest possible processing stage. If the 10km IOAP

and IGAP/MGAP grids are not the same it would be necessary to store retrievals at finer spatialresolution in the IGAPs or tolerate some degradation (factor 2) in the MGAP resolution.

Many approaches are possible to define a grid which has a nominal spatial resolution of 10km.

It is fundamentally impossible to define a grid in which all boxes have exactly similar area and

shape. Even neglecting the ellipticity of the Earth, there is no regular tessellation of the sphere

having more faces than the icosahedron - i.e. 20 equilateral triangles.

It is assumed that the following approach is adopted to up-scale finer resolution radiances or

aerosol retrievals:

A set of grid-points are defined (see below) corresponding to the centre of implied grid-

boxes.

Each fine-resolution radiance / retrieval is associated with the grid-point nearest to it.

All radiances / retrievals associated with each grid-point are averaged to form the 10 km

resolution product.

The grid-boxes are therefore implicitly defined by polygons centred on each grid-point. Depending

on the gridding scheme, the polygons may be more or less irregular. Although not required directly

for production of the GAPs, the edges and vertices of each polygon (in latitude and longitude

coordinates) are required to validate the GAPs and by users to properly interpret the data.

This approach can be adopted whatever the method used to define the grid-points themselves.

Each of the schemes described below could therefore be implemented without particularly com-plicating the GAP retrieval / merging schemes.

Note that representation of the global field in terms of basis functions analogous to spherical

harmonics (i.e spectral grids) are not considered suitable here because of the spatially discrete and

sparse nature of the GAPs. (On a single day there will be data gaps due to clouds in all products,

between the Envisat swaths and outside the SEVIRI field-of-view. It is not considered sensible to

base GAPs on a gridding scheme which would implicitly interpolate across such data-gaps.)

Regular latitude-longitude grids

I.e. a grid is defined which is equally spaced in latitude and longitude. A grid spacing of

0.1degree would give 10x10km spatial resolution at the equator.

Advantages:

Simple to implement.


11/26

7. DEFINITION OF THE GLOBAEROSOL GRID 11

Easily understood by users.

Disadvantages:

The area of each grid box rapidly reduces towards higher latitudes (tending to 0 at the poles).

Gaussian grids

Gaussian grids are used by ECMWF to provide a compact representation of atmospheric and sur-

face data which is amenable to rapid conversion to and from the spectral grids used by their global

circulation model.

To quote from ECMWF documentation 1:

A Gaussian grid is a latitude/longitude grid. The spacing of the latitudes is not regular. How-

ever, the spacing of the lines of latitude is symmetrical about the Equator. Note that there is no

latitude at either Pole or at the Equator. A grid is usually referred to by its number N, which is

the number of lines of latitude between a Pole and the Equator.

The longitudes of the grid points are defined by giving the number of points along each line of

latitude. The first point is at longitude 0 and the points are equally spaced along the line of latitude.

In a regular Gaussian grid, the number of longitude points along a latitude is 4N. In a reduced

Gaussian grid, the number of longitude points along a latitude is specified. Latitudes may have

differing numbers of points but the grid is symmetrical about the Equator. A reduced Gaussian

grid may also be called a quasi-regular Gaussian grid.

In the reduced grids used by ECMWF, the number of points on each latitude row is chosen

so that the local east-west grid length remains approximately constant for all latitudes, with the

restriction that the number should be suitable for the Fast Fourier Transform used to interpolate

spectral fields to grid point fields, ie number= 2p 3q 5r. (Wherep,q,r are integers.)ECMWF currently employ grids up to number N512 which corresponds to a latitude / longitude

spacing of 0.08785 or 20km. For the purposes of this project a N1024 grid would be required to

give 10km spatial resolution. For illustration purposes, the N32 grid is defined in table1.

Advantages:

Grid boxes are much closer to having equal area in the reduced Gaussian grid scheme than a

simple latitude-longitude grid.

Easily understood by users already familiar with ECMWF data.

Code to handle data in this format appears to be available from ECMWF (tbc).

Meteorological data from ECMWF is available in a similar form (though at reduced resolu-

tion).

Disadvantages:

To enable proper interpretation of the data, the vertices and edges of each grid-box must be

defined and supplied to users.

Code to transform to a simple latitude-longitude grid would be needed by users.

The implied grid boxes are irregularly shaped.

1http://www.ecmwf.int/publications/manuals/libraries/interpolation/gaussianGridsFIS.html


12/26


13/26

7. DEFINITION OF THE GLOBAEROSOL GRID 13

Sinusoidal grid

Longitude, is transformed as a function of latitude,, as follows

= cos() (4)A regular grid in transformed longitude and (normal) latitude is then adopted. See figure

2.

The surface of the earth is covered by rectangular tiles in the transformed latitude and longitude

coordinates. These tiles can be identified by coordinates uandv.

u= Rgcos()+u0 (5)

v= Rg+v0 (6)

Where

Rg =Neq/360 (7)

u0= 180.Rg+ 0.5 (8)

v0 = 90.Rg+ 0.5 (9)

Neq is the number of tiles around the equator and should be an even integer. IfNeq is divisible by4 then the equator lies on the boundary between grid boxes. Integer values ofu,v correspond tothe centre of each tile in the sinusoidal grid.

Data can be stored easily in a 2-d array indexed by integer values of u and v which run from 1 to

Neqand 1 toNeq/2, respectively. However this storage is somewhat wasteful as many points in the

array do not correspond to valid latitude / longitude combinations. It may therefore be convenientto store data in a 1-d array containing only valid grid points. u andv coordinates can be convertedto an index to this array as follows:

i= u+Bv Neq/2 +Nv/2 (10)

Whereu andv are here assumed to be rounded to the nearest integer and are therefore consideredas indices to the column and row of a given grid box. Nv is the number of valid grid boxes in rowvandBv is the first index assigned to a grid box in rowv . Nv is equal tocos().Neq, rounded upto the nearest even integer. Bv is the sum ofNv over all rows from 1 to v 1, except for the firstrow whereB1= 0.

Advantages:

Grid boxes are approximately equal area.

A similar grid is used by the MODIS land albedo product (used by the AATSR retrieval

scheme).

Easily understood by users already familiar with MODIS data.

Disadvantages:

Code to transform to a simple latitude-longitude grid would be needed by users. (This could

possibly be based on existing MODIS code.)

Grid boxes are poorly defined at +/180 degrees longitude (which is of little consequencefor land products but of some importance for atmospheric products).


14/26

14

Icosahedral grids

Approximately regular grids can be defined as follows:

Inscribe a regular polygon with triangular faces into the sphere. Usually an icosahedron is

chosen (20 equilateral triangles, 12 vertices).

Subdivide each face into 4 triangles (projected onto the surface of the surface of the sphere)

by bisecting the great circle between each vertex.

Iterate the above step until the desired resolution is reached.

The process is illustrated in figure2.

Advantages:

Grid boxes are approximately equal area.

All grid boxes are approximately regular hexagons, with the exception of 12 pentagon-

shaped boxes centred on the original icosahedron vertices.

Disadvantages:

Code to transform to a simple latitude-longitude grid would be needed by users.

Grid spacings can only be chosen within a factor of 2 of the desired resolution (due to the

iterative bisections).

Most users are unlikely to be familiar with this approach.

7.2 Selected grid

For GlobAerosol it has been decided to adopt a sinusoidal grid withNeq= 4008.

Figure 2: Illustration of the sinusoidal (left) and icosahedral (right) gridding schemes.


15/26

8. THE MERGING ALGORITHM 15

8 The Merging algorithm

8.1 Temporal characteristics of the IGAPs and MGAP

A single merged product will be produced per day which corresponds to the Envisat observ-ing time (i.e. approximately 10:30am local solar time). This implies:

For most of the globe no interpolation in time is required for AATSR and MERIS (but

see below).

SEVIRI data must be interpolated in time from the universal time (UT) at which each

scene is acquired to the Envisat observation time (which is dependent on longitude and,

to lesser extent, latitude). This will be implemented as follows:

The UT corresponding to the Envisat sub-satellite point when over each grid point

will be determined for the nominal Envisat orbit. This will be defined in a merg-

ing parameter map (see below) once and for all before the merging scheme isapplied. The nominal times for each grid square are illustrated in figure 3

The two SEVIRI IOAPs which bound in time the UT of each grid point will be

identified2

SEVIRI products from both the bounding IOAPs will be input to the merging

algorithm. The covariance associated with each of the two SEVIRI products (see

equation12below) will be weighted to effectively perform interpolation in time,

simultaneously accounting for any differences between the retrieval uncertainties

of the two products:

Sx

1 = ek(tnomts)2

Sx

1 (11)

Where Sx is the original matrix valid at SEVIRI GAP time ts before or after theMGAP nominal time,tnom. k is a temporal de-correlation factor. At present it isassumed that k = 0.0192541hours2, such that the variances are doubled for a6 hour time difference. S

x

for each of the two SEVIRI products will be used in

equation12.

Near the poles multiple retrievals may be available from overlapping AATSR and (par-

ticularly) MERIS swaths. These cases will be treated in the same way as SEVIRI: The

two retrievals which bound in time the nominal UT of the Envisat nadir point will be

identified and merged with other results using the same de-correlation factor, k. I.e. in

general the approach outlined for SEVIRI should, in fact, be implemented for all instru-ments. Over most of the globe, this will result in no temporal interpolation for AATSR

and MERIS, but will exploit multiple retrievals at high latitudes where appropriate.

9 Merging co-located retrievals

9.1 General approach

Assumptions:

2In principle all retrieval throughout the day could be considered to allow cloud-free retrievals from distant times

to be interpolated when those nearest in time are contaminated by cloud. It is not proposed to implement this level

of sophistication since the changes in meteorological conditions implied by the changing cloud cover increase in an

ill-defi ned way the interpolation uncertainties.


16/26

16

0

5

10

15

20

Time / GMT

Figure 3: Nominal time in UT of each grid square in the GlobAerosol grid, approximately cor-

responding to the time at which Envisat over-flies each square (i.e. 10:30 local solar time of thedescending node crossing). Negative values indicate a time on the preceeding day (offset by 24

hours).


17/26

9. MERGING CO-LOCATED RETRIEVALS 17

Merging is only applied to retrievals which correspond to the same GlobAerosol grid box.

No attempt is made to spatially interpolate retrieved data.

For a given box, we are given Nobservations of the same set of quantities described by

vectorsxi=1...N. Eachxiis subject to errors described by a Gaussian probability density function with covari-

ance Sxi.

Observation errors are not correlated between the N instances ofx.

The most likely estimate ofxis given by the weighted mean:

x=

Ni=1Sx1i

1

Ni=1Sx1i xi

(12)

The covariance of this merged result is given by:

Sx = Ni=1

Sx1

i1

(13)

In the current context eachi corresponds to a different IOAP. No spatial interpolation will beperformed, so N will vary from 0 to 6 (AATSR, MERIS, SEVIRI observations at two times which

bound the nominal time of the merged product), depending on the presence of valid results from

each IGAP at each grid-point.

A number of issues arise when considering practical implementation of this approach. These

are discussed below:

9.2 Merging parameter maps

For the purpose of controlling the merging scheme a number of parameters must be defined. Many

parameters will be expected to exhibit spatial variation to some degree. To maintain generality and

simplicity in the merging code, most parameters will be defined in a merging parameter map

(MPM), a file in which a given parameter is specified at each of the MGAP grid points.

The spatial variation within each file will be defined on a case-by-case basis in WP5200, but

whatever variation is implemented will have no implications for the coding of the merging scheme.

In many cases the quantities in the MPMs will vary to account for differing retrieval scheme

performance under specific geophysical conditions (e.g. over land / sea or particular surface type).

Multiple MPMs may be required to cope with varying observing conditions (particular solar zenith

angle). For these cases valid ranges (e.g. of solar zenith angle) will be defined and the appropriate

MPM selected (no interpolation between MPMs will be required).

9.3 Treatment of bad data

Before merging a number of tests will be performed on each IGAP retrieval at each grid point. If

a test is failed, the particular AATSR / SEVIRI / MERIS IGAP point will be flagged as bad and

not included in the merging. Tests include:

Satisfactory cost function value at solution.

Satisfactory cloud fraction.

Satisfactory estimated errors from the retrieval scheme.

Level of contamination by sun-glint.

Each threshold will be defined in an MPM (though some may not vary spatially).

Specific settings are given in section10, below.


18/26

18

9.4 Treatment ofAngstrm coefficient

To ensure consistency in the final merged product only the two optical depths (0.55 and 0.865 m)will be merged via equation12. Angstrm coefficient will be derived from the resulting merged

optical depths using equation3

9.5 Treatment of aerosol speciation

It is an assumption of equation12that the xicorrespond to the same quantity. This assumption willbe violated if the IGAP optical depths are derived assuming different aerosol types. It is therefore

important to first identify the type which is to be assigned to the merged product. The merging

equation12can then be applied to combine AODs from SEVIRI and ASTSR which correspond to

the same type with that from MERIS (which has no assigned type). I.e. an overall best type is

identified for each scene in the MGAP and it is the AATSR and SEVIRI retrievals for the overall

best type that are merged (provided satisfactory retrievals exist for this type in each case), and

these will not necessarily be the retrievals which are flagged as best in the individual IOAP files.

The ORAC-based AATSR and SEVIRI IGAPs include AOD for all 5 modelled aerosol types.

Criteria are defined to select the best type based on experience gained during the preliminary val-

idation of the products. However there is generally insufficient information in the ORAC retrieval

for an unambiguous identification of the aerosol type. It is therefore likely that the AATSR and

SEVIRI IOAPs, and even the IOAPs from the same sensor at the two times bounding the nominal

merging time, will not agree as to which is the best aerosol type for a given grid box.

Since SEVIRI provides more uniform coverage over the disk it observes, it is likely to result in

fewer / less pronounced discontinuities in the MGAP if SEVIRI is used to define the aerosol type

where SEVIRI data is present. However the SEVIRI scene at 16:15 UT is quite distant in time

from the nominal merging time (within the SEVIRI observed disk).The merging algorithm will select the aerosol type by taking the type identified as best from

the first IOAP in the following priority list with valid retrievals for a given scene:

SEVIRI IOAP at the time nearest to the nominal merging time,t0. (This will be the 10:15UT scene.)

AASTR IOAP att0.

SEVIRI IOAP at the other time bounding the nominal merging time, t1. (This will be the16:15 UT scene.)

AASTR IOAP att1.

If only MERIS data is present then the aerosol type will be flagged as unknown.

9.6 Determination of the covariances

The Sxfor each IGAP will be a 2x2 matrix (corresponding to the two AODs).

Currently it is proposed to neglect error correlations between the two optical depths and there-

fore the off-diagonals of this matrix are set to 0. The variances are defined as follows:

For ORAC-based retrievals variances are taken from the estimated retrieval error reported in

the IOAP. For MERIS no errors are reported in the IOAP and so the variances are estimated from the

spread of the retrieval values about co-located Aeronet measurements, as determined in the

preliminary validation.


19/26

10. STEP-BY-STEP IMPLEMENTATION OF THE MERGING ALGORITHM 19

9.7 Correcting sensor dependent biases

Equation12assumes errors are not correlated between the IGAPs. In practice each IGAP will

have specific geo-temporally dependent biases. It is proposed to correct for these biases by (again)

using results from the preliminary IGAP validation: Scatter-plots of IGAP AOD vs Aeronet AOD will be produced.

Linear (optionally second order) fits to the relationship between IGAP and Aeronet AOD

will be calculated (using a least-squares fitting approach).

The coefficients of these fits are stored in regions local to the corresponding Aeronet stations

in MPMs.

Each IGAP AOD is corrected by the fitted first or second order polynomial before application of

equation12.I.e. each IGAP AOD is bias corrected relative to the Aeronet AODs.

Coefficients will be defined for each IGAP, for each of the five aerosol types in a specific MPM.

MPMs may be defined for particular ranges of solar zenith angle depending on the outcome of the

validation.

10 Step-by-step implementation of the merging algorithm

Having defined the basic principles and equations behind the merging process in the preceeding

sections, the following provides a step-by-step description of the complete process, filling-in de-

tails required for the coding of the algorithm.

The process is also illustrated in figures4and5. Figure4illustrates the overall process, includ-

ing the operation of the ORAC retrievals on level 1 radiance data. Figure5illustrates the mergingprocess, including the identification of the best type for the MGAP.

Note that the scheme is applied 5 times: Four times to the IOAPs of each sensor individually

to generate IGAPs, and a final time including IOAPs from all sensors (other that ATSR-2) together

to form the MGAP. Here we describe the MGAP formation, since this is the general case. IGAP

formation is identical, but taking only IOAPs from the specific sensor as input. IOAPs are also gen-

erated using ATSR-2 data, which is not included in the MGAP because ATSR-2 is not processed

over the period contemporary with the other instruments.

To generate ATSR-2, AATSR and SEVIRI data ORAC is run using input files which contain

cloud-free radiances averaged onto the GlobAerosol sinusoidal grid. MERIS data is also

produced on the same grid. This ensures that no spatial interpolation is required to generatethe IOAP, IGAP or MGAP.

ORAC output files which are produced for every orbit and every aerosol type are amalga-

mated into the IOAPs. These contain all the information in the original ORAC output files

for a given orbit for all aerosol types. Additionally, the best aerosol type is indicated by a

flag. The best type is determined by (i) applying quality control criteria to screen out bad

retrievals (ii) selecting the retrieval with lowest cost from those retrievals which pass these

tests. The following quality control criteria have been determined during the preliminary

GAP validation and are stored in MPMs:

For SEVIRI:

urban-type retrievals are never considered valid for this purpose. Urban and

polluted maritime results are not considered in the amalgamation (doing so led to

confusing results).


20/26

20

Cloud fraction according to the SEVIRI mask is 0.

The surface albedo at 0.55m is less than 0.12.

The retrieval has converged.

The optical depth at 0.55

m is less than 3.

More than one iteration of the retrieval was performed.

The retrieved effective radius is less than 8m.

The following cost function thresholds are applied depending on aerosol type and

whether the grid-box is over land or sea (partial land is considered as land for this

purpose).


21/26

11. OUTPUTS OF THE MERGING SCHEME 21

2. The SEVIRI, AATSR and MERIS retrievals in the same grid box which are closest in

time are identified. The closest time (in UT) of each instrument is referred to ast0.

3. In each case, if it exists, the retrieval at the other time,t1which boundstnomis identi-fied. t0 is always the closest in time so it may be the case that either t0 < tnom < t1ort0 > tnom > t1.t0and/ort1 may be on the previous or next day relative to the nominalday of the MGAP. Retrievals more than 12 hours from the nominal time will not be

considered in any case.

4. For the ORAC-based retrievals (ATSR-2, AATSR and SEVIRI), the retrievals which

pass the quality control criteria above are again identified.

5. ??? GMV Define MERIS Quality Control here ???

6. The type which will be assigned to the given grid-box is identified by taking the identi-

fied best retrieval from the first data-set with valid retrievals in the priority list defined

in section9.5. If the only valid data present is MERIS, then the type will be classified

as unknown.

7. The set of retrievals which (a) pass the quality control criteria and (b) correspond to the

selected type or are from MERIS (unknown type) are included in the merged value for

the given grid point. These retrievals are handled as follows:

(a) The22estimated error covariance matrix for the optical depths at 0.55 and 0.87m is formed for each retrieval, as described in section9.6.It must be ensured thatthese error covariances are appropriate to the representation of the retrieval optical

depths inlog10units.

(b) Equation11is applied to each covariance to account for the error growth with time

away from the nominal merging time.(c) The optical depth values are bias-corrected using the gain and offset parameters

determined from linear-fits to the Aeronet vs. retrieved optical depth scatter plots

during the preliminary validation exercise.

() =c0+c1() (14)

Where()is the retrieved optical depth,()is the corrected optical depth, andc0, c1are offset and gain parameters defined in MPMs. Note that the correction isdefined in absolute optical depth units, hence the stored (log10) values must be con-verted into absolute units before applying this equation and the result subsequently

converted back intolog10units.

(d) Equation12Is applied to the corrected optical depths to determine the merged

optical depths. Note this equation should be applied directly to the storedlog10optical depths.

8. The merged Angstrm coefficient is determined from the merged optical depths using

equation3.

11 Outputs of the merging schemeThe following parameters are required in the MGAP product.

A version number identifying the IGAP processor versions.


22/26


23/26

11. OUTPUTS OF THE MERGING SCHEME 23

Figure 5: The merging algorithm including quality control and selection of the merged aerosol

type.


24/26

24

A version number identifying the MGAP processor versions.

A version number identifying the set of MPMs used.

Time and date of data applicability and processing.

The latitude, longitude and UT of the MGAP grid points. (The implied grid-box polygons

should also be defined in a separate, time independent file).

For each grid-point:

Index of the grid point.

AOD at 0.55 and 0.865 microns.

Estimated covariance of the two AODs.

Angstrm coefficient .

Aerosol type indication.

Indication of most likely type from the AATSR / SEVIRI IGAP if different.

Flag indicating which IGAPs were valid and therefore merged, including identification

of the two possible observations bounding the nominal MGAP UT for each instrument.

12 Requirements on retrieval algorithms imposed by Merging

scheme

The 5 aerosol classes selected for retrieval should employ similar optical models and altitudeprofile shapes.

The schemes should employ either (a) the same gridding scheme as the MGAP or (b) retain

retrievals at finer spatial resolution such that interpolation onto the MGAP grid does not

result in degraded information.

13 MPM file format

13.1 Basic assumptions

The following is a description of the format adopted for the MPM files. The following general

points apply:

Parameters in an MPM file can be defined as a function of month. Month is an integer

giving the month number since April 1995 (the start of the ATSR-2 mission). If a specific

parameters is only defined for 1 month then it is to be assumed that the given values apply

for all time. Linear interpolation in time is to be assumed where MPMs are defined for more

than one month. Extrapolation should not be performed, the defined value which is closest

in time to the required time should be taken if the required value is outside the defined time-

range.

MPM files are binary files written by IDL on a 32-bit linux machine.

Long integers are 4 bytes long. Short integers are 2 bytes. Floats are 4 bytes. Doubles are 8

bytes.


25/26

13. MPM FILE FORMAT 25

13.2 File format

The following describes the contents of each file:

An MPM string (see13.5) containing the file version.

Long integer defining the MPM type, MPM_TYPE. This can have value 1 or 2:

MPM_TYPE=1: Each tile in the sinusoidal grid is given a region number (e.g. 0 for

land, 1 for ocean). MPM parameters are defined for each of the numbered regions. If

only one region is defined then the parameter is assumed to apply globally.

MPM_TYPE=2: Each tile in the sinusoidal grid is given a specified value of the merging

parameter.

The type is chosen on a parameter by parameter basis to minimise the size of the MPM file.

Long integer defining the number of grid points round the equator in the sinusoidal grid.

This will be 4008 for GlobAerosol:

MPM string containing the name of the file when originally produced.

MPM string describing the origin of the file (RAL, Oxford, GMV etc).

MPM string describing the history of the file.

MPM string containing the creation data of the file as returned by the IDL systime command,

e.g. Wed May 31 12:59:41 2006.

MPM string defining the instrument to which the MPM applies. This can be one of the

following:

SEVIRI, AATSR, ATSR-2, MERIS

1 long integer defining the number of parameters in the file: N_PAR.

N_PARMPM strings giving the name of each parameter.

1 long integer defining the number of aerosol types for which the parameters are defined:N_AER. If this is 1 then the parameters apply to all aerosol types.

N_AERMPM strings giving the name of aerosol type.

1 long integer defining the number of wavelengths for which the parameters are defined:

N_WL. If this is 1 then the parameters apply to all wavelengths.

N_WLfloat values giving the value of each wavelength.

1 long integer defining the number of months for which the parameters are defined: N_MONTHS.

If this is 1 then the parameters apply to all months.

N_MONTHS long values giving the value of each month.

The remaining file contents depend on MPM_TYPE.


26/26

26

13.3 MPM file type 1

1 long integer defining the number of regions for which parameters are defined:N_REG.

1 long integer giving the type, I_TYPE, of the MPM variable as follows:

1: byte

2: short signed integer (2 bytes)

3: long signed integer (4 bytes)

4: float (4 bytes)

5: double (8 bytes)

An array, PARS, of typeI_TYPE with dimensions

N_PAR, N_WL, N_AER, N_MONTHS, N_REG

This contains the values of the merging parameter.

If the number of regions is greater than 1 then regions are defined on the sinusoidal grid as

follows:

1 long integer giving the number of tiles in the sinusoidal grid being used:N_GRID.

N_GRID short signed integers giving the region number for each grid tile, I_REG (zero-

based index to

N_GRIDshort signed integers giving the u-coordinate of each grid tile (see section 7.1).

N_GRIDshort signed integers giving the v-coordinate of each grid tile (see section 7.1).

13.4 MPM file type 2

1 long integer giving the number of tiles in the sinusoidal grid being used:N_GRID.

1 long integer giving the type, I_TYPE, of the MPM variable as defined in section13.3.

An array, PARS, of typeI_TYPE with dimensions

N_PAR,N_WL,N_AER,N_MONTHS,N_GRID

This contains the values of the merging parameter.

N_GRIDshort signed integers giving the u-coordinate of each grid tile (see section 7.1).

N_GRIDshort signed integers giving the v-coordinate of each grid tile (see section 7.1).

13.5 MPM string

1 long integer defining the length of the string: LEN

LENbytes of ASCII encoded text (if LEN>0).

Algorithm Theoretical Basis Document (ATBD) Merging_atbd_v2p1

Documents