Institut für Umweltphysik/Fernerkundung Physik/Elektrotechnik Fachbereich 1 Overview about new processor baseline H. Bovensmann, G. Lichtenberg, R. Snel, C. Lerot, T Fehr (on behalf of SQWG) ENVIVAL-LIFE Meeting, 6.12.2010, Bremen
Jan 16, 2016
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
Overview about new processor baselineH. Bovensmann, G. Lichtenberg, R. Snel, C. Lerot, T Fehr
(on behalf of SQWG)ENVIVAL-LIFE Meeting, 6.12.2010, Bremen
Overall Goals SQWG
• The main goal for the SQWG is establishing L1 and L2 high data quality for the operational data products of the full SCIAMACHY data set, which allows substantial contributions to long term atmospheric data sets in the areas of:
» the development of the O3 layer
» global air pollution » Global Greenhouse gas observations» Solar-Terrestrial interactions
• In addition, the data SCIAMACHY data set shall allow to investigate new scientific methods and new products.
SCIAMACHY QWG
• SCIAMACHY Quality Working Group– Inter-Agency Activity: ESA-DLR-NSO– Maintenance and development of the ESA operational
SCIAMACHY processors and tools• Level 1b / Level 2
• SCICAL
• SciaL1C
– Established 2006– Extension 2011-2012 currently prepared
• SQWG Statement of Work currently finalised (2011/12)
• SCIAMACHY ESL contract: Proposal received from DLR (2010-2012)
– Coordinator: Heinrich Bovensmann
SQWG TeamIUP: S. Noel, K. Bramstedt, P. Liebing, M. Buchwitz, C. von Savigny, A.
Richter, A. Rozanov, F. Wittrock, M. Vountas, A. Kokhanovsky, K. U. Eichmann, M. Weber, J.P. Burrows, H. Bovensmann
SRON: R. van Hees, P. van der Meer, J.M. (Matthijs) Krijger, R. Snel, I. Aben
BIRA: C. Lerot, M. van Roozendael
KNMI: L.G. (Gijsbert) Tilstra, P. Stammes
DLR IMF: G. Lichtenberg, A. Doicu, S. Gimeno Garcia, F. Schreier, M. Meringer, D. Sherbakov, S. Slijkhuis, B. Aberle, K. Kretschel, S. Hrechanyy, M. Gottwald
ESA: T. Fehr
SERCO: A. Dehn, G. Brizzi
• Level 1 IPF 7.04 compliant with ENVISAT 2010+ mission extension– No product format changes– No algorithm changes wrt IPF 7.03– CFI 5.8.1 implemented
• Level 2 Off-Line Processor SGP 5.01– Product format substantially changed. – Additional trace gases:
• Nadir columns of SO2, BrO, H2O, OClO and CO and BrO Limb profiles
• Limb Cloud indicators for cloud presence and type
• Improved Absorbing Aerosol Index.
• Level 2 “Fast-Delivery” processing with SGP 5.01 (<24h) at D-PAC based on Level 1b NRT and predicted ADFs
Operational SCIA Processors activated Feb. 2010
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
Level 1 v7
Level 0-1 processor (IPF L1 v7)
Stray light matrix channel 2 introduced
Writing of limb mesosphere states to limb MDS introduced
On-board error of writing scan encoder values corrected in processor
SciaL1c
Version 2.1 delivered October 2008
New:
M-factor handling
Bug fixes
ASCII output
Build environment improved
Degradation Correction: M-Factor’s• L1b Version 7 m-factors:
– NRT:• Quasi-operational forward-processing V7 running since Feb 2010 at IUP.• Variable limits in m-factor quality checks implemented by ESA• M-factor plots are included in the bi-monthly report
– Reprocessing:
• L1b monitoring data 08/2002 – 01/2010 completely provided by DLR.• M-factors for V7 re-processing are complete and delivered to ESA.
• Documentation, m-factor data base, plots & applicator s/w are also available via m-factor website:
http://www.iup.uni-bremen.de/sciamachy/mfactors
• M-factors for L1b version 6.0x (08/2002 – 01/2010), still available too.
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
Level 2 v5
Level 1b-2 Version 5 – Product Overview(Blue: New, Green: Improved)
Nadir UV/VIS:O
3, NO
2, BrO (BIRA, SDOAS/GDP4)
SO2 (volcanic & anthropogenic), OClO (IUP, algorithm using
SDOAS/GDP4 formalism),
H2O (IUP, AMC DOAS)
Nadir SWIRCO (IMF, BIRRA)
Nadir Clouds & AAI:Cloud Top Height & Optical Thickness (IUP, SACURA)
Nadir Cloud Fractions (IMF, OCRA)
Absorbing Aerosol Index (KNMI)
Limb:O
3, NO
2, BrO Profiles (IMF: operational version, IUP: retrieval
settings)
Cloud flagging (IUP, SCODA)
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
Nadir Products
Nadir O3 total columns
• Algorithm Heritage: SDOAS (BIRA)
• Basic algorithm settings as L2 v3.01
• Retrieval settings:
– Fit window: 325 – 335 nm
– Daily calibrated ESM irradiance spectra (D0).
– Radiances fully calibrated.
– M-factors.
– Cloud parameters from OCRA/SACURA.
• Expected quality / known issues:– Small temporal trend (up to
0.5%/year), minimised by applying m-factors at least in the tropics.
Impact of m-factors on total O3 (%)
SCIAMACHY OL 3.01- reference (%)
See also C. Lerot et al., Atmos. Meas. Tech., 2, 87-98, 2009
O3: SCIAMACHY trend m-factor impact
• Trend with respect to GOME1:– systematic trend of -4%/decade
for nearly all latitudes w.r.t. GOME
• This trend does not dependent on algorithm (see Lerot et al., 2009, Loyola et al., 2009)
– m-factor corrections improves situation in the tropics (see also Bramstedt et al., 2009)
– But: m-factors do not completely remove the observed trend in SCIAMACHY total ozone
L1: -all calibrations on -solar reference (ESM diff.) L1 w./m-factor: -all calibrations on, but no etalon
M. Weber
Nadir NO2 total columns
• Algorithm Heritage: SDOAS (BIRA)
• Basic algorithm settings as L2 v3.01
• Retrieval settings:
– Fit window: 426.5 – 452.5 nm
– Daily not calibrated ASM irradiance spectra (A0).
– Radiances extracted without radiometric calibration (no m-factors)
Higher fit quality and equivalent NO2 columns
– Cloud parameters from OCRA/SACURA• Expected quality / known issues:
– To be validated …
Nadir BrO total columns
• New implementation
• Algorithm Heritage: SDOAS (BIRA) Slant column retrieval as SGP/SDOAS AMF calculation uses new BrO profile stratospheric climatology
developed by BIRA used (Theys et al. ACP 2009)
• Retrieval settings:– Fit window: 336 – 351 nm
– Daily not calibrated ASM irradiance spectra (A0)
– Radiances extracted without radiometric calibration (no m-factors)
– Cloud parameters from OCRA/SACURA
• Expected quality / known issues:– small difference in VCDs of about 2x1012 molec cm-2 on average
between the reference algorithm and operational processor
– Stratospheric AMF not representative for trop. enhanced BrO
Nadir OClO slant columns• New implementation
• Algorithm Heritage:
– Slant column retrieval as SGP/SDOAS– Settings from IUP
• Retrieval settings:– Fit window: 365 – 389 nm
– Daily not calibrated ASM irradiance spectra (A0)
– Radiances extracted without radiometric calibration (no m-factors)
– SZA cut-off = 95°
• Expected quality / known issues:– Small but relatively constant low bias of 3x1013
molec cm-2
– polynomial degree will be adapted to minimise bias
Note: OClO only present in chlorine activated polar vortex at SZA > 90°
Nadir SO2 total column
• New implementation
• Algorithm Heritage:
• Slant column retrieval as SGP/SDOAS• Retrieval and AMF settings from IUP
• Retrieval settings:• Fit window: 315 – 327 nm
• Daily not calibrated ASM irradiance spectra (A0)
• Radiances extracted without radiometric calibration (no m-factors)
• SCD post-processing step: reference sector subtraction• daily subtraction of reference sector SCD (Pacific) to
compensate for SZA and ozone sensitivity of retrieval etc.
• SCD is converted to VCD using simulated AMF’s for • “anthropogenic pollution”
• “volcanic eruption” scenarios
SO2: Volcanic Columns
• Globally, there are several artefacts:
• Strongly negative values at high latitudes
• Sea – land contrast• Very large values in
winter high latitudes• Some scattered high
values• Artefacts in IUP Bremen
product have similar pattern but much smaller values
=> Product under revision
SAA
SAA
Nadir H2O total column
Precision estimate (w.r.t. ECMWF)• 3 - 5% for columns >0.6 g/cm2 • small columns: up to 10%• Absolute precision always better than 0.2 g/cm2
• New implementation
• Algorithm Heritage:
– Air Mass Corrected (AMC)-DOAS (Noël et al., 2007)
• Retrieval settings:– Fit window: 688 – 700 nm
– Daily not calibrated ASM irradiance spectra (A0)
– Radiances calibration: memora effect, leakage, wavelength (no m-factors)
• Key product entries: – Vertical column density (VCD), error in VCD, air
mass factor (AMF) correction
• Expected quality / known issues:– Average bias -0.2 g/cm2 w.r.t. ECMWF
Nadir CO column
• New implementation• Algorithm Heritage:
– BIRRA (direct non-lin. least square fit to radiance, Schreier et al. 2009)
• Retrieval settings:
– Fit window: 2324 – 2335 nm (channel 8)
• Expected quality / known issues:• highly sensitive to DBPM• Over sea signal too low, numerical effects can play a role• Most likely use case: Means over many orbits• Mean values will depend on filtering of data• Current version: not to be used• Algorithm and settings under revision
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
Limb Products
Limb Changes
• The following changes were made to the Limb retrievals
– New Product Limb cloud flags
– New Product BrO Profile
– Limb Retrievals recognise the cloud height from cloud flag product, i.e. the retrieval starts at the first cloudfree height found above the minimum height
• Note: small trend in pointing knowledge over time may affect trends in all limb products – under continues investigation
V3.01 V4
Limb O3 Profile
• Optimised Retrieval
• Algorithm Heritage: Optimal Estimation (IMF, Doicu et al.) Optimisation of retrieval settings IUP
• Retrieval settings: Spectral range 520 – 590 nm Fine altitude grid (33 layers instead 24) 4th order polynomial Picard iteration – Limb radiance ratio w.r.t. 46 km TH– LOWTRAN aerosol model (background conditions) used in RTM– Retrieval starts above clouds using cloud parameters from SCODA
• Expected quality / known issues:– Bias as obseved in v3.01 O3 should be much smaller
– Clouds have impact on O3 around tropopause
• Validation ongoing
Limb O3 improvements vs. LIDAR
L2 v4.4 L2 v5L2 v3.01
Comparison: Anne Gijsel, RIVM via ESA VALID / EQUAL project
Limb NO2 Profile
• Optimised Retrieval
• Algorithm Heritage: Optimal Estimation (Doicu et al. Optimisation of retrieval settings IUP
• Retrieval settings: Optimised settings
Fine altitude grid (33 layers instead 24) Picard Iteration Extended spectral range 420 – 470 nm Reduced regularization parameter
– Limb radiance ratio w.r.t. 43 km TH
– Cloud parameters from SCODA
– Retrieval starts above clouds
• Expected quality / known issues:– To be validated
Limb BrO Profile
• New Retrieval
• Algorithm Heritage: Optimal Estimation (Doicu et al. Optimisation of retrieval settings IUP
• Retrieval settings:– Spectral range 337 – 357 nm
– Limb radiance ratio w.r.t. 35 km TH
– Cloud parameters from SCODA
– Retrieval starts above clouds
• Expected quality / known issues: deviations on average up to 40% were found above with a large standard
deviation For not too large SZA‘s and in certain latitude bands on
average difference < 20%– Some „data gaps“ – under investigation
Limb Cloud Flagging• New Product
• Algorithm Heritage: SCODA using colour index ratios to identify
strong scattering layers in limb (IUP, Eichmann et al.)
• New Limb MDS introduced:• FLAG: 0 = no cloud 1 = patially
cloudy2 = fully cloudy 3 = bad
data• MAXVAL: max. color index ratio,• MAXHEIGHT: tangent height (in km) where
MAXVAL is found• MAXHEIGHTIDX: index of MAXHEIGHT• each for three different cloud types
– WCL: normal water clouds,– ICL: ice clouds– PSC: polar stratospheric clouds.
• Expected quality / known issues:– To be validated
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
Summary/Outlook
From Level 2 V 3.01 to V 5.01Product Improvements
Level 2 V 3.01 V5.01
2007 2010
Nadir Products
Absorbing Aerosol Index
quality not sufficient improved algorithm and usage of degradation corrections
Ozone total column
slight trend over time (< 0.5% Per year),
GDP 4
degradation correction taken into account;
smaller trend expected
NO2 offset removed reference spectra
BrO VCD
SO2 VCD, volcanic and pollution
OClO SCD
H2O VCD
CO / xCO VCD xCO quality tbc
Cloud parameter (Cloud fraction, Cloud Top Height)
OCRA/SACURA improvements due to degradation correction;
new minimum reflectance data base improved OCRA CF
From Level 2 V 3.01 to V 5.01 Product Improvements
Level 2 V 3.01 V5.01
2007 2010
Limb Products
Ozone profile TH offset removed, maximum of 4 limb
O3 profiles per tangent height
improved forward model, optimised
retrieval settings, => substantially smaller low bias; clouds and
aerosol improvements lower stratosphere
NO2 profile TH offset removed improved forward model, optimised
retrieval settings => improved lower strat.;
clouds taken into account
BrO profile product newly implemented
Limb cloud flagging
product implemented, verfied and used in O3, NO2 and BrO retrieval
Preliminary Validation• SCIAVALIG Meeting 07/08 September 2010, BIRA
– In general good agreement of the SCIAMACHY products and available validation data
• Ozone Limb profiles are clearly improved
• New products are considered of good or sufficient good quality
• SO2 slant and anthropogenic columns show product deficits.
• CO column retrievals need an optimisation
– SCIAMACHY Level 2 reprocessing interrupted– Improved baseline delivery expected February 2011
• OClO, CO and SO2 improvements
– ESA/SQWG decision to postpone reprocessing re-start until availability of corrected baseline
– SCIAMACHY delta-Validation Report made available:http://envisat.esa.int/earth/www/object/index.cfm?fobjectid=7279
Level 2 Version 5.0x (1. Quarter 2011)
• Nadir OClO: bug-fix in degree of polynomial will reduce bias in SC
• Nadir SO2: bug-fix in reference sector data base routine
• Nadir CO: bug-fix in
• Limb O3: impact of aerosol a-priori under investigation
• Limb BrO gaps under investigation
Level 1 version IPF version 8 (2011/12)
Work in progress ...
• Changes (Verified/ To Be Verified)– Memory Effect Limb improved– Non-Linearity PMD scaling noise check introduced– Hot Pixel Check Limb improved– Polarisation for occultation switched off– SAA check for etalon switched off– Selection of individual dark states– Stray light matrix channels 3-8– Key-data and scan angle dependent degradation
Next Baseline: Level 1b-2 Version 6 (2011/12)
New in version 6 (Verified)• Porting to 64bit + current versions of libraries and compilers• AAIA improvements (use Ozone columns from retrieval)• Nadir Snow/Ice separation• VCD HCHO• VCD CHOCHO• Trop. NO2 (limb-nadir)• Improved CO• XCH4 or XCO2 (tbc)• NLC detection Limb• Ozone Profile extension to lower Mesosphere
ESA Atmospheric Mission2nd half 2013
mid-2015 (to be confirmed)
2018 ?
2017 ?
12/2014
04/2013
2018
2020
12/2013 ?
06-07/2011 !
Outlook: ESA‘s „atmospheric“ missions
Figure courtesy T. Fehr, ESA
We need to lobby for a mission extension beyond 2014 !
Coming soon: The SCIAMACHY „Biography“
Institut für Umweltphysik/Fernerkundung Physik/ElektrotechnikFachbereich 1
The End