Interfaces EO data with Atmospheric and Land Surface Model: Progress report Liang Feng, Paul Palmer
Feb 24, 2016
Interfaces EO data with Atmospheric and Land Surface Model: Progress
report
Liang Feng, Paul Palmer
Reference OSSE system High-resolution model CO:CO2 ratios
Project Tasks
I. Progress overviewOverall Aim: Developing a reference OSSE simulation for community researchers to evaluate impacts of space-borne atmospheric composition measurements on surface flux estimates. Outputs:
A. OSSE framework with flexible modules and libraries. Status: Revised and ready for beta-test.
B. One complete example OSSE system for 1) simulating OCO-like XCO2 observations; and 2) estimating regional CO2 fluxes by assimilating XCO2 observations using an Ensemble Kalman Filter (EnKF).
Status: close to be finished . C. Detailed documents. Status: beta version due in Dec, 2012. D. Visualization Tools. Status: to be started.
Observing System Simulation Experiment Tool
II. OSSE framework.
III. Example OSSE system
Today we will focus on:
II. OSSE framework
Data flow and directory structureMain modulesStyles
Obs+random errors
Flux Forecasts
Obs operator
Forecast(3-D concentrations)
CTM
Prio
r + e
rror Posteriori +
error
Observation
Simulation
EnKF assimilatio
n
ETKF Model ObsEnsemble
Ensemble forecasts(3-D fields)
Surface flux Ensemble CTM
Obs operator
Model world: Object, process, measurements, and estimates.
Measure
Transport
Top-down estimate
Atmosphere
Observation
Surface flux
Flux modelinventory
Met files
Disk IO
observation
ctm
lib
surface_flux
atmosphere
etkf
instrument
We have hence constructed a directory structure for OSSE tools as:
Main modules
Module name Description
etkf_m.py Calculate a posteriori ; gain matrix (K); transform matrix (T) , by using ETKF algorithm.
state_vectort_c.py Class for state vector and its projection to surface fluxes: a). construct, manage, and update state vectors and their perturbation ensembles. b). maintain a (virtual) projection to surface fluxes.
construct_error_cor_m.py Construct (spatial) error correlations from regional flux perturbation (basis) functions
contruct_state_vector_m.py Interface for user to construct state vector from flux perturbation (basis) functions
1. etkf
Module name Description
ctm_grid_c.py Class for tempo-spatial grid for modelling atmospheric compositions.
ctm_field_c.py Class for gridded model data of single atmospheric tracer. It also contains horizontal and vertical interpolation functions etc.
ctm_slice_c.py Class for tracer profiles along sampling track etc. ctm_world_c.py General container for tracer data ( field or slice). It
provides: a) member management (add, group, remove, update, search, retrieve etc) ; b) (virtual) disk IO.
Io_file_c.py Class for helping to access CTM outputs, it helps to :1) group file names for parameters to be read. 2) finalize extensions for file names.
ctm_file_read_m.py Interface for reading data from CTM outputsctm_file_write_m.py Interface for writing data to be read by CTM.
2. atmosphere
Module name Description
surface_grid_c.py Class for tempo-spatial grid for surface fluxes.surface_flux_c.py Class for gridded surface flux. It also contains functions
for: a) disk IO; b) re-gridding. flux_pb_c.py Class to contain perturbation (basis) flux maps. It has a
virtual disk IO for flux files used to force CTM. region_map_c.py Class for generating multi-layer region maps according to
user’s choices. gen_region_pb_flux_m.py construct class flux_pb_c from multi-layer regional maps
and the map for flux errors over the world.
3. surface_flux
Module name Description
observation_c.py Class for observations. It also contains functions to a) read (write) observations from/to disk files; b) filter and weight observations for assimilation.
ctm_observation_c.py Class for sampling model outputs at observation time and locations.
ens_observation_c.py Class for sampling ensemble model outputs at observation time and locations.
4. observation
Module name Description
ctm_config_c.py Class to generate configuration file to control CTM forecast simulations.
ctm_restartfile_c.py Class to (re-) construct CTM restart files.
5. ctm
Module name Description
sat_orbit_c.py Class for generating (reading ) satellite orbits.sat_avk_c.py Class for generating satellite averaging kernels from pre-
calculated tables.sat_oerr_c.py Class for generating satellite observation errors from pre-
calculated tables.check_cloud_m.py Functions for checking the probabilities of cloud
contaminations along satellite observation tracks from given cloud cover (probability) map and the viewspot penalty function
check_aerosol_m.py Function for calculating AOD along satellite tracks from given AOD probability maps.
6. instrument
Module name Description
time_m.py Functions for time conversions. gp_axis_c.py General class for axis. It has functions for indexing and
interpolating etc error_message_m.py Record and print error or warning messagefort_lib.f90 Functions for (faster and parallelized) array operations,
and equation solving.
7. lib
Styles
1. Simple, short and stable
2. Object - and task –orientation.
Only use stable python features and well-established packages
Example: ctm_field_c.py: class ctm_field: #members
obj_id:<integer>: id of the class instance. name:<string>: name of the tracerunit: <string>: unit of the datagrid: <ctm_grid_c>: grid. data:<array>: gridded dataid:<integer>: id of the tracer. category: <string>: category of the tracergroup: <string>: tracer grouptime_tag: <string>: time step for the data __attr_dict:<dictonary>: dictionary of the attribute
# functions (12 in total) def get_profiles(self, lon_lst, lat_lst, time_lst=None):
For example: by writing less than 30 lines, we can use class region_map to generate a complicated map for estimating regional fluxes at different spatial resolutions.
3. Easy to use
4. Easy to change/expand
Example: In ctm_world_c.py, efforts have been made to minimize the efforts needed to communicate with user’s CTM via disk files. The user now can read their model input/outputs by overriding member function read_ctm_to_fields in class ctm_world, or by overriding the interface module ctm_file_read_m.py.
III. Example OSSE system
Aim: use modules in OSSE framework to build a complete OSSE system for OCO-like measurements (Feng et al, 2009).
The system uses GEOS-Chem (which is, however, not part of the delivered package) as the chemistry transport model.
Obs ETKF
Flux Forecasts(climatology)
Obs operator
forecast
GEOS-Chem
Model ObsEnsemble
Ensemble forecasts
Surface CO2 Ensemble GEOS-Chem
Obs operatorPrio
r + e
rror Posterior +
error
1) Observation simulation
2) Data Assimilatio
n(+Perturbations)
Observation Simulation module
A. Surface Fluxes We use CO2 flux climatology as ‘the true fluxes’. We provide interfaces to store them into binary files for
GEOS-Chem CTM to read. B. Chemistry Transport Model
Functions in ctm_restart_c.py and ctm_config_c.py are changed (overridden) to generate input.geos and restart files for GEOS-Chem forward simulations.
C. Sampling• We override IO interfaces so that we can use class
ctm_world to read and sample 3D model fields along the satellite orbits.
• We used cloud and aerosol climatology files and the default algorithm as the inputs to screen cloud and aerosols.
D. Vertical integration/conversion• A standard module is used to convert model vertical profiles
to satellite retrievals, by applying the averaged kernels read from pre-calculated tables for OCO-like instrument.
Example: Simulating OCO XCO2 1) Sampled along Aqua
orbitsGEOS-Chem transport model: Biosphere (CASA), Biomass (GFED), Fossil (+Bio) fuel (ODIAC), Ocean (Takahashi)
2) Scenes with cloud or AOD > 0.3 removed
Jan
1-day
3) Averaging kernels applied
Pres
sure
[hPa
]Glint mode
Averaging kernels
Top-down Optimal surface flux estimation
x: regional surface fluxesyobs: measurements of atmospheric concentrations. H: Jacobian (CTM model).
)( fobs
fa HxyKxx
K=PfHT(HPfHT+R)-1 – Kalman gain matrix.Pf: a priori uncertainty matrix R: observation error matrix
Posteriori Priori gain observation model
Data Assimilation Module (Ensemble Kalman Filter)
Ensemble Approach
Key features: (Feng et al., 2009 ; 2011)• Represent a-priori uncertainties by an ensemble of flux
perturbations. )T
• Use a lag windows to limit computational costs. Any emission will only be constrained by observations within a following limited time period. After that period, it is considered to be well-known.
• Project the ensemble of perturbations together with prior estimates into the observation space using CTM.
• Use Ensemble Transform Kalman Filter (ETKF) to determine posterior fluxes, and the associated uncertainties from digesting observations.
A. State vector and ensemble representation of its uncertainties
),,(),,(),,( 0 tyxBFctyxFtyxF l
ll
Surface fluxes Prior estimates of surface fluxesBasis functions for pulse-like flux perturbations,
:coefficients to be estimated: State vector: , , , …, ]; Perturbations: , , , …, ]Error covariance: Full or partial representation: , , , …, ],
• We use region_map_c.py to define 144 global regions.
• We use gen_region_pb_flux_m.py to generate BFs from biospheric GPP map and the multi-layer map for the 144 regions.
B. Ensemble forecasts
• We override IO functions in pb_flux_c.py to generate the ensemble of flux perturbations according to the defined basis functions and pre-defined uncertainties, and save them in GEOS-Chem bpch2 format.
• We also override functions in ctm_config_c.py, and ctm_restart_c.py to support the GEOS-Chem tagged runs for projecting flux perturbation ensemble to the ensemble of atmospheric tracer concentrations.
C. Projection of ensemble forecasts to the observation space.
GEOS-Chem outputs are sampled by ctm_world_c.py with over-ridden IO functions.
D. Inversion algorithm:Ensemble Transform Kalman Filter (ETKF)
• A posteriori and the associated uncertainties are simultaneously calculated using SVD (or sparse-matrix LU) technique.
+R]-1
TTTT=[1+ -1
• Corresponding updates are made to the ensemble of model 3D concentrations to retain the contributions of fluxes outside the assimilation (time) windows (Feng et al, 2009)
Algorithm validation: Comparison with the LSCE 4d-var system.
In the comparison experiments, we have assimilated the same ACOS-GOSAT B210 XCO2 retrievals (real data) over 2009 and 2010, by using 2 different approaches: EnKF (UoE) and 4d-var (LSCE) (Feng et al, 2013).
VI. Summary
• OSSE system will be issued for beta test soon.
• Whole package, including detailed documents will be delivered in next march.
INTERNAL DEADLINES
1) Deliver revised code and documentation to Paul end of November 2012. This is before the Reading meeting. I will check the code and documentation. Return to LF a week later.
2) Circulate to Silvia, Annemarie, and Siegfried the week of the 10th December 2012. Start beta testing.
3) End of beta testing period at the end of January 2013.
4) Debug, edit, package, and produce online material by the week of the 21st February 2013.
5) Final edits and release the end of February.
We are about two weeks behind the internal deadline, but months ahead the project schedule.