Robin Hogan, Julien Delanoë, Nicky Chalmers, Robin Hogan, Julien Delanoë, Nicky Chalmers, Thorwald Stein, Nicola Pounder, Anthony Illingworth Thorwald Stein, Nicola Pounder, Anthony Illingworth University of Reading University of Reading Thanks to Richard Forbes, Steve Woolnough, Thanks to Richard Forbes, Steve Woolnough, Alessandro Battaglia, Doug Parker Alessandro Battaglia, Doug Parker What can we learn about clouds and What can we learn about clouds and their representation in models from their representation in models from the synergy of radar and lidar the synergy of radar and lidar observations? observations?
43
Embed
Robin Hogan, Julien Delanoë, Nicky Chalmers, Thorwald Stein, Nicola Pounder, Anthony Illingworth University of Reading Thanks to Richard Forbes, Steve.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
OverviewOverview• What do spaceborne radar and lidar see?• Towards a “unified” retrieval of ice clouds, liquid clouds, precipitation
and aerosol – Variational retrieval framework
• Results from CloudSat-Calipso ice-cloud retrieval– Consistency with top-of-atmosphere radiative fluxes– Evaluation and improvement of models– Spatial structure of mid-latitude and tropical cirrus
• CloudSat simulator to evaluate Unified Model over Africa• Challenges and opportunities from multiple scattering
– Fast forward model– Multiple field-of-view lidar retrieval
• First results from prototype unified retrieval• Outlook for model evaluation and improvement
What do CloudSat and Calipso see?
Cloudsat radar
CALIPSO lidar
Target classificationInsectsAerosolRainSupercooled liquid cloudWarm liquid cloudIce and supercooled liquidIceClearNo ice/rain but possibly liquidGround
• Lidar: ~D2, more sensitive to thin cirrus and liquid clouds but attenuated
CloudSat and Calipso CloudSat and Calipso sensitivitysensitivity
5
• In July 2006, cloud occurrence in the subzero troposphere was 13.3%
• The fraction observed by radar was 65.9%
• The fraction observed by lidar was 65.0%
• The fraction observed by both was 31.0%
Ingredients of a variational retrieval
• Aim: to retrieve an optimal estimate of the properties of clouds, aerosols and precipitation from combining these measurements– To make use of integral constraints must retrieve components
together• For each ray of data, define observation vector y:
• Define state vector x of properties to be retrieved:– Ice cloud extinction, number concentration and lidar-ratio profile– Liquid water content profile and number concentration– Rain rate profile and number concentration– Aerosol extinction coefficient profile and lidar ratio
• Forward model H(x) to predict the observations– Microphysical component: particle scattering properties– Radiative transfer component
The cost function• The essence of the method is to find the state vector x that minimizes
a cost function:
2 2
2 21 1
( )y x
i i
n ni i i
i iy b
y H x bJ
x+ Smoothness
constraints
Each observation yi is weighted by the inverse of
its error variance
The forward model H(x) predicts the observations from the state vector x
Some elements of x are constrained by a
prior estimate
This term can be used to penalize curvature in the retrieved profile
Retrieval Retrieval frameworframewor
kkIngredients developed before
In progressNot yet developed
1. New ray of data: define state vector
Use classification to specify variables describing each species at each gateIce: extinction coefficient , N0’, lidar extinction-to-backscatter ratio
Liquid: liquid water content and number concentrationRain: rain rate and normalized number concentrationAerosol: extinction coefficient, particle size and lidar ratio
3a. Radar model
Including surface return and multiple scattering
3b. Lidar model
Including HSRL channels and multiple scattering
3c. Radiance model
Solar and IR channels
4. Compare to observations
Check for convergence
6. Iteration method
Derive a new state vectorEither Gauss-Newton or quasi-Newton scheme
3. Forward model
Not converged
Converged
Proceed to next ray of data
2. Convert state vector to radar-lidar resolution
Often the state vector will contain a low resolution description of the profile
5. Convert Jacobian/adjoint to state-vector resolution
Initially will be at the radar-lidar resolution
7. Calculate retrieval error
Error covariances and averaging kernel
Lidar observations
Radar observations
Visible extinction
Ice water content
Effective radius
Lidar forward model
Radar forward model
Example ice cloud
retrievalsDelanoe and Hogan (2010)
Evaluation using CERES TOA Evaluation using CERES TOA fluxesfluxes
• Radar-lidar retrieved profiles containing only ice used with Edwards-Slingo radiation code to predict CERES fluxes
• Small biases but large random shortwave error: 3D effects?
Nicky Chalmers
ShortwaveBias 4 W m-2, RMSE 71 W m-2
LongwaveBias 0.3 W m-2, RMSE 14 W m-2
CERES versus a radar-only CERES versus a radar-only retrievalretrieval
• How does this compare with radar-only empirical IWC(Z, T) retrieval of Hogan et al. (2006) using effective radius parameterization from Kristjansson et al. (1999)?
Bias 10 W m-2
RMS 47 W m-2
ShortwaveBias 48 W m-2, RMSE 110 W m-2
LongwaveBias –10 W m-2, RMSE 47 W m-2
Nicky Chalmers
How important is lidar?How important is lidar?• Remove lidar-only pixels from radar-lidar retrieval• Change to fluxes is only ~5 W m-2 but lidar still acts to improve
retrieval in radar-lidar region of the cloud
ShortwaveBias –5 W m-2, RMSE 17 W m-2
LongwaveBias 4 W m-2, RMSE 9 W m-2
Nicky Chalmers
A-Train A-Train versus versus
modelsmodels• Ice water
content• 14 July 2006• Half an orbit• 150°
longitude at equator
Delanoe et al. (2010)
Gridbox Gridbox mean:mean:
In-cloud In-cloud mean:mean: • Both models lack
high thin cirrus• Met Office has too
narrow a distribution of in-cloud IWC
• ECMWF lacks high IWC values, remedied in new model version
Cascade projectCascade project• How can we identify & cure errors in modelling African convection?• Unified Model simulations at a range of resolutions• Evaluate using A-Train retrievals• Also run “CloudSat simulator” to obtain radar reflectivity from model
Moist monsoon flow
African easterly jet Saharan air layer
Mid-level outflow
Parker et al. (QJRMS 2005)Location of African easterly jet
Cascade 40-km model versus Cascade 40-km model versus CloudSatCloudSat• Frequency of occurrence of reflectivity greater than –30 dBZ
• Plot versus “dynamic latitude” (latitude relative to location of AEJ)
• Anvil cirrus too low in model• Little sign of mid-level outflow
Unified Model CloudSat (~01.30 LT) CloudSat (~13.30 LT)
Thorwald Stein
Cascade 12-km model versus Cascade 12-km model versus CloudSatCloudSat
Unified Model CloudSat (~01.30 LT) CloudSat (~13.30 LT)
• Fairly similar behaviour to 40-km model• Larger diurnal cycle in anvil
Thorwald Stein
Cascade 4-km model versus Cascade 4-km model versus CloudSatCloudSat
Unified Model CloudSat (~01.30 LT) CloudSat (~13.30 LT)
• Note increase from 38 to 70 levels
• Anvil cirrus now at around the right altitude• Slightly more mid-level cloud• Large overestimate of stratocumulus (and too low)
Thorwald Stein
Structure of Southern Ocean Structure of Southern Ocean cirruscirrus
Observations- Note limitations
of each instrument
Retrievals
-5/3: Cloud-top turbulence &
upscale cascade
Fall-streaks & wind-shear remove smaller scales lower in cloud: steeper power spectra
Hogan and Kew (QJ 2005)
Outer scale 50-100 km
– Slice through Hogan & Kew 3D fractal cirrus model
– Southern Ocean cirrus is just like Chilbolton cirrus!
90 km
Tropical Tropical Indian Ocean Indian Ocean
cirruscirrus
Stratiform region in upper half of cloud?
Turbulent fall-streaks in lower half of cloud?
BurmaIndian Ocean
120 km
Stratiform upper region dominated by larger scales
Turbulent lower
region
600 km
– Sum of two fractal cirrus simulations
– Fall-streak paradigm unsuitable for cloud top
Thorwald performing spectral analysis on
Cascade model
Unified retrieval: Forward model
• From state vector x to forward modelled observations H(x)...
Ice & snow Liquid cloud Rain Aerosol
Ice/radar
Liquid/radar
Rain/radar
Ice/lidar
Liquid/lidar
Rain/lidar
Aerosol/lidar
Ice/radiometer
Liquid/radiometer
Rain/radiometer
Aerosol/radiometer
Radar scattering profile
Lidar scattering profile
Radiometer scattering profile
Lookup tables to obtain profiles of extinction, scattering & backscatter coefficients, asymmetry factor
Sum the contributions from each constituent
x
Radar forward modelled obs
Lidar forward modelled obs
Radiometer fwd modelled obs
H(x)Radiative transfer models
Adjoint of radar model (vector)
Adjoint of lidar model (vector)
Adjoint of radiometer model
Gradient of cost function (vector)
xJ=HTR-1[y–H(x)]
Vector-matrix multiplications: around the same cost as the original forward
operations
Adjoint of radiative transfer models
yJ=R-1[y–H(x)]
• First part of a forward model is the scattering and fall-speed model– Same methods typically used for all radiometer and lidar channels– Radar and Doppler model uses another set of methods
Scattering modelsScattering models
Particle type Radar (3.2 mm) Radar Doppler Thermal IR, Solar, UVAerosol Aerosol not
Graupel and hail Mie theory TBD Mie theoryMelting ice Wu & Wang
(1991)TBD Mie theory
Radiative transfer forward models
• Infrared radiances– Delanoe and Hogan (2008) model– Currently testing RTTOV (widely used, can do microwave, has
adjoint)• Solar radiances
– Currently testing LIDORT• Radar and lidar
– Simplest model is single scattering with attenuation: ’= exp(-2)– Problem from space is multiple scattering: contains extra
information on cloud properties (particularly optical depth) but no-one has previously been able to rigorously make use of data subject to pulse stretching
– Use combination of fast “Photon Variance-Covariance” method and “Time-Dependent Two-Stream” methods
– Adjoints for these models recently coded– Forward model for lidar depolarization is in progress
Examples of multiple scattering
LITE lidar (<r, footprint~1 km)
CloudSat radar (>r)
StratocumulusStratocumulus
Intense thunderstormIntense thunderstorm
Surface echoSurface echoApparent echo from below the surface
• Regime 0: No attenuation– Optical depth << 1
• Regime 1: Single scattering– Apparent backscatter ’ is easy to
calculate from at range r : ’(r) = (r) exp[-2(r)]
Scattering regimes
Footprint x
Mean free path l
• Regime 2: Small-angle multiple scattering
– Occurs when l ~ x– Only for wavelength much less than particle size, e.g. lidar & ice clouds
– Functioning algorithm framework exists– C++: object orientation allows code to be completely flexible:
observations can be added and removed without needing to keep track of indices to matrices, so same code can be applied to different observing systems
– Code to generate particle scattering libraries in NetCDF files– Adjoint of radar and lidar forward models with multiple scattering
and HSRL/Raman support– Interface to L-BFGS quasi-Newton algorithm in GNU Scientific
Library• In progress / future work:
– Implement full ice, liquid, aerosol and rain constituents– Estimate and report error in solution and averaging kernel – Interface to radiance models– Test on a range of ground-based, airborne and spaceborne
instruments, particularly the A-Train and EarthCARE satellites
Outlook• Use of radiances in retrieval should make retrieved profiles consistent with
broadband fluxes (can test this with A-Train and EarthCARE)• EarthCARE will take this a step further
– Use imager to construct 3D cloud field 10-20 km wide beneath satellite – Use 3D radiative transfer to test consistency with broadband radiances
looking at the cloud field in 3 directions (overcome earlier 3D problem)• How can we use these retrievals to improve weather forecasts?
– Assimilate cloud products, or radar and lidar observations directly?– Assimilation experiments being carried out by ECMWF– Still an open problem as to how to ensure clouds are assimilated such
that the dynamics and thermodynamics of the model are modified so as to be consistent with the presence of the cloud
• How can we use these retrievals to improve climate models?– We will have retrieved global cloud fields consistent with radiation– So can diagnose in detail not only what aspects of clouds are wrong in
models, but the radiative error associated with each error in the representation of clouds
Unified algorithm: state variables
State variable Representation with height / constraint A-priori
Ice clouds and snow
Visible extinction coefficient One variable per pixel with smoothness constraint None
Number conc. parameter Cubic spline basis functions with vertical correlation Temperature dependent
Lidar extinction-to-backscatter ratio Cubic spline basis functions 20 sr
Riming factor Likely a single value per profile 1
Liquid clouds
Liquid water content One variable per pixel but with gradient constraint None
Droplet number concentration One value per liquid layer Temperature dependent
Rain
Rain rate Cubic spline basis functions with flatness constraint None
Normalized number conc. Nw One value per profile Dependent on whether from melting ice or coallescence
Melting-layer thickness scaling factor One value per profile 1
Aerosols
Extinction coefficient One variable per pixel with smoothness constraint None
Lidar extinction-to-backscatter ratio One value per aerosol layer identified Climatological type depending on region
Ice clouds follows Delanoe & Hogan (2008); Snow & riming in convective clouds needs to be added
Liquid clouds currently being tackled
Basic rain to be added shortly; Full representation later
Basic aerosols to be added shortly; Full representation via collaboration?
• Proposed list of retrieved variables held in the state vector x
and 2nd derivative (the Hessian matrix):
Gradient Descent methods
– Fast adjoint method to calculate xJ means don’t need to calculate Jacobian
– Disadvantage: more iterations needed since we don’t know curvature of J(x)
– Quasi-Newton method to get the search direction (e.g. L-BFGS used by ECMWF): builds up an approximate inverse Hessian A for improved convergence
– Scales well for large x– Poorer estimate of the error at the
end
Minimizing the cost function
Gradient of cost function (a vector)
Gauss-Newton method
– Rapid convergence (instant for linear problems)
– Get solution error covariance “for free” at the end
– Levenberg-Marquardt is a small modification to ensure convergence
– Need the Jacobian matrix H of every forward model: can be expensive for larger problems as forward model may need to be rerun with each element of the state vector perturbed
112 BHRHxTJ
axBaxxyRxy 11
2
1)()(
2
1 TT HHJ
axBxyRHx 11 )(HJ T
JJii xxxx
12
1 Jii xAxx 1
Comparison of convergence Comparison of convergence ratesrates
• Solution is identical• Gauss-Newton method converges in < 10 iterations• L-BFGS Gradient Descent method converges in < 100 iterations• Conjugate Gradient method converges a little slower than L-BFGS• Each L-BFGS iteration >> 10x faster than each Gauss-Newton one!• Gauss-Newton method requires the Jacobian matrix, which must be
calculated by rerunning multiple scattering model multiple times
• Computational cost can scale with number of points describing vertical profile N; we can cope with an N2 dependence but not N3
Radiative transfer forward models
Radar/lidar model Applications Speed Jacobian Adjoint
Single scattering: ’= exp(-2) Radar & lidar, no multiple scattering N N2 N
Platt’s approximation ’= exp(-2) Lidar, ice only, crude multiple scattering N N2 N