Science Processing Status of the Earth Observing System Microwave Limb Sounder (EOS MLS) Elmain Martinez September 27, 2010 Aura Science Meeting - Data System Working Group Boulder, Colorado Jet Propulsion Laboratory California Institute of Technology Copyright 2010 California Institute of Technology. Government sponsorship acknowledged.
19
Embed
Science Processing Status of the Earth Observing System Microwave Limb Sounder (EOS MLS) Elmain Martinez September 27, 2010 Aura Science Meeting - Data.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Science Processing Status of the
Earth Observing System Microwave Limb Sounder
(EOS MLS)
Elmain MartinezSeptember 27, 2010
Aura Science Meeting - Data System Working GroupBoulder, Colorado
Jet Propulsion LaboratoryCalifornia Institute of Technology
Copyright 2010 California Institute of Technology. Government sponsorship acknowledged.
MLS L0 – PDSAura SC Engr.Aura SC Definitive OrbitAura SC AttitudeOperational Meteorological DataLeap Second & UTC Pole Data
DATA PROCESSING STATUS
27 September 2010 EOS MLS 4
MLS Near-Real-Time Status
• SIPS NRT configuration is continuing to deliver Ozone and Temperature L2 products using V2.23– Ingests pre-processed NRT MLS PDS data from the GES DISC
• 15 minute granules – L1 and L2 PGS software was modified in 2008 to improve product
delivery timeliness– L2 products are delivered to SCF and GES-DAAC; available ~3 hours
after observation• Customers include GMAO, NOAA/NCEP, NOAA/NESDIS/STAR,
Environment Canada, ECMWF, and Met Office
27 September 2010 5EOS MLS
Forward Processing Status
• SIPS is processing all incoming data with V2.23– L1/2 products delivered to SCF and GES-DAAC
• V2.23 in operation since March 2008– This is the 4th minor versions of V2
• v02.20 – initially released Nov 2006• v02.21 – de-convolve DACS radiances in Level 1• v02.22 – increased performance without change of quality • v02.23 – dealt with data variances produced by Aura anomaly
– (data since Dec 4, 2008)
• We will transition from V2.23 to V3.3 once the Data Quality Document is released
27 September 2010 6EOS MLS
Reprocessing Status
• V3.10 reprocessing halted in Nov 2009 due to data quality concerns– Products delivered to GES DISC, but never publicly released; now removed
• SIPS is currently reprocessing all science data with V3.30– Reprocessing began in April 2010
• More than halfway through the 72 months (2273 days) to be reprocessed– V3.30 products being delivered to GES DISC; to be publicly released in Oct.
27 September 2010 EOS MLS 7
Day
s P
roce
ssed
SOFTWARE STATUS
27 September 2010 EOS MLS 8
Major Improvements in V3.3
• Software changes:– Processing with GEOS5 meteorology for dates after Sept 1, 2009
• MERRA data for dates prior
– Using HDF5 1.8.3 and SDP Toolkit 5.2.16– Products are augmented to be HDF5-NetCDF compliant– Software built with 64-bit compiler– Additional error handling added to scheduler– Products to be released to public once the Quality Document is finalized– See the poster “Version 3.30 Products from EOS MLS” for further details
• Science improvements– CH3CL (Methyl Chloride) is a new standard product
• See the Validation and Climatology poster on this species
– Revised spectroscopy to reflect best-fit to unknown continuum– Special handling of Core+R5 phase if THz is turned off
27 September 2010 EOS MLS 9
Version 4 Plans• Product Generation Software
– Target release date is Spring 2012– Improvements currently planned:
• Read various parameters using HDF5, e.g. temperature • Retrieve species in clouds• Flag radiances to not be used when clouds are detected• Generating a new product, CH3OH
– Intended to account for radiances seen in other species
– Integrate latest available version of HDF5 and SDP Toolkit
• NRT improvements:– XML-formatted metadata– Browse product for choosing granules to download– Capability for Data Casting (RSS)– Better forward model such as 2nd derivative
27 September 2010 EOS MLS 10
PROCESSING ENVIRONMENT IMPROVEMENTS
27 September 2010 EOS MLS 11
SIPS Processing Environment Improvements
• Replaced old IBM 32 bit production cluster with a Dell 64 bit cluster in June, 2010– Specs
• 48 nodes• 8 CPUs per node• 146 GB storage per node• 2.93 GHz• 24 GB memory (vs 16GB on older clusters)
• Benchmark tests for a single day completed 3 hours faster compared to an older cluster; ~27% improvement in speed– About 6.34 additional days per week with the 27% performance
improvement– With the additional cluster and its higher relative performance,
weekly output has increased by apx. 22 additional days per week
27 September 2010 EOS MLS 12
SIPS HW/Network Architecture
27 September 2010 EOS MLS 13
sips-switch (LAN)
Operations Dev/Test NRT
DB Server
sftp Server
SIPS Server
L1B (V2.23)
L2 (V2.23)
L2 (V3.30)
L2 (V3.30)
sftp Server
NRT Server (2)
JPL router
JPL net
L2 (V3.30)
Changes since 9-09:• Operations: - Replaced aging L2 Ops cluster - Replaced aging L1 server• Development - Replaced aging NFS/L1 server
L1B (V2.23)
DB Server
File Server/L1B (V3.30)
Dev/Test (2)
L2 (V3.30)
L1B (3, V30)
Replaced
SIPS Future Plans
• No upgrades planned for next FY• FY12 and beyond we will be looking at:
– Tropical Cyclone Information System (TCIS) brings together satellite and in situ data sets from various sources (e.g. MLS, OMI, QuikSCAT, and others) so users can find information for a particular tropical cyclone.
• Site up since April 2010• 2005 data loaded, other years to follow• Site has an analysis tool that allows users to plot histograms, maps, and profiles for
many different data sets and products.• Uses MLS data products for temp, ozone, CO2, water vapor
• MLS webpage improvements, e.g. NRT Viewer– See the poster “The Microwave Limb Sounder Website: Improved Access to Data and
Information” for more information
27 September 2010 EOS MLS 17
SUMMARY
27 September 2010 EOS MLS 18
Recap of Future Plans• V4 is about 19 months away
– Algorithm cycle is usually 18 months or more, but it could be sooner if Level 2 algorithms can mature
• SIPS hardware is stable and should be for a couple years• SCF to make another leap to keep up…to purchase a 64 bit
development machine and additional storage in the future