TIGGE Data Archive at NCAR 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva 22-24 February, 2010 Doug Schuster Steven Worley Dave Stepaniak Others
Jan 12, 2016
TIGGE Data Archive at NCAR
8th GIFS-TIGGE Working GroupWorld Meteorological Organization
Geneva22-24 February, 2010
Doug SchusterSteven Worley
Dave StepaniakOthers
Archive Summary
Online DataPeriod, most recent two weeks~ 5 TB , public products~ 1 TB, data preparation, subsetting~ 65 GB Tropical Cyclone Track Data (CXML)
Offline DataFull period of record~ 300 TB, NCAR MSS system
Provider Summary
Archive Status Service Improvements Special Projects Metrics Future Service Additions
Data Delivery (02/2010)
• Data exchanges using UNIDATA LDM, HTTP and FTP
• 440 GB/day (ECMWF = 332 GB/day) • 1.7+ million 2-D gridded fields/day
Archive Inhomogeneities
EC
MW
F
UK
MO
JMA
NC
EP
CM
A
CM
C
BO
M
MF
KM
A
CP
TE
C
0
10
20
30
40
50
60
70
80# fields, # ensemble members
Conforming parame-ters
Ensemble Members
PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level
Variable LvL ECWF UKMO JMA NCEP CMA CMC BOM MetF KMA CPTC
Geopotential Z PL
Specific H PL
T PL
U-velocity PL
V-velocity PL
Potential Vor PT
Potential T PV
U-velocity PV
V-Velocity PV
U 10m SL
V 10m SL
CAPE SL
Conv. Inhib. SL
Land-sea SL
Mean SLP SL
Orog. SL
Skin T SL
Snow D. H20 SL
Snow F. H20 SL
Parameter vs Provider Matrix
Variable LvL ECWF UKMO JMA NCEP CMA CMC BOM MetF KMA CPTC
Soil Moist. SL
Soil T SL
Sunshine D. SL
Surf. DPT SL
Surf. ATmax SL
Surf. ATmin SL
Surf. AT SL
Surf. P SL
LW Rad. Out SL
LH flux SL
Net Rad SL
Net Therm. Rad SL
Sensible Rad. SL
Cloud Cov SL
Column Water SL
Precipitation SL
Wilt. Point SL
Field Cap. SL
PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level
Parameter vs Provider Matrix
2009 Service Improvements Improved Web Portal Performance
User data selection interface runs smoother.Automated capability to stage offline data for
download.Added “Client for URLs” (cURL) based
parameter subsetting capability.Directly extract selected 2D grids from files across
multiple data providers.Downloads managed from user’s workstation.Script templates are provided to drive “subscription
style” parameter download capability.Strengthened data ingest processing.
System monitoring (disk capacity, LDM status, etc)Failover mechanisms for LDM downtime.Data integrity checking.
2009/2010 Supported Special Projects
NCAR Research Applications Laboratory (RAL) Google African Meningitis Forecast InitiativeDr. Thomas Hopson Integrate weather and health data to manage
African meningitis outbreaks (in Ghana as starting point).
Looking for 2-week lead time to position vaccines.Processing ECMWF data from present
backwards through the archive for model calibration.Total Precipitation, 2m Temperature, 2m Dew Point.Global Subarea 40 deg N to 40 Deg S.0.5 x 0.5 degree resolution.Assist RAL and Unidata in providing “Real-time”
access to TIGGE based Multi-Model Ensemble Forecast data.
http://www.ral.ucar.edu/projects/ghana/http://www.google.org/predict.html
NCAR TIGGE Full Model Data Archive Usage
YEAR GB AccessedGB
DownloadedFiles
DownloadedActive Users / Unique Users
2006 5 5 10 0 / 5
2007 900 900 32,000 10 / 50
2008 1,300 1,100 40,000 10 / 20
2009 13,700 1,900 68,000 10 / 30
2010 80 20 500 3 / 5
Total 15,985 3,925 140,510 33 / 80
2006 2007 2008 20090
2000400060008000
10000120001400016000
NCAR TIGGE Data Access Metrics
TB AccessedTB Downloaded
Year
TB cURL
impact
NCAR TIGGE CXML Archive Usage
YEARGB
DownloadedFiles
Downloaded# Active Users /# Unique Users
2008 1 860 7 / 30
2009 26 12,300 17 / 35
2010 0.5 500 3 / 6
Total 27.5 13,660 27 / 81
Planned Service ImprovementsAdditional Online Storage
45 TB data online (3 months of most recent data).15 TB for data processing request output.Interface driven data processing and immediate
download capability extended from 2 weeks to 3 months.
Improved “Offline” Storage.Data being transitioned from current NCAR MSS
system to new HPSS system.Improved access speeds to offline data.Add data processing capability to offline data?
Expanded/Updated Computing.Improved LDM ingest reliability.Faster “turn-around” time on user sub-set requests.Potential for subscription services?
Potential Service Improvements
Model Validation Data Portal (Pending Funding)? Wide variety of observational and analyzed data
sources. In Situ Observation data (NCEP operationally collected data). Ocean based observations (ICOADS). Global tropical cyclone position and intensity data. Atmospheric Reanalysis (JRA, ERA, NNRR, CFSR). Global PCP datasets. Suggested TWG sets to support FDP’s. Will need discussions to set priorities for development.
Data sub-set and output format selection capability. Integrated With TIGGE model data portal
Data analysis tools and codes. Model Evaluation Tools offered by the Developmental
Testbed Center at NCAR
http://www.dtcenter.org/mrt/users
http://tigge.ucar.edu/