Data Discovery and Access to The International Surface Pressure Databank (ISPD)

1

Data Discovery and Access to The International Surface Pressure

Databank (ISPD)

Thomas Cram Gilbert P. Compo*Doug Schuster Chesley McColl*Steven Worley

National Center for Atmospheric Research, Boulder, CO*NOAA/CIRES, Boulder, CO

AGU 2012 Fall Meeting: IN44A-05

2

Research Data Archive (RDA) at NCARrda.ucar.edu

1. 600+ distinct datasets for climate and weather research

2. Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs

3. Free and open accesshttp://rda.ucar.edu


3

ISPD Overview• World’s largest collection of surface & sea level pressure observations

Land station Marine observations Tropical cyclone best track

• Period (version 2): 1768 – 2010• Volume: 465 Gbyte• Available since Aug 2010


4

ISPD Overview• 60+ Contributors

o Atmospheric Circulation Reconstructions over the Earth (ACRE)o Australian Bureau of Meteorologyo British Antarctic Surveyo Cook Islands Meteorological Serviceo Danish Meteorological Instituteo Deutscher Wetterdienst (DWD; German Weather Service)o European and North Atlantic Daily to Multidecadal Climate Variability

(EMULATE)o ETH Zurich, Switzerlando GCOS/WCRP Working Group on Observational Data Sets for

Reanalysiso MANY MANY MORE….

• Assembled by NOAA/ESRL, CIRES (Univ. of Colorado), & NOAA/NCDC


5

20th Century Reanalysis• Global reanalysis of atmospheric

circulation• Period: 1869 – 2010• Assimilates ISPDv2 as input• Compo et al. (2011) QJRMS


6

20th Century Reanalysis:Oct 1950 mean 1000 hPa temperature

ISPD stations ISPD marine obs


7

ISPD stations ISPD marine obs


20th Century Reanalysis:Nov 1960 mean 1000 hPa temperature

8

• ISPD obs assimilated into 20CR• 20CR data quality control feedback

contained in ISPD Provides estimated uncertainty in obs Helps improve underlying

observational database


ISPD & the 20th Century Reanalysis

9

ISPD Sample Annual Station Distribution

1850 * Land stations only* No marine stations


10

18501900

* Land stations only* No marine stations



11

18501900

1950* Land stations only* No marine stations



12

18501900

19502000

* Land stations only* No marine stations



13

ISPD Observations/Year

Figure courtesy Chesley McColl, NOAA/ESRL

2010: 53 Million

~ 1.5 Billion total observations


14

Data Access: Problem Background• Large computational/storage resources

needed– Store data– Extract desired data from large grids/files– Convert data to desirable format(s)

Scientific data centers have these resources

Individual researchers generally don’t


15

• Goals– Make data more accessible and easier to use for

individual researchers• Reasonable access volumes• Desired data formats• User defined parameters/grids

• Researchers stay focused on research


Data Access: Problem Background

16

ISPD Data Access Services

• Powerful computing resources @ NCAR

• Large disk storage (~ 0.5 PB)• Rich and detailed metadata• Direct file download via web• Customized data sub-setting• HDF-5 to ASCII software tools


17

ISPD Metadata Features• Both group- and file-level metadata• Drive interfaces for file grouping and sub-

setting tools• Support efficient back-end processing• Improve scalability• Provide “quick look” at data samples


18

ISPD Metadata Interface Example


19AGU 2012 Fall Meeting: IN44A-05



22

Data Access: ISPD Subset Interface


23

ISPD Data Access ServicesTemporal range sub-

setting (daily)

Spatial sub-setting Lat/Lon region Individual station ID

24


Data sub-setting options (cont.)• Observation type

Land station Marine obs Radiosonde Dropsonde TC best track

25


• Subsetting processed in delayed mode• E-mail notification• Download via server-provided scripts

(wget)

26

ISPD 2012 Subset metrics

Data accessed: ~ 6.5 TB

Data served: ~ 46 GB

27

Summary & Future Directions

• RDA – Supply “User Friendly” Data Parameter & spatial sub-setting Metadata discovery Format conversion Improved and additional services

• NWSC-Cheyenne opening – more computing power


28

• DOI assignment

• Geoscience Data Journal article

• ISPD v3 (1755-2010) Spring 2013


ISPD Forthcoming

29

rda.ucar.edu/[email protected]


Data Discovery and Access to The International Surface Pressure Databank (ISPD)

Documents

fall meeting

goalsmake data

ispd overview4

marine stationsagu

land stations

observational data sets

large gridsfilesconvert

ncar data discovery