From Launch Pad to Laptop:  Accessing and Using Data from NASA's Earth Observing System

1

From Launch Pad to Laptop:  Accessing and Using Data from NASA's Earth Observing System

Forrest MeltonCSU Monterey Bay, Seaside, CA

Ecological Forecasting LabNASA Ames Research Center, Moffett Field, CA

With contributions from:Rama Nemani, Petr Votava, Andrew Michaelis, Christina Milesi, Hirofumi Hashimoto, Weile Wang

Support provided by: NASA Applied Sciences Program

Harvard Remote Sensing Technology & Applications Workshop Cambridge, MA, Feb. 16, 2007

Terra Launch, Dec. 18, 1999

2

Outline

- NASA Earth Observing System (EOS)

- Satellites

- Instruments

- Applications

- EOS Data Information System

- Accessing data

- Data formats

- Viewing and utilizing data

- Future NASA satellite missions

- Integrating satellite data with in-situ observations and ecosystem models

- Terrestrial Observation & Prediction System

Northeastern U.S., January 25, 2005Credit: MODIS Rapid Response Team

3

A Skyful of Satellites

Union of Concerned Scientists, Satellite Database http://www.ucsusa.org/global_security/space_weapons/whats-in-space.html

Credit: Union of Concerned ScientistsCredit: New York Times, 2/6/2007

3,100 orbiting satellitesOver 845 844 active satellites

4

Earth Observing Satellites

NASA

NOAA

USGS

Brazilian Space Agency (INPE)

Canadian Space Agency

Centre National d'Etudes spatiales (CNES, France)

Chinese National Space Admin.

European Space Agency (ESA)

Japan Aerospace Exploration Agency (JAXA)

Indian Remote Sensing Agency

Commercial satellites and many others

5

NASA’s Earth Observing System (EOS)

EOS is collecting Earth remote sensing data for a 15 year globalchange research program

Phase II began in December 1999 with the launch of the first EOS satellite, Terra

Integrated Earth observations to advance understanding of Earth System

http://eospso.gsfc.nasa.gov/

6

Multiple Instruments per Mission

Terra SatelliteLaunched Dec. 18, 1999 with five instruments (ASTER, CERES, MISR, MODIS, MOPITT)

Aqua SatelliteLaunched May 4, 2002 with six instruments (AIRS, AMSR-E, AMSU, CERES, HSB, MODIS)

MODerate resolution Imaging Spectroradiometer

Orbit: 705 km, 10:30 a.m. descending node (Terra) or 1:30 p.m. ascending node (Aqua), sun-synchronous, near-polar, circular

Swath Dimensions: 2330 km (cross track) by 10 km (along track at nadir)

Data Rate:10.6 Mbps (peak daytime); 6.1 Mbps (orbital average)

Spatial Resolution: 250 m (bands 1-2), 500 m (bands 3-7), 1000 m (bands 8-36)

Design Life: 6 years

Example: MODIS on Terra & Aqua

7

EOS / Sensor Web Overview

UAV/AircraftUAV/Aircraft

NASA SatellitesNASA Satellites

EOSDIS / EOSDIS / DAACsDAACs

Fire / EventFire / Event

Ground SensorsGround Sensors

Ancillary DataAncillary Data

EOS UsersEOS Users

8

EOS Science Focus Areas

9

EOS Atmospheric Measurements

http://www-calipso.larc.nasa.gov/about/atrain.php

Mission Aura PARASOL CALIPSO Cloudsat Aqua OCO

Instruments HIRDLSMLSOMITES

POLDER CALIOPIIRWFC

Cloud profiling radar

AIRS/AMSU-A/HSBAMSR-RCERESMODIS

3 grating spectrometers

Primary Observations

Atmospheric pollutants and greenhouse gases

Aerosols and clouds; distinguish natural vs anthropogenic aerosols

Aerosols and clouds

Cloud properties

Atmospheric water vapor

Carbon dioxide concentrations

A-Train Satellite Constellation

Collects nearly simultaneousmeasurements of:

• aerosols

• clouds

• temperature

• relative humidity

• radiative fluxes

Credit: Langley Research Center

10

EOS Atmospheric Measurements

Measurement SensorsCloud Properties (amount, optical properties, height)

MODIS, GLAS, AMSR-E, MISR, AIRS, ASTER, SAGE III

Radiative Energy Fluxes (top of atmosphere, surface)

CERES, ACRIM III, MODIS, AMSR-E, GLAS, MISR, AIRS, ASTER, SAGE III

Precipitation TMI/VIRS, AMSR-E

Tropospheric Chemistry (ozone, precursor gases)

TES, MOPITT, SAGE III, MLS, HIRDLS, LIS

Stratospheric Chemistry (ozone, ClO, BrO, OH, trace gases)

MLS, HIRDLS, SAGE III, OMI, TES

Aerosol Properties (stratospheric, tropospheric)

SAGE III, HIRDLS, MODIS, MISR, OMI, GLAS

Atmospheric Temperature AIRS/AMSU-A, MLS, HIRDLS, TES, MODIS

Atmospheric Humidity AIRS/AMSU-A/HSB, MLS, SAGE III, HIRDLS,

Poseidon 2/JMR/DORIS, MODIS, TES

Lightning (events, area, flash structure) LIS

EOS Project Science Officehttp://eospso.gsfc.nasa.gov/eos_homepage/for_scientists/index.php

11

EOS Atmospheric Measurements: Aerosol monitoring

Credit: Jeff Schmaltz, NASA GSFC

Credit: NASA

Credit: K. Mubenga

MODIS Rapid ResponseTOMS

MODIS MOD04 Aerosol Optical Depth

12

EOS Cryosphere / Solar Radiation Measurements

Measurement SensorsCryosphere

Land Ice (ice sheet topography, ice sheet volume change, glacier change)

GLAS, ASTER, ETM+

Sea Ice (extent, concentration, motion, temperature)

AMSR-E, MODIS, ASTER, Jason-1 (Poseidon 2/JMR/DORIS), ETM+

Snow Cover (extent, water equivalent) MODIS, AMSR-E, ASTER, ETM+

Solar Radiation

Total Solar Irradiance ACRIM III, TIM

Solar Spectral Irradiance SIM, SOLSTICE

Terra / Aqua ICESat JASON-I LANDSAT 7

ACRIMSAT

SORCE

13

EOS Cryosphere: Monitoring Ice Sheets and Glaciers

Larsen B Ice Shelf Collapse, 2002Credit: NSIDC

GLAS onboard ICESat will extend NASA airborne instrument record to track changes in ice sheet extent and thickness.

MODIS, ETM+, and ASTER can be used to monitor extent of glaciers and ice sheets.

Jan. 31, 2002

Mar. 5, 2002

14

EOS Land and Ocean Measurements

Measurement Sensors

Land Cover & Land Use Change ETM+, MODIS, ASTER, MISR

Vegetation Dynamics MODIS, MISR, ETM+, ASTER

Land Surface Temperature ASTER, MODIS, AIRS, AMSR-E, ETM+

Fire Occurrence (extent, thermal anomalies) MODIS, ASTER, ETM+

Volcanic Effects (frequency of occurrence, thermal anomalies, impact)

MODIS, ASTER, ETM+, MISR

Surface Wetness AMSR-E

Ocean Surface Temperature MODIS, AIRS, AMSR-E

Phytoplankton & Dissolved Organic Matter MODIS

Surface Wind Fields SeaWinds, AMSR-E, Poseidon 2/JMR/DORIS

Ocean Surface Topography (height, waves, sea level)

Poseidon 2/JMR/DORIS

15

EOS Measurements: Fire & Vegetation Monitoring

MODIS MOD14 Southeast Asia Thermal Anomalies, January 28, 2007

Credit: MODIS Rapid Response

Tracking vegetation condition with MODIS EVI in the Amazon Basin

Credit: Huete et al. 2006. GRL 33.

16

EOS Measurements: Tracking Landscape Change in Mato Grasso

ASTERCredit: Morton et al. 2006. PNAS, 103(39).

MODISCredit: MODIS Rapid Response Team

17

NASA Sub-orbital Science Program

A range of sensors, including:• AVIRIS (Airborne Visible Infrared

Imaging Spectrometer)• AIRDAS (Airborne Infrared

Disaster Assessment System)• HyMap• LIDAR and many others

http://suborbital.nasa.gov/index.html

AVARIS Data Cube

15 aircraft platforms supporting satellite missions, field campaigns, disaster response, and science missions

NASA AVIRIS, Dec. 2005 Mosaic, Buck Island, St. Croix, USVI Credit: Dr. Liane Guild, NASA ARC

18

EOS Data Information System (EOSDIS)

• EOSDIS is the comprehensive ground system for processing, archiving, and distributing data from all the EOS spacecraft

• Data processed, archived and distributed from nine distributed active archive centers, each with its own focus

• Accepts, process, and archives 3+ TB per day of data from EOS

• > 4 petabytes of data archived

• 2.3 TB of user data requests per day fulfilled

• Also provides mission operation systems that perform command and control of the spacecraft and instruments, health and safety monitoring, mission planning and scheduling, initial data capture, and Level 0 processing.

http://nasadaacs.eos.nasa.gov/about.htmlhttp://eospso.gsfc.nasa.gov/eos_homepage/data_services.php

19

EOS Standard Data Products: Processing Levels

Data Processing Level Description

Level 0 Reconstructed, unprocessed instrument and payload data at full resolution, with any and all communications artifacts removed.

Level 1A Reconstructed, unprocessed instrument data at full resolution, time-referenced, and annotated with ancillary information, including radiometric and geometric calibration coefficients and georeferencing parameters (e.g., platform ephemeris) computed and appended but not applied to the Level 0 data.

Level 1B Level 1A data that have been processed to sensor units (not all instruments have Level 1B data).

Level 2 Derived geophysical variables at the same resolution and location as Level 1 source data.

Level 3 Variables mapped on uniform space-time grid scales, usually with some completeness and consistency.

Level 4 Model output or results from analyses of lower level data (e.g., variables derived from multiple measurements).

MODIS Level 0 through Level 4 data products

21

Data Access: EOS Data Gateway

• EOS data access provided by EOS Data Gateway (EDG) http://delenn.gsfc.nasa.gov/~imswww/pub/imswelcome/ (http://eos.nasa.gov/imswelcome)

• Integrated search interface for all EOS data products

• Facilitates search by data set, sensor, data type, location, and date

• On-line manual and tutorial provided

• NOAA Data Access– CLASS,

http://www.class.ncdc.noaa.gov/nsaa/products/welcome

• USGS Data Access– Earth Resources Observation

Systems (EROS) Data Center (EDC), http://edc.usgs.gov/

• Access to some datasets requires NASA / NOAA / USGS collaboration

EDG Data Query Interface

22

EDG Data Listing

23

EOS Standard Products: Data Format

HDF (Hierarchical Data Format)

• Designed at NCSA to facilitate sharing of scientific data

• Embedded metadata for units, labels, other descriptors

• Platform independent

• User extendible

• Supports multidimensional array, text, table, raster image, and palette

HDF-EOS (HDF for the Earth Observing System

• Implemented as a C library extension of the standard HDF library

• Adds support for grid, point, and swath formats to standard HDF objects

• Allow file contents to be reference to Earth coordinates and time

• Contains EOS Core System (ECS) core metadata essential for ECS search services (if the product contains ECS metadata, it is a valid HDF-EOS file)

• Can be read by any tool that processes standard HDF files

http://science.hq.nasa.gov/research/earth_science_formats.htmlhttp://hdfeos.org/; http://earthobservatory.nasa.gov/Study/HDFEOS/

24

Other Common Data Formats

GeoTIFF, http://remotesensing.org/geotiff/spec/geotiffhome.html• Public domain metadata standard that allows georeferencing information to be embedded within a TIFF• The potential additional information includes projections, coordinate systems, ellipsoids, datums, and

everything else necessary to establish the exact spatial reference for the file.

Network Common Data Form (netCDF), http://www.unidata.ucar.edu/software/netcdf/• Interface for array-oriented data access and a freely distributed collection of software libraries for C,

FORTRAN, C++, Java, and Perl that provide implementations of the interface• Many of the same features and advantages of HDF: (1) self-describing; (2) architecture-independent; (3)

directly readable• Widely used for climate data, climate modeling, and other applications with multidimensional gridded

datasets

National Land Archive Processing System (NLAPS) Data Format (NDF), and Level 1 Product Generation System (LPGS), http://eros.usgs.gov/products/satellite/landsat7.html#processing

• NLAPS and LPGS and Landsat data processing systems; image data are in GeoTIFF

Flexible Image Transport System (FITS), http://fits.gsfc.nasa.gov/  • Used by astronomy research community (and the TRMM VIRS fire product)• Header records in ASCII text format, followed by data records, consisting of a byte stream.

National Imagery Transmission Format (NITF) 2.1 / 2.0• Used by Intelligence and Defense community (and Quickbird)

Binary (flt / hdr) and American Standard Code for Information Interchange (ASCII)

http://science.hq.nasa.gov/research/earth_science_formats.htmlhttp://outreach.eos.nasa.gov/EOSDIS_CD-03/docs/data_formats.htm

25

Other U.S. Sources of Satellite Data: NOAA & DMSP

NOAA National Environmental Satellite, Data, and Information Service (NESDIS)

• NESDIS, GOES, POES, METEOSAT, AVHRR

• Data access via the NOAA Comprehensive Large Array-data Stewardship System (CLASS) http://www.osd.noaa.gov/class/

• Data formats: GeoTIFF, HDF, Binary, shapefile

• Also provides access to the Defense Meteorological Satellite Program (DMSP) Archive

NESDIS data organized by four data centers:

• Climate Data Center

• Geophysical Data Center

• Ocean Data Center

• Coastal Data Center

http://www.nesdis.noaa.gov/ http://www.noaa.gov/satellites.html

26

Other U.S. Sources of Satellite Data: USGS

USGS maintains data for Landsat 1-7 missions and archive of aerial photography

Data Access via EDC • http://landsat.usgs.gov/

data_products/ordering_data.php

• Earth Explorer• GloVis• IGS Network

Landsat 7 SLC failure on May 31,2003 continues to be problematic

Progression of deforestation in Bolivia from 1975 to 2000. The MSS image on the left was acquired on June 17, 1975; the middle TM image on July 10, 1992; and the right ETM+ image on August 1, 2000. Credit: USGS

Agriculture in Nevada. ScanLine Corrector anomaly creates a zigzag pattern along satellite ground track. Image acquired August 28, 2003. Credit: USGS

http://landsat.usgs.gov/project_facts/files/landsat_fact_sheet_20023-03.pdf

27

Viewing and Analyzing Data

Multiple commercial tools for viewing and analyzing commercial data:

Application Description

RSI ENVI/IDL Geospatial Image Processing http://www.rsinc.com/

ERDAS Imagine Geospatial Image Processing http://gis.leica-geosystems.com/

PCI Geomatics(eCognition)

Geospatial Image Processing http://www.pcigeomatics.com/

Idrisi Geospatial Image Processing http://www.clarklabs.org/

ERMapper Geospatial Image Processing http://www.ermapper.com/

Matlab Technical Computing & Image Processing

http://www.mathworks.com/

HDF Explorer Pro Data Visualization http://www.space-research.org/

http://edcdaac.usgs.gov/landdaac/tools/hdf/tools_table.asp

28

Viewing and Analyzing Data

Open Source Remote Sensing Applications

• RemoteSensing.org, http://remotesensing.org/Home.html

Application Description

MultiSpec Multispectral Image Data Analysis http://dynamo.ecn.purdue.edu/~biehl/MultiSpec/

MSPHINX Satellite Process Handling Images uNder Xwindow

http://www-loa.univ-lille1.fr/Msphinx/Msphinx_gb.html

MODIS Reprojection Tool

Projection/Format Conversion http://lpdaac.usgs.gov/landdaac/tools/modis/index.asp

MRT – Swath Swath-to-Grid Conversion http://lpdaac.usgs.gov/landdaac/tools/mrtswath/index.asp

LDOPE Tools MODLAND Quality Assessment http://lpdaac.usgs.gov/landdaac/tools/ldope/index.asp

MODIS Swath-to-Grid IDL-Based Conversion http://nsidc.org/data/modis/ms2gt/

Cube Visualization (CV)

Data Visualization http://isis.astrogeology.usgs.gov/Isis2/isis-bin/isis.cgi/

WebWinds Data Visualization/Processing http://www.openchannelsoftware.com/projects/WebWinds/

WinVicar Windows-Based Image Processing http://www.openchannelsoftware.com/projects/WINVICAR/

HEW Web-based HDF-EOS Subsetter http://subset.itsc.uah.edu/hew2k/

NCSA HDF Tools HDF Utilities http://hdf.ncsa.uiuc.edu/hdftools.html

http://edcdaac.usgs.gov/landdaac/tools/hdf/tools_table.asp

29

Viewing and Analyzing Data: HDF Tools

HDF-EOS Tools & Libraries, http://www.hdfeos.org/software.php • Provide functionality for viewing, analyzing, and converting HDF to other formats

Examples

Binary_Dumper -- dumps HDF-EOS objects in binary format

EOSVIEW -- displays and verifies HDF and HDF-EOS files

HDFExplorer -- visualization program that reads HDF, HDF-EOS and HDF5 files

HDFView -- a visual tool for browsing and editing NCSA HDF4 and HDF5 files

HE5View -- a file viewing tool for examining and verifying HDF-EOS 5.x files

HEG -- converts HDF-EOS to GeoTIFF

HEMU -- a tool to update metadata inside an HDF-EOS file

HMR -- reads metadata from HDF-EOS file and writes to a text file

HMU -- updates the metadata embedded in an HDF-EOS file

MS2GT -- reads HDF-EOS files containing MODIS swath data and produces flat binary files containing gridded data

PGS_Toolkit -- a set of tools to manage the metadata that are generated with each EOS product

PHDIS -- reads any HDF-EOS file containing data gridded in the Lambert Azimuthal Equal Area projection

hdf2bin -- converts HDF or HDFEOS file into plain binary file

hdfeos-netcdf -- converts HDF-EOS4/5 files to netCDF

view_hdf -- a visualizationtool for accessing data stored in HDF and HDF-EOS files

30

Current NASA Satellite Missions

31

Future NASA Satellite Missions

32

Future NASA Satellite Missions: 2008

Glory

December 2008

Continue SORCE Mission. Determine atmospheric aerosol properties from the polarization ofbackscattered solar radiation, and measure total solar irradiance.

Ocean Surface Topography Mission (OSTM)

May 2008

Continue Jason and TOPEX/Poseidon Missions. Determine ocean surface topography to study ocean circulation and its environmental applications (cooperative with France, EUMETSAT, and NOAA).

Orbiting Carbon Observatory (ESSP / OCO)

September 2008

Provide space-based observations of atmospheric carbon dioxide (CO2).

Earth System Science Pathfinder (ESSP) Program

ESSP missions address unique, specific, highly-focused requirements in Earth science research. Primarily low to moderate cost, small to medium sized missions that are capable of being built, tested and launched in a short time interval.

33

Future NASA Satellite Missions: 2009-2010

Global Precipitation Measurement (GPM) Mission

TBD

Measure global precipitation with improved temporal resolution and spatial coverage .

Landsat Data Continuity Mission

TBD

Extend the Landsat record of multispectral, 30-m resolution, seasonal,global coverage of Earth’s land surface (joint with USGS).

NPOESS Preparatory Project (NPP) Mission

September 2009

Extend key measurements in support of long-term monitoring of climate trends and global biological activity (joint NOAA-DoD-NASA mission). NPP extends the measurement series being initiated with EOS Terra and Aqua by providing a bridge between NASA’s EOS missions and NPOESS, scheduled to replace the separate NOAA and DoDoperational systems in ~2010.

ESSP Aquarius

March 2009

Measure global sea surface salinity (cooperative mission withArgentina).

LDCM

34

NASA Applied Sciences

NASA Applied Sciences Application Plan, http://science.hq.nasa.gov/strategy/AppPlan.pdf

35

Monitoring

Modeling

Forecasting

Multiple scales

Nemani et al., 2003, EOM White & Nemani, 2004, CJRS

Applying EOS Data: TOPS

Predictions are based onchanges in biogeochemicalcycles

Developing a Common Modeling Framework

Use of ecosystem models to integrate satellite observations, climate data, and other ground-based observations and produce ecological forecasts.

http://ecocast.arc.nasa.gov/

36

TOPS California Ecological Daily Nowcasts at 1km

Biome-BGCSimulation models

Outputs include plant growth, irrigation demand, streamflowSalt water incursion, water allocation, crop coefficients

T P

RAD

Climate + Satellite Carbon and water cycles

ET

0 2.5 5

GPP

GPP (gC/m2/d) ET (mm/d)

37

Near Real-time Monitoring of global NPP anomalies

Running et al., 2004, Bioscience, 54:547-560

Mapping changes in global net primary productionnear real-time depiction of the droughts in the Amazon and Horn of Africa, May 2005

38

Lots of Data, One Laptop

Data issues to consider:

Spatial scale

Temporal scale

Cost

Availability

Data quality

Length of data record

Resources available for data analysis

39

Resources

NASA Earth Science Reference Handbook http://eospso.gsfc.nasa.gov/eos_homepage/for_scientists/data_products/refbook2006.php

EOS Data Products Handbook Vol. 1 (2004) and Vol. 2 (2000) http://eospso.gsfc.nasa.gov/eos_homepage/for_scientists/data_products/vol1.php

Finding DataEDG: http://delenn.gsfc.nasa.gov/~imswww/pub/imswelcome/DAACs: http://nasadaacs.eos.nasa.gov/search.htmlEOSDIS: http://spsosun.gsfc.nasa.gov/eosinfo/Welcome/index.html EOS-WEBSTER: http://eos-webster.sr.unh.edu/about.jspEOS Imagery http://visibleearth.nasa.gov/

EOS MissionsMission Overviews: http://science.hq.nasa.gov/missions/earth.htmlMissions Profiles: http://eospso.gsfc.nasa.gov/eos_homepage/mission_profiles/index.phpInstruments: http://eospso.gsfc.nasa.gov/directory/instrument/index.php

Further ResourcesNASA Remote Sensing Tutorial: http://rst.gsfc.nasa.gov/GIS Cafe Forums: http://www10.giscafe.com/wwwthreads-5.3/wwwthreads.phpInstrument Science Teams

40