Stocker - Page 1Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Evolution of a Measurement-Based Data Processing System for Precipitation Erich.

Stocker - Page 1 Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006

Evolution of a Measurement-Based Data Processing System for Precipitation

Erich Franz Stocker NASA/GSFC Code 610.2Mary Cleave NASA Headquarters


Topics

• Determining measurements

• Architectural characteristics of a measurement-based system

• Evolving TRMM mission data processing system to measurement based Precipitation Processing System (PPS)

• Architecture-based process in use

• Summary


Establish of a Measurement

• NASA Earth Science is evolving– Measurement-based programs for mature measurements – Away from single point missions.

• NASA intends to evolve to measurements – The measurement has been made via remote sensing on previous missions– When a program has matured to the point that follow-on flights with a number of

appropriate instruments have been identified. – Flights and instruments need not all be provided by NASA

• Measurement programs are warranted when NASA needs – A parameter or set of related parameters needed for the measurement have emerged as

key to a climate research area with an NASA Earth Sciences research program– To support climate research with a seamless data set from mission to mission.– Consensus of the research community as to the importance of the parameter to that

research

• The selection of parameters that constitute a climate data record (measurement) is being developed within NASA Earth Sciences by

– No fixed “one-size fits all” process but focused on an identified research need– Science Divisions lead and – Managed by the science program managers – in conjunction with the science community


Characteristics of Measurement Approach

• Evolution from a science team per mission to a science team per measurement

– TRMM science team has become the Precipitation Measurement Missions (PMM) team

– Funding and research announcements are focused on the single mission science team

• Programmatic direction from a single Program Scientist within NASA Hq Earth Sciences.

• Measurement based data processing systems may have evolved from a single mission system but support multiple missions (instruments) for processing the measurement

– Build on existing relationship with the measurement science team

– Maintain processing capabilities from one mission to next

– Handle new missions, parameter data, or related instrument data as needed with minimum incremental cost for the processing


Advantages of the Measurement Approach

• Ensures continuity of measurements and data products key to NASA Earth Science research focus

• Helps ensures that missions are focused on the science measurement needs rather than on the hardware being developed

• Places missions within the overall direction of the measurement science community

• Ensures that measurement based science data processing systems– Are under science umbrella rather than the computer science one

– Have the science expertise to produce measurement products correctly, efficiently and with continuity

– Provides infrastructure support from mission to mission to

• Keep costs low – through incremental expansion costs

• Focus on the needs of the measurement data collection and processing

• Provide measurement expertise and continuity

• Ensure that processing not so general that all focus on the measurement is lost

• Facilitates science research on the measurement even when no mission is immediately available.


Precipitation Processing System

• Precipitation identified as ready for evolution to measurement– Many missions

– Long history of producing the measurement

– Consensus of the science community

• TRMM Data and Information System (TSDIS) designated as a prototype measurement based data processing system

• Basic requirements– Be able to handle data from a number of different sources

– Be able to able to support TRMM, existing precipitation data products and the Global Precipitation Measurement mission

– Be able to add or remove data streams (with a known incremental cost for additions)

– Be able to transition the data processing system to another provider when desired (i.e. move processing )

– Provide seamless consistent data set of precipitation through many missions


Single Point to Measurement Based


PPS Evolution Approach

• Totally architectural approach

– Analysis

– Requirements

– Design

– Implementation

– Testing

– Configuration Management

• Start with the TSDIS architecture and reuse as many architectural components

– Identify all architectural components

– Identify those that are fixed in single source approach

– Determine all dependencies (language, design, and code)

• From this starting part evolve PPS by

– Identifying the characteristics needed because of the given charge

– Identify all the components required

– Identify all the properties and constraints on processes, data flows, and connectors

– Multiple architectural walkthroughs before first design/implementation and walkthroughs throughout the process


ConfigurableConfigurable

FlexibleFlexible

Architecture Characteristics for “Measurement Based”

GPM Req for AlgorithmsSimplified deletionsSimplified modificationsSimplified additionsRun in different modesEase of maintenance

Expand functionality beyond initial evolution

Adaptable to new and different situations/uses

Usable in unanticipated manner

Support running of “special or injected code” in plug-in mode

Run on small Equipment

Run on large equipment

No hard software restrictions

No breakage on scaling

No lock-up on scaling

Operations settable via data

Data streams addable or changeable via data

Process interactions settable via data

DB and other tools settable via data

Easily moved to different hardware platforms

Easily moved to different operating system platforms

Ability to use with different supporting COTS

Minimum human interaction

Recover automatically for predetermined anomalies

Able to call for help

Recover automatically from failure

Portable

Extensible

Portable

Extensible Scalable

RecoverableRecoverable

Scalable

Configurable Portable Recoverable

Flexible Extensible Scalable


Evolving for “Measurement Based” Processing

Architecture

Processing Infrastructure

Algorithms

Expertise


PPS Architecture Overview

Project Specific

Existing Core


Some PPS Aspects

• Tied to the ongoing NASA REASON CAN community standards efforts

– Support ESML (GHCC)

– Support OpenDaPS (URI)

– Data storage in HDF4 and HDF5

– Evaluating other standards as identified

• Working to ensure availability in generic GIS data formats

• Allow dynamic subsetting online (in mulitple formats)

– Geographic

– dataset parameter subsetting

• Flexible toolkit

– Easily add formats (currently binary, HDF4, HDF5, NetCDF)

– Add languages (currently C, C++, and F77)

– Dynamically produce read/write routines for subsetted data

• Use of SOAP to provide access to special services (e.g. user code within PPS)

• Close working with other precipitation centers


Examples Working with Others

• Working with CSU to prototype – Intercalibrated 1C radiometer data from multiple sensors – A appropriate logical format for the representation of such data (this also with a

wider community using radiometer data

• Working with Univ of Utah and MSFC on precipitation features dataset

• Working with UAH/GHCC on prototyping aspects important for precipitation

– XML based data ingest formats– ESML for HDF4 and F77

• Working with several different groups on GIS versions of precipitation data

• Testing dynamic subsetting capabilities– To satisfy user requests on TSDIS– To satisfy study needs for GPM

• Driven by the needs of the precipitation community and the work done through REASON CAN


Prototype Science Discipline Center


PPS/GPM Operations Concept

Stocker - Page 1Evolution Measurement Based System-IGARSS 2006 Aug 1, 2006 Evolution of a Measurement-Based Data Processing System for Precipitation Erich.

Documents

science measurement

evolution measurement

measurement based system

measurement science

measurement products

measurement science

measurement expertise

measurement consensus