Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: [email protected]],

Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)

Churngwei Chu [email: [email protected]], Fenny Wang, Walt Baskin, and Kay RoweScience Systems and Applications, Inc. Hampton, VA

and

Juliet PaoNASA Langley Research CenterAtmospheric Science Data CenterHampton, VA

September 26, 2007

National Aeronautics and Space Administration

www.nasa.gov

Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)National Aeronautics and Space Administration

Outline• Introduction

• Geodetic Model vs Geospatial Model (PostGIS)

• Steps to Project a Polygon in Geodetic Model to a 2-D map

• Results

• Summary & Future Work

Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 3National Aeronautics and Space Administration

Atmospheric Science Data Center

Radiation Budget

Clouds

Aerosols

Tropospheric Chemistry

• Providing data services for over 42 Projects• In 2006, 251 Terabytes of data distributed to 20,500 customers in 176 countries• >2 Petabytes of data in the archive• Increasing rate of ~28 Terabytes/month

Processing, Archiving, and Distributing Earth Science Data at NASA Langley Research Center

http://eosweb.larc.nasa.gov


NASA's Earth Observation System (1/3)

EOS Spacecraft

EOS Ground Stations



Data Processing

&MissionControl

Data Transport to Data Centers



Data Centers

InstrumentTeams/SIPS

Research Users

Education Users

Value-Added Providers

International Partners & Data Centers

Interagency Data Centers

Internet


ANGē (Archive Next Generation)• Distributed, multi-tier, service-based architecture enabling new methods for managing science

data

• Goals

Merge two systems into one.

Improve efficiency and automation.

Use commodity hardware (PCs) to reduce hardware and maintenance cost.

Use Open Source Software such as PostgreSQL/PostGIS, JBoss and GNU compilers.


Sample Data File


Data Flow for Informix

Data

GeoBox((-60,80),(60,-160),ANY,(1993-12-31 22:30:00,1994-01-01 01:30:00))

Informix Geodetic Datablade

GRing


Geo-object Types– Polygons

– Points

– Bounding boxes

– Line strings

Operations

Intersection between a query bounding box and above data types.


Challenges of Moving Data to PostGIS• Informix Geodetic Datablade uses geodetic model – spherical coordinate.

• Geospatial Model (PostGIS) uses projections – planar coordinate.

• Invalid polygons created by GRing stored in Informix.

• GRing still generates some invalid polygons.


Geodetic Model - Polygon• “A line segment is represented as a geodesic. On an ellipsoidal datum, a geodesic is the

shortest path between two points.” -- Informix Geodetic DataBlade Module User's Guide

• “You must specify the vertices in a ring that encloses a region so that the interior of the polygon is on your left as you traverse each ring from vertex to vertex.”

-- Informix Geodetic DataBlade Module User's Guide


Sample Data Across Anti-meridian in Geodetic Model


Cylindrical Projection in Geospatial Model


Projected Geoobj from GRing on 2-D map


Polygons in 2-D Map


Sample Data Covering South Pole in Geodetic Model




Projected Polygon on 2-D map


Data Flow for PostGIS

Data GRing

Polygon(.......)

ASDC ConversionProcess

A set of counter-clockwise sub-polygonsA set of clockwise sub-polygonsType of the original projected polygon

PostGIS


Steps to project a polygon into PostGIS Model

1) Project a polygon to 2-D map by adding points on anti-meridian.

2) Determine type of projected polygon:

a line string

a clockwise polygon

a counter-clockwise polygon

3) Partition the mapped polygon into two groups of sub-polygons, clockwise and counter-

clockwise.

4) If the polygon is projected into a line, more edges are added to enclose north pole or south pole which becomes a counter-clockwise polygon.


Polygon Covers South Pole Generated by GRing


Polygon in 2-D Map


Covered Region in 2-D Map


Data Covers both poles in Geodetic Model




Original Polygon From GRing on a 2-D map


After Adding Points on Anti-meridian


Two Clockwise Polygons


Intersection Query• In our PostGIS DB table, we have four columns to keep geo-spatial data, a polygon type,

counter-clockwise sub-polygons, clockwise sub-polygons, and a polygon (text format) in the geodetic model.

• The geometries, points or line strings are stored in the column with counter-clockwise sub-polygons.

• The following is a sample intersection query with bounding box (-40, -40) and (40, 40).

select granule_id from ctest where (((polygon_type > 1) and intersects(geoobj_ccw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1)) and (geoobj_cw is null or not contains(geoobj_cw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1)))) or (polygon_type = 1 and ((intersects(geoobj_ccw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1)) or (geoobj_cw is null or not contains(geoobj_cw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1))))))) order by granule_id;


Results

139913990(60, 160)(89.9, -160)5

145314530(-89.9, -120)(-60, -60)4

141814171(60, 60)(89.9, 120)3

1413139815(-40, -40)(40, 40)2

1807179413(-40, 120)(40, -120)1

No. of returned records from PostgreSQl/PostGIS

No. of returned records from

Informix

No. of Different Records

Bounding Box


Summary• Improved geospatial queries on a global scale.

Future Work• Enhance Performance of the algorithm and the queries.• Move our geo-spatial data in Sybase Space Query Server to Postgres/PostGIS.• Integrate with other open source GIS utilities and applications to expand ASDC user community.


Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: [email protected]],

Documents

earth science data

data services

data types

petabytes of data

terabytes of data

space administrationdata

science datagoalsmerge

invalid polygons