OnEarth: An Open Source Solution for Efficiently Serving High-Resolution Mapped Image Products Charles K. Thompson, Lucian Plesea, Jeffrey R. Hall, Joe.

OnEarth: An Open Source Solution for Efficiently Serving High-Resolution Mapped Image

ProductsCharles K. Thompson, Lucian Plesea, Jeffrey R. Hall, Joe

T. Roberts, Matthew F. Cechini, Jeffrey E. Schmaltz, Christian Alarcon, Thomas Huang, John M. McGann,

George Chang, Ryan A. Boller, Shriram Ilavajhala, Kevin J. Murphy, Andrew W. Bingham

Jet Propulsion Laboratory, California Institute of Technology

Goddard Space Flight Center

Outline

• Description of software• Current development and applications• Navigating the open source process

OnEarth fact sheet

• Set of web services supporting common protocols• FAST response unbound by image size and resolution• Leverages existing open source packages• Includes Apache modules, MRF access, scripts

• Georeferenced tile pyramid at discrete resolutions• Composed of data, index, & metadata files• Extension to GDAL

Key to speed:Meta Raster Format (MRF)

MRF data file (.ppg/.pjp)

• Concatenated individual PNG or JPG images• RGB and indexed support• Modifications to file are only via appends

MRF index file (.idx)

• Pointers to individual tiles in data file• Updated as tiles are modified

123413341384

10050150

Tile 1

Tile 1053

Tile 2

Tile 1054

Tile 3

101230054000

222100120

Offset Size

Tile 1055

indexfile

Datafile

Tile 4Waste

MRF metadata file (.mrf)

• Descriptive information about imagery• For use with GDAL routines

<MRF_META>

<Raster>

<Size x="81920" y="40960" c="1" />

<Compression>PPNG</Compression>

<DataValues NoData="0" />

<Quality>85</Quality>

<PageSize x="512" y="512" c="1" />

</Raster>

<Rsets model="uniform" />

<GeoTags>

<BoundingBox minx="-180" miny="-90" maxx="180" maxy="90" />

</GeoTags>

</MRF_META>

OnEarth: primary data flows

Layer Generation Layer Access

configurelayer

updateservice

documents

Layer Archive

createMRF

processrequest

returnserver

information

returnimage

• Global Imagery Browse Services• Collaboration between JPL and GSFC• Value-added image archive & web services for

NASA Earth science data• Image processing and workflow components• http://earthdata.nasa.gov/gibs

Ongoing development: GIBS

TheImagery

Exchange(TIE)

OnEarth

OnEarth applications

• Worldview• http://earthdata.nasa.gov/worldview

• State of the Ocean (SOTO)• http://podaac-tools.jpl.nasa.gov/soto/• http://podaac-tools.jpl.nasa.gov/soto-2d/

• Lunar Mapping & Modeling Project (LMMP)• http://lmmp.nasa.gov• http://moontours.nasa.gov

So…why the push for open source?• Share JPL-authored code with GIBS team• Promote open source paradigm at JPL• Change in personnel

Step 1: Navigating the evolving institutional process

• File a New Technology Report (NTR)• Maneuver through the management strata• Submit paperwork for review• Receive approval e-mail• Total time required: ~11 months!• Process being streamlined

Step 2: Pushing the code to the outside world

• Select appropriate software license• Select hosting service• Upload source to repository

A whole set of considerations to mull over!

• External company contributions• GDAL extensions• Institutional restrictions• Downstream development• Integration approach• Future change of course

• https://github.com/nasa-gibs/onearth

Current status

hostingservice

nasajpl nasa-gibsuser

tiledwms onearthsource

user

fork

softwarelicense

nod to origins active development

Related oral and poster presentations at AGU 2013

• Monday• IN14A-04. Expanding Access and Usage of NASA Near Real-

Time Imagery and Data• Wednesday

• IN31C-1512. See It First: Interactively and Visually Discovering Interesting Satellite Data with NASA Worldview

• Thursday• IN41C-1623. NASA Polar Imagery: Have It Your Way or Have It

Our Way• IN43C-05. The Imagery Exchange (TIE): Open Source Imagery

Management System• Friday

• IN51A-1535. Rapid Global Imagery Management and Generation In Action

Thank You!

OnEarth open source repository

GIBS

OnEarth applications

https://github.com/nasa-gibs/onearth

http://earthdata.nasa.gov/worldviewhttp://podaac-tools.jpl.nasa.gov/soto/

http://podaac-tools.jpl.nasa.gov/soto-2d/http://lmmp.nasa.gov

http://moontours.nasa.gov

http://earthdata.nasa.gov/gibs