ICEDS demo – preparing an OpenGIS Web Map Server Oliver Greening, ESYS plc Jeremy Morley, Prof. Jan-Peter Muller Dept. Geomatic Engineering, UCL Presented.

ICEDS demo – preparing an OpenGIS Web Map Server

Oliver Greening, ESYS plc

Jeremy Morley, Prof. Jan-Peter MullerDept. Geomatic Engineering, UCL

Presented by Rob Scott ESYS plc

Start-up Aims

• To prepare an OpenGIS Web Map Server + website

• To demonstrate Web access to large datasets– Europe (to Urals) and Africa

• To provide feedback on data preparation process

• Eventual aim: to use open access (I.e. open source / freeware / free to download) software– Deegree: deegree.sourceforge.net

• Current implementation: vendor software

Available Datasets

• 3 arcsec. unedited SRTM DEM data– Downloadable from Eros Data Center (EDS)

• VMAP-0 vector data– Downloadable from NGA (was NIMA)

• NASA MODIS "Blue Marble" global composite– Downloadable from NASA– (Elsewhere too in easier format but not instant ftp)

• JPL Landsat 5 full-res mosaic of Africa– 15 DVD-ROMs!– Courtesy of Nevin Bryant

Datasets URLs

• 3 arcsec. unedited SRTM DEM data– ftp://edcftp.cr.usgs.gov/pub/data/srtm/Eurasia/– ftp://edcftp.cr.usgs.gov/pub/data/srtm/Africa/

• VMAP-0 vector data– http://geoengine.nima.mil

• NASA MODIS "Blue Marble" global composite– ftp://mitch.gsfc.nasa.gov/pub/stockli/bluemarble/

Areas of Interest

• Two areas selected for demonstration:– British Isles (UK and Ireland)– Madagascar

• AOIs:– Madagascar:

43º East - 51 º East; 11 º South - 26 º SouthExpanded area: 30ºE-55ºE; 5ºS-30

– British Isles:11º West - 2º East; 49º North - 61º NorthExpanded area: 11ºW - 15ºE; 45ºN - 65ºN

SRTM processing

1. Download ZIPped DEM tiles (1º x 1º)• Madagascar: 85 tiles; British Isles: 182 tiles

2. Unzip tiles

3. Import into ArcInfo grids (adapted AML from EDC, invoked from a Unix shell script)

4. Merge grids into 1º latitudinal strips

5. Merge strips into a full DEM grid of AOI

6. Create a mask grid from DCW polygonised coastline• 1 inside coastline; NODATA outside

SRTM processing (cont.)

7. Mask DEM to set to NODATA all areas outside coastline

8. Project DEM to British National Grid• ArcInfo requires projected system for hill shading• choice of projection not too critical

9. Hill-shade the DEM• Selected a factor of 5 exaggeration for British Isles;

factor of 3 for Madagascar

10.Project hill-shaded grid back to Geographic• NB: Important to use nearest-neighbour resampling

Blue Marble processing

1. Import files into ArcInfo (or into ENVI)– Available from EDC as 2 binary data files – eastern

and western hemisphere– 30arcsec resolution: 21600 x 21600 pixels– Although text files stated 'interleaved', did not specify

interleaving type– In fact (third attempt!), files are BIP (Band Interleaved

by Pixel)– For ArcInfo, requires a HDR header file to be

constructed (straightforward – simple text format)

2. Merge eastern and western hemispheres

Web Map Server

• Using Cadcorp SIS 6.0 in Win2K3 Server running Microsoft IIS 6 for this demo

• Why?– Very simple to set up OGC WMS:– Open datasets and symbolise in desktop application– Save a "Saved Window Definition" (SWD) project file– Include ISAPI interface DLL + SIS DLLs in directory– Set up as a Virtual Directory in IIS– The maps are automatically served as they appear in

the desktop system

Data formats

• Cadcorp 6 will read a large variety of data formats• However not done anything clever in subdividing

the datasets => LARGE!• => memory and speed of access issues• => some data compression needed

• Choice: for this demo, use compressed data formats

Data formats (2)

• Hill-shaded DEMs: PackBits compressed TIFF– British Isles: 62.8 MB– Madagascar: 72.4 MB

• Blue Marble global composite: PackBits TIFF– Extracts taken of extended AOIs:– British Isles/NW Europe: 22.6MB– Madagascar/E Africa: 39.2MB

• VMAP-0: original VPF data (no import)– Cadcorp will read this vector format– NW Europe: 532MB (though mostly unused layers)– Sothern Africa: 362MB (also mostly unused layers)

On-line Services

• OpenGIS services can be made to 'cascade'– The map server receives layers for its map from other

OpenGIS services

• E.g.:JPL OnEarth server:– http://onearth.jpl.nasa.gov– Landsat 7 global mosaic

• No data holding or data preparation• However, relies on secondary server being up and

responsive, and on fast network link

JPL Landsat 5 Africa mosaic

• 15 DVD-ROMs– 13 DVD-ROMs holding 30m mosaic– Other two holding, e.g. subsampled versions

• Data stored as 5º x 5º tiles– 1 file per band; 3 bands (Landsat 1,2,3)– 324MB per file– GeoTIFF format (also, easy to construct HDR files)

• Madagascar: 18 files (i.e. 6 tiles x 3 bands)

Landsat 5 mosaic import

1. Copy from DVD and unzip

2. Create HDR files (ArcInfo doesn't read GeoTIFF)• NB: ArcInfo Workstation on Solaris

3. Import each band per tile to ArcInfo grid

4. Create an image stack for each tile• Quick – logical operation only

5. Merge tiles

6. Export to image format (JPEG)• 65.8MB for Madagascar (40-50ºE; 15-30ºS)• 6 – 20MB per tile, unmerged

Current total data holding

• ~ 35.9 GB

• Including intermediate products• Very little archiving or compression of intermediate

products yet!

Web site

• WMS images are generated in response to a URL containing map specification, e.g.:

– http://webtest.ge.ucl.ac.uk/iceds/SisISAPI.dll?REQUEST=GETMAP&VERSION=1.1.0&SRS=EPSG:4326&STYLES=&WIDTH=400&HEIGHT=400&FORMAT=IMAGE/JPEG&EXCEPTIONS=IMAGE/JPEG&LAYERS=COMPLETE---0,COMPLETE---1,COMPLETE---4,&BBOX=-10.9074225697008,45.1003094517055,6.48085238195591,62.4885844033622

• Requires a web site to provide user interface controls for map browsing, layer control, etc.

Web site (2)

• Current web site served from an average PC• Client-side processing only HTML 3.2 (basic!)

– Forms for UI controls– Tables for interface layout

• Server-side processing by a simple ASP script– ASP = Active Server Pages– Server-side scripting in Microsoft IIS web server– VBScript language embedded in HTML page– IIS server processes VBScript on demand– VBScript outputs HTML in place in document

Web site (3)

• Current interface would require only minor changes to work with another WMS package

• WMS interface is only through URL• URL is standardised by OpenGIS specifications• Only root of URL to specify active component and

its virtual directory change

Web site (4)

• http://webtest.ge.ucl.ac.uk/iceds/SisISAPI.dll?REQUEST=GETMAP&VERSION=1.1.0&SRS=EPSG:4326&STYLES=&WIDTH=400&HEIGHT=400&FORMAT=IMAGE/JPEG&EXCEPTIONS=IMAGE/JPEG&LAYERS=COMPLETE---0,COMPLETE---1,COMPLETE---4,&BBOX=-10.9074225697008,45.1003094517055,6.48085238195591,62.4885844033622

Time to assemble

• SRTM processing:– Approx. 1.5 weeks

• Blue marble processing:– Approx. 3 days

( + perhaps 2 days to establish pixel format)

• Setting up map in Cadcorp SIS:– ½ day

• Server configuration– ½ day (from base install of Win2K3 Server + IIS 6)

Time to assemble (2)

• Landsat 5 mosaic (Madagascar only)– Time to copy from DVD to server: 2 days– Extract and uncompress tiles in AOI, import into ArcInfo:

1½ days– Merge tiles & export: 1 day

• ASP Web page creation:– 1 day

• Work carried out on-site, and remotely via SSH and X Windows display

Hardware

• Most of the processing carried out on a Sun Sparc Ultra 60 machine, with a 20GB local disk and using a Linux-based RAID array.

• Web server: Dell Precision 210 - Pentium 3, 497MHz, 512MB RAM, 60GB local disk. Currently, the virtual directory contains about 3.4GB of data.

http://webtest.ge.ucl.ac.uk/iceds

Immediate plans

• Landsat 5 mosaic not currently integrated with server– Pressure of time– File sizes require additional memory in server?

• Switch to Deegree Open Source system– Requires data re-engineering

• Store full pixel data, not JPEG compressed• Improve user interface

– Demo interface at present

Conclusions

• Main data sets only recently acquired, therefore work is in progress

• Basic mechanisms are working well with temporary interface

• Some server, software and data issues to solve• An initial system with full data access for

Europe/Africa expected to be ready by WGISS 18