Cloud computing and web processing services

RTS Techniques de l'Information et de la CommunicationCNES - Toulouse, France - October 1st, 2013

https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services

Cloud Computing & Web Processing Services

[email protected]

Jérôme Gasperi

Cloud Computing Introduction

What we have done Issues

Web Processing ServicesIntroduction

What we have done Issues

What's next ?WPS on the cloud

R&T Cloud Computing (2011) Use a cloud infrastructure to process Earth Observation data

?

host build consumeIAAS PAAS SAAS

host build consumeIAAS PAAS SAAS

Private or public ?

process 1 €

store 5 €

access 0.5 €

to

+

1 time

per year

each time

(Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)

process 1 €

store 5 €

access 0.5 €

to

+

1 time

per year

each time

(Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)

Data security and user privacy cannot be guaranteed in public clouds

Majority of cloud providers are subject to governmental law (e.g. US patriot act)

...so we choose a private solution

Web browser

What we have done

eNovance (Paris)

Web browser

What we have done

eNovance (Paris)

SRTM tile

Data

Web browser

Processing (Orfeo Toolbox)

What we have done

eNovance (Paris)

SRTM tile

Data

Hill Shading

Web browser

Processing (Orfeo Toolbox)

What we have done

eNovance (Paris)

SRTM tile

Data

Hill Shading Projecting

Web browser

Processing (Orfeo Toolbox)

What we have done

eNovance (Paris)

SRTM tile

Data

Hill Shading Projecting

Web browser

Processing (Orfeo Toolbox)

What we have done

VISUALIZEWMS Server

Tiling

eNovance (Paris)

SRTM tile

Data

Conclusions

Private cloud technologies are quite simple to implement. They reduce the cost of operation and maintenance by sharing a common infrastructure across multiple projects

However...

Transfering large volumes of data is problematic

However...

Transfering large volumes of data is problematicThis is true for upload and download

However...

...so

Data and processes should be colocated

...so

Data and processes should be colocatedProcessing results should be downloaded only if needed

...so

Data and processes should be colocatedProcessing results should be downloaded only if neededBetter to offer a Web service to visualize/manipulate results

...so

Data and processes should be colocatedProcessing results should be downloaded only if neededBetter to offer a Web service to visualize/manipulate resultsStandardize processes inputs/outputs description (e.g. WPS)

...so

Web Processing Service (2012-2013)Set WPS on top of Orfeo Toolbox processing chains

Web Processing Services (WPS) is an OGC standard which defines interfaces to publish, describe and execute geospatial processes

WPS a ProcessingWeb application

From pixel...Remote sensing image

What we have done

From pixel...Remote sensing image

...to informationLand cover thematic map

What we have done

mapshup Web application

http://mapshup.info

WPSConstellationhttp://www.constellation-sdi.org

Orfeo Toolbox Processing

http://orfeo-toolbox.org

Orfeo Toolbox Remote sensing image libraryOpen SourceDevelopped by the French Space Agency

Processing

Orfeo Toolbox More than 70 high level processing chains orthorectification segmentation classification etc.

Processing

Orfeo Toolbox More than 70 high level processing chains orthorectification segmentation classification etc.

Processing

Supervised learning(land cover is computed from a set of "well known areas" given by user)

Based on SVM(http://en.wikipedia.org/wiki/Support_vector_machine)

Constellation

WPS

Geospatial Networking ServicesOGC/ISO implementationsMulti platform (JAVA)OpenSource

WMS WFS CSW WPS

mapshup Improves access to geospatial dataHTML + javascriptBased on OpenLayers and jQuery

Web application

mapshup Supported data sources

WMS WFS CSW WPS OpenSearch

etc.Flickr

Youtube

GeoRSS

KML

Wikipedia

Google maps

Bing maps

OpenStreetMap MapBoxWMTS

Web application

mapshup Supported data sources

See WPS demohttps://vimeo.com/67288678

WMS WFS CSW WPS OpenSearch

etc.Flickr

Youtube

GeoRSS

KML

Wikipedia

Google maps

Bing maps

OpenStreetMap MapBoxWMTS

Web application

https://vimeo.com/67288677

What havewe done ?

Conclusions

WPS 1.0 lacks a true asynchronous process management...There is no way to cancel a launched process !

However...

WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume

However...

WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and ResumeThis is a must have to deploy asynchronous WPS on the cloud

However...

WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and ResumeThis is a must have to deploy asynchronous WPS on the cloudShould be an official OGC standard by the end of 2013

However...

What's nextWPS on the cloud - OGC OWS-10 Testbeb

Orthorectifying images hosted by a cloud infrastructure using WPS standardIn kind contribution to the «Open Mobility» thread of the OGC OWS-10 TestbebFinal delivery and demonstration April/May 2014

Raw images

Processing

Data (Pléiades samples)

Client

Interoute (location : UK)

Services

PROCESSWPS Server

Orthorectify(OTB)VISUALIZE

WMS Server

SEARCHCSW Server

Metadata

+1

2

3

Assess Quality

Improve Quality

Services

Processing

Astrium GEO

Result

QualityWPS Server4 VISUALIZE

WMS Server

Landsat Landsat

Landsat2. Click on '+' to process a new orthorectification. Process parameters are set by user and sent to an asynchronous WPS orthorectification process located on the

3. Result is displayed within the map trough a WMS

Landsat

4. Orthorectified image quality can be checked through "Assess Quality" WPS process located on the Result is displayed within the map as a WMS quality layer stored on the

1. Select raw image to orthorectify within CSW catalog located on theLaat

RTS Techniques de l'Information et de la CommunicationCNES - Toulouse, France - October 1st, 2013

https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services

Cloud Computing & Web Processing Services

[email protected]

Jérôme Gasperi