AstroGrid’s Common Execution Architecturergm/sc4devo/sc4devo1/sc4devo1_11rixon.pdfAstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate

A PPARC funded project

AstroGrid’s Common Execution Architecture

Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developersSC4DEVO-1, Pasadena, July 2004

AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 2

AstroGridUK national virtual-observatory projectSee cover slide for participants and fundingTasked to produce a VO toolkit to be reused by service providersCollective author of a SOA using web servicesCommon Execution Architecture:

A major part of AstroGrid architecturePart of the service-oriented architectureA way of making and using web servicesSome important applications to data mining


SOA for data mining

How can we carve up the distributed application into services?

And where do we put the application logic?


Distributed app: conservative

Desktop

Application

Data-centre 1

Data source

Data-centre 2

Data transform

Application logic in here


Distributed app: browser-based

Desktop

Browser

Data-centre 1

Data source

Data-centre 2

Data transform

Application logic in here


Distributed app: intragrid

Desktop

Browser/app

Data-centre 1

Data source

Data-centre 2

Data transform

Compute grid

NodeNodeNodeNode

Some application logic in here;

Rest still in client or in user


Distributed app: scripted

Desktop

Browser/app

Data source

Data-centre 2

Data transform

Data centre 3

Data transform

Data-centre 1

Portal

Workflowengine

Script

App logic in here


Distributed app: embedded

Desktop

Browser

Data source

Data-centre 2

Data transform

Data mine

Application

Data-centre 1

Portal

Workflowengine

Script

Most app logic here

Little app logic here


Distributed app: data grid

Desktop

Browser/app

Data source

Data-centre 2

Data transform

Data centre 3

Data transform

Data-centre 1

Portal

Workflowengine

Script


Issues to be addressedBoring, technical:

How to write apps as w/s?How to write w/s clients?How to connect apps to storage?How to register functions?

Interesting, scientific:Who has useful algorithms?How to match algorithms to data?Best place to run computations?How to share code?Provenance?How to share kudos?Is more science possible?


AstroGrid architecture and productsAstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate on the science issues.AstroGrid supports:

Browser-based appScripted appData gridEmbedded app (pre-installed exes)

Common Execution Architecture lets all parts but exe for embedded app be reusable code.


Common Execution Architecture (1)Desktop

Browser/app

Data source

Data-centre 2

Data transform

Data centre 3

Data transform

Data-centre 1

Portal

Workflowengine

Script

Web service(CEC)

Application(local exe)

MySpace

Data mine

“CEC” = CommonExecutionConnector

CECconfiguration


Common Execution Architecture (2)Desktop

Browser/app

Data source

Data-centre 2

Data transform

Data centre 3

Data transform

Data-centre 1

Portal

Workflowengine

Script

Workflow executor (JES)

MySpace

Pagepresentation

Workflowdefinition

Portal

Scripts stored here,as files


Common execution architecture (3)

Common language throughout system allows s/w reuse.


How to write an app using CEA?Don’t write a web service!Write a command-line exeConfigure a CEC to run your app:

I.e. describe parameters in the CEA language

Or: get your app configured on someone else’s CEC


How to publish an app using CEA?Take the <ApplicationDefinition> that informs the CEC.Wrap it in a <VOResource> of sub-type CeaApplicationType.Publish the <VOResource> in IVO registry.


How to call an app-service using CEA?Look it up in the resource registry.Get the <ApplicationDefinition> from its <VOResource>.Give the <ApplicationDefinition> to a CEA delegate => define SOAP call.Add parameters from UI according to <ApplicationDefinition>.Bake in pre-heated SOAP engine @ HTTP 1.1.


CEA language example<ApplicationDefinition>

<Parameters><ParameterDefinition name="DetectionImage" type="binary">

<UI_Name>Detection Image</UI_Name><UI_Description>The image that is used to detect sources.

Basic position, shape and size information is derived from this image</UI_Description>

</ParameterDefinition><ParameterDefinition name="PhotoImage" type="binary">

<UI_Name>Measurement Image<UI_Name><UI_Description>The Image that is used to measure photometric

parameters</UI_Description></ParameterDefinition><ParameterDefinition name="config_file" type="text">

…


CEA language schema (1)


CEA language schema (2)


CEA facilities so far

PALCEC

HyperZCEC

Otherservice

MySpace

JESCEA Delegate Other Delegate

SIAPCEC


Future evolutionAdd facilities to CEC:

AsynchronicitySecurityMore parameter types

New facility: field-programmable CEC (PCEC)


PCEC (1): local code libraries

Client

PCEC

Data mine

Code library

Load exe

Select exe


PCEC (2): distributed code-library

Client

PCEC

Data mine

Code library

Download exe

Select exe

VOSpace

SandboxRun exe

Upload exe

AstroGrid’s Common Execution Architecturergm/sc4devo/sc4devo1/sc4devo1_11rixon.pdfAstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate

Documents