Top Banner
Service Oriented Architecture and Globus Toolkit Ravi K Madduri Argonne National Laboratory University of Chicago
39
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Session18  Madduri

Service Oriented Architecture and Globus Toolkit

Ravi K MadduriArgonne National Laboratory

University of Chicago

Page 2: Session18  Madduri

Agenda Principles of Service Oriented Architecture The Globus Toolkit Web Services Basics Grid Services What people punt on ?

Intro to Globus Security, Service Registries Workflows we created Lessons learned

Page 3: Session18  Madduri

Principles of Service Oriented Architecture

Guiding principles define the ground rules for development, maintenance, and usage of the SOA Reuse, granularity, modularity, composability,

componentization and interoperability Standards compliance (both common and

industry-specific) Services identification and categorization,

provisioning and delivery, and monitoring and tracking

Page 4: Session18  Madduri

Architectural Principles Service encapsulation – Many web services are

consolidated to be used under the SOA. Service loose coupling – Services maintain a

relationship that minimizes dependencies and only requires that they maintain an awareness of each other

Service contract – Services adhere to a communications agreement, as defined collectively by one or more service description documents

Service abstraction – Beyond what is described in the service contract, services hide logic from the outside world

Page 5: Session18  Madduri

Architectural Principles Service reusability – Logic is divided into

services with the intention of promoting reuse

Service composability – Collections of services can be coordinated and assembled to form composite services

Service autonomy – Services have control over the logic they encapsulate

Page 6: Session18  Madduri

Architectural Principles Service optimization – All else equal, high-

quality services are generally considered preferable to low-quality ones

Service Discoverability - Services are designed to be outwardly descriptive so that they can be found and assessed via available discovery mechanisms

Service Relevance – Functionality is presented at a granularity recognized by the user as a meaningful service

Page 7: Session18  Madduri

Globus Software: dev.globus.orgGlobus Software: dev.globus.org

Security ExecutionMgmt

InfoServices

CommonRuntime

Globus Projects

Other

MPICH-G2

GridWay

Data Mgmt

IncubatorMgmt

GridFTP

ReliableFile

Transfer

OGSA-DAI

GRAM

MDS4CAS

DataRepDelegation Replica

LocationJava

Runtime

C Runtime

Python Runtime

GT4

C Sec GT4 Docs

GSI-OpenSSH

MyProxy

Page 8: Session18  Madduri

Web Service Basics Web Services are basic distributed

computing technology that let us construct client-server interactions

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s02.html

Page 9: Session18  Madduri

Web Service Basics 2 Web services are platform independent

and language independent Client and server program can be written in

diff langs, run in diff envt’s and still interact Web services describe themselves

Once located you can ask it how to use it Web services are ideal for loosely coupled

systems Unlike CORBA, EJB, etc.

Page 10: Session18  Madduri

WSDL: Web Services Description Language

Define expected messages for a service, and their (input or output parameters)

An interface groups together a number of messages (operations)

Bind an Interface via a definition to a specific transport (e.g. HTTP) and messaging (e.g. SOAP) protocol

The network location where the service is implemented , e.g. http://localhost:8080

Page 11: Session18  Madduri

Real Web Service Invocation

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s02.html

Discover

Describe

Invoke

Page 12: Session18  Madduri

Web Services Server Applications Web service – software that

exposes a set of operations SOAP Engine – handle SOAP

requests and responses (Apache Axis)

Application Server – provides “living space” for applications that must be accessed by different clients (Tomcat)

HTTP server- also called a Web server, handles http messages

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s02.html

Container

Page 13: Session18  Madduri

Let’s talk about state Plain Web services are stateless

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html

Page 14: Session18  Madduri

However, Many GridApplications Require State

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html

Page 15: Session18  Madduri

Keep the Web Serviceand the State Separate

Instead of putting state in a Web service, we keep it in a resource

Each resource has a unique key

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html

Page 16: Session18  Madduri

Web Service+

Resource=

WS-Resource

Address of a WS-resource is called an end-point reference

Resources Can Be Anything Stored

Page 17: Session18  Madduri

Web Services So Far Basic client-server interactions Stateless, but with associated resources Self describing using WSDL

But we’d really like is a common way to Name and do bindings Start and end services Query, subscription, and notification Share error messages

Page 18: Session18  Madduri

Standard Interfaces Service information State representation

Resource Resource Property

State identification Endpoint Reference

State Interfaces GetRP, QueryRPs,

GetMultipleRPs, SetRP Lifetime Interfaces

SetTerminationTime ImmediateDestruction

Notification Interfaces Subscribe Notify

ServiceGroups

WebService

GetRP

GetMultRPs

SetRP

QueryRPs

SubscribeSetTermTimeDestroy

Client

Page 19: Session18  Madduri

WSRF & WS-Notification Naming and bindings (basis for virtualization)

Every resource can be uniquely referenced, and has one or more associated services for interacting with it

Lifecycle (basis for fault resilient state management) Resources created by services following factory pattern Resources destroyed immediately or scheduled

Information model (basis for monitoring & discovery) Resource properties associated with resources Operations for querying and setting this info Asynchronous notification of changes to properties

Service Groups (basis for registries & collective svcs) Group membership rules & membership management

Base Fault type

Page 20: Session18  Madduri

WSRF vs XML/SOAP The definition of WSRF means that the Grid and

Web services communities can move forward on a common base

Why Not Just Use XML/SOAP? WSRF and WS-N are just XML and SOAP WSRF and WS-N are just Web services

Benefits of following the specs: These patterns represent best practices that have

been learned in many Grid applications There is a community behind them Why reinvent the wheel? Standards facilitate interoperability

Page 21: Session18  Madduri

WS Core Enables Frameworks:E.g., Resource Management

Web services(WSDL, SOAP, WS-Security, WS-ReliableMessaging, …)

WS-Resource Framework & WS-Notification (*)(Resource identity, lifetime, inspection, subscription, …)

WS-Agreement(Agreement negotiation)

WS Distributed Management(Lifecycle, monitoring, …)

Applications of the framework(Compute, network, storage provisioning,

job reservation & submission, data management,application service QoS, …)

* An evolution of Open Grid Services Infrastructure (OGSI)

Page 22: Session18  Madduri

Globus and Web Services

WSDL, SOAP, WS-Security

WS-A, WSRF, WS-Notification

GlobusWSRF Web

Services

Reg

istry

and

Adm

in

Glo

bus

Con

tain

er(e

.g.,

Apa

che

Axi

s)User Applications

Globus Core: Java , C (fast, small footprint), Python

Page 23: Session18  Madduri

Globus and Web Services

WSDL, SOAP, WS-Security

CustomWeb

Services

WS-A, WSRF, WS-Notification

CustomWSRF

Services

GlobusWSRF Web

Services

Reg

istry

and

Adm

in

Glo

bus

Con

tain

er(e

.g.,

Apa

che

Axi

s)User Applications

Globus Core: Java , C (fast, small footprint), Python

Page 24: Session18  Madduri

Globus Security Extensible authorization framework

based on Web services standards SAML-based authorization callout

Security Assertion Markup Language, OASIS standard Used for Web Browers authentication often Very short-lived bearer credentials

Integrated policy decision engine XACML (eXtensible Access Control Markup Language)

policy language, per-operation policies, pluggable

Page 25: Session18  Madduri

Delegation Service Higher level

service Authentication

protocol independent

Refresh interface

Delegate once, share across services and invocation Client

Service1

Service2

Service3Delegation Service

Hosting Environment

Resources

DelegateEPR

EPR

Refresh

Delegate Refresh

Rachana Ananthakrishnan

Page 26: Session18  Madduri

Delegation

Secure Conversation Can delegate as part of protocol Extra round trip with delegation Types: Full or Limited delegation Delegation Service is preferred way of

delegating Secure Message and Secure Transport

Cannot delegate as part of protocol

Rachana Ananthakrishnan

Page 27: Session18  Madduri

Globus’s Use ofSecurity Standards

Supported, Supported, Fastest, but slow but insecure so default

Page 28: Session18  Madduri

Monitoring and Discovery System(MDS4)

Grid-level monitoring system Aid user/agent to identify host(s) on which to

run an application Warn on errors

Uses standard interfaces to provide publishing of data, discovery, and data access, including subscription/notification WS-ResourceProperties, WS-BaseNotification,

WS-ServiceGroup Functions as an hourglass to provide a

common interface to lower-level monitoring tools

Page 29: Session18  Madduri

Taverna

caGrid Scavenger with semantic/metadata based caGrid service query

A sample caGrid workflow

Page 30: Session18  Madduri

Sample Workflow with caDSR Scientific value

To find all the UML packages related to a given context (‘caCore’).

Not a real scientific experiment.

Simple. Important in caGrid.

Steps Querying Project object. Do data transformation. Querying Packages object

and get the result.

Workflow input

caGrid services

“Shim” services

Workflow output

Page 31: Session18  Madduri

Protein sequence information query Scientific value

To query protein sequence information out of 3 caGrid data services: caBIO, CPAS and GridPIR.

To analyze a protein sequence from different data sources.

Steps Querying CPAS and get the id,

name, value of the sequence. Querying caBIO and GridPIR

using the id or name obtained from CPAS.

Page 32: Session18  Madduri

Microarray clustering* Scientific value

A common routine to group genes or experiments into clusters with similar profiles.

To identify functional groups of genes.

Steps Querying and retrieving the

microarray data of interest from a caArrayScrub data service at Columbia University

Preprocessing, or normalize the microarray data using the GenePattern analytical service at the Broad Institute at MIT

Running hierarchical clustering using the geWorkbench analytical service at Columbia University

Workflow in/output

caGrid services

“Shim” servicesothers

*Wei Tan, Ravi Madduri, Kiran Keshav, Baris E. Suzek, Scott Oster, Ian Foster. Orchestrating caGrid Services in Taverna. ICWS 08.

Page 33: Session18  Madduri

Execution trace

Execution result as xml

1936 gene expressions

Page 34: Session18  Madduri

Lymphoma prediction type prediction Scientific value

Using gene-expression patterns associated with DLBCL and FL to predict the lymphoma type of an unknown sample.

Using SVM (Support Vector Machine) to classify data, and predicting the tumor types of unknown examples.

(Major) steps Querying training data from

experiments stored in caArray. Preprocessing, or normalize the

microarray data. Adding training and testing data

into SVM service to get classification result.

*Fig. from MA Shipp. Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nature medicine, 2002

*

Page 35: Session18  Madduri

Querying

Preprocessing

Classifying & predicting

Page 36: Session18  Madduri

Lymphoma type prediction Result snippet *Classification errors are

highlighted.

Acknowledgement: Juli Klemm, Xiaopeng Bian, Rashmi Srinivasa (NCI)Jared Nedzel (MIT)

Page 37: Session18  Madduri

Lessons Learned Service abstraction not applicable to

everything Virtual Organization concepts still good Web services is one way to create service

oriented architectures but not always the best way

Make implementation agnostic of tools underneath

True value in ability to create workflows

Page 38: Session18  Madduri

Service-Oriented Science People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …!!

“Service-Oriented Science”, Science, 2005

Page 39: Session18  Madduri

Questions ?