Managing Grid and Web Services and their exchanged messages Authors Harshawardhan Gadgil (his PhD topic), Geoffrey Fox, Shrideep Pallickara, Marlon Pierce.

ManagingManagingGrid and Web ServicesGrid and Web Servicesand their exchanged messagesand their exchanged messages

AuthorsAuthorsHarshawardhan Gadgil (his PhD topic), Geoffrey Fox, Harshawardhan Gadgil (his PhD topic), Geoffrey Fox,

Shrideep Pallickara, Marlon PierceShrideep Pallickara, Marlon PierceCommunity Grids Lab, Indiana UniversityCommunity Grids Lab, Indiana University

Presented byPresented byGeoffrey FoxGeoffrey Fox

[email protected]@indiana.edu

June 19, 2006June 19, 2006 Community Grids Lab, BloomingtonCommunity Grids Lab, Bloomington IN :CLADE 2006: IN :CLADE 2006:

22

Talk OutlineTalk Outline Management (Configuration/QoS Management (Configuration/QoS

Implementation)Implementation) Existing Management Approaches Existing Management Approaches Overview of NaradaBrokering Messaging Overview of NaradaBrokering Messaging

infrastructureinfrastructure NaradaBrokering Use CasesNaradaBrokering Use Cases Our Management ArchitectureOur Management Architecture Management Prototype and some results Management Prototype and some results

from itfrom it Future work and conclusionFuture work and conclusion


33

Management IManagement I Characteristics of today’s (Grid) applicationsCharacteristics of today’s (Grid) applications

– Increasing complexityIncreasing complexity– Components widely dispersed and disparate in Components widely dispersed and disparate in

nature and accessnature and access Span different administrative domainsSpan different administrative domains Under differing network / security policiesUnder differing network / security policies Limited access to resources due to presence of Limited access to resources due to presence of firewalls, firewalls,

NATsNATs etc… ( etc… (major focus in prototype)major focus in prototype)– Dynamic Dynamic

Components (Nodes, network, processes) may failComponents (Nodes, network, processes) may fail Services Services mustmust meet meet

– General General QoSQoS and Life-cycle features and Life-cycle features– (User defined) (User defined) Application specificApplication specific criteria criteria

Need to “manage” services to provide these Need to “manage” services to provide these capabilitiescapabilities


44

Management IIManagement II Management OperationsManagement Operations** include include

– Configuration and Lifecycle operations (CREATE, DELETE)Configuration and Lifecycle operations (CREATE, DELETE)– Handle RUNTIME eventsHandle RUNTIME events– Monitor status and performanceMonitor status and performance– Maintain system state (according to user defined criteria)Maintain system state (according to user defined criteria)

Protocols like Protocols like WS-Management/WS-DMWS-Management/WS-DM define inter- define inter-service negotiation and how to transfer metadataservice negotiation and how to transfer metadata

We are designing/prototyping a system that will We are designing/prototyping a system that will manage a manage a general world wide collection of services and general world wide collection of services and their network linkstheir network links

We are starting with our We are starting with our messaging infrastructuremessaging infrastructure as as – we need this to be robust in Grids we are using it in (Sensor and we need this to be robust in Grids we are using it in (Sensor and

amterial science)amterial science)– we are using it in management system we are using it in management system – and it has critical network requirements and it has critical network requirements

** From WS – Distributed Management http://devresource.hp.com/drc/slide_presentations/wsdm/index.jsp

http://devresource.hp.com/drc/slide_presentations/wsdm/index.jsp

http://devresource.hp.com/drc/slide_presentations/wsdm/index.jsp


55

Some Existing Management Some Existing Management Protocols IProtocols I SNMPSNMP – Simple Network Management Protocol – Simple Network Management Protocol

– Application layer protocol, based on reliable Application layer protocol, based on reliable connection-oriented protocol.connection-oriented protocol.

– Enables network admins to manage network Enables network admins to manage network performance and find and solve problemsperformance and find and solve problems

– Lacks security (Authentication), hence vulnerable Lacks security (Authentication), hence vulnerable to masquerading occurences, information to masquerading occurences, information modification etc…modification etc…

– In most cases “SET” operation not implemented In most cases “SET” operation not implemented and hence degenerates to monitoring facility only.and hence degenerates to monitoring facility only.

– Mainly deals with hardware resources.Mainly deals with hardware resources. CMIPCMIP – Common Management Information – Common Management Information

services and Protocolsservices and Protocols– Provides improved security (access control, Provides improved security (access control,

authorization and security logs) over SNMPauthorization and security logs) over SNMP


66

Some Existing Management Some Existing Management Protocols IIProtocols II CIMCIM – Common Information Management – Common Information Management

– Object oriented model that represents and organizes Object oriented model that represents and organizes informationinformation

– Allows extending existing management standardsAllows extending existing management standards Web Service based (Web Service based (WS-Management, WS-Distributed WS-Management, WS-Distributed

ManagementManagement), Upcoming merger to a common Web ), Upcoming merger to a common Web Service based management architectureService based management architecture– Helps making management interoperableHelps making management interoperable– Work underway to map CIM constructs to WSDMWork underway to map CIM constructs to WSDM– Provides a SOAP binding for various verbs (CREATE / DELETE / Provides a SOAP binding for various verbs (CREATE / DELETE /

SET / GET)SET / GET)– Defines negotiation between services and some metadataDefines negotiation between services and some metadata– Manage Manage services or non-service entities wrapped as servicesservices or non-service entities wrapped as services

HARDWARE – Processors, printers etc…HARDWARE – Processors, printers etc… SOFTWARE – Processes (E.g. Brokers in our case)SOFTWARE – Processes (E.g. Brokers in our case) Managers are ALWAYS ServicesManagers are ALWAYS Services

June 19, 2006 Community Grids Lab, Bloomington IN :CLADE 2006:

7

Stream

NB supports messagesand real-time streamsIdeas related to Skype

NB role for Grid isSimilar toMPI role for MPP

Queues

SensorsNaradaBrokering


88

NaradaBrokering Core Features NaradaBrokering Core Features II

Supports Supports general linkagegeneral linkage of threads, processes and services of threads, processes and services Message-level Message-level security security (See Grid06 paper (See Grid06 paper

http://www.naradabrokering.org/papers/NB-Security.pdfhttp://www.naradabrokering.org/papers/NB-Security.pdf)) Message Payload OptionsMessage Payload Options

– CompressionCompression and and DecompressionDecompression of payloads of payloads– FragmentationFragmentation and and CoalescingCoalescing of payloads of payloads

Message ComplianceMessage Compliance– Java Message Service (Java Message Service (JMSJMS) 1.0.2b compliant ) 1.0.2b compliant – (Obsolete) Support for routing P2P (Obsolete) Support for routing P2P JXTA JXTA interactions.interactions.

Grid Feature SupportGrid Feature Support– NaradaBrokering enhanced NaradaBrokering enhanced Grid-FTPGrid-FTP. (Old) bridge to . (Old) bridge to GlobusGlobus..

Web Service SupportWeb Service Support– Implementations of Implementations of WS-ReliableMessagingWS-ReliableMessaging, , WS-ReliabilityWS-Reliability

and and WS-EventingWS-Eventing..

S1 S2NB

http://www.naradabrokering.org/papers/NB-Security.pdf


99

NaradaBrokering Core Features NaradaBrokering Core Features IIII Multiple protocol transportMultiple protocol transport support support

– Transport protocols supported include TCP, Parallel TCP Transport protocols supported include TCP, Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS.streams, UDP, Multicast, SSL, HTTP and HTTPS.

– Communications through authenticating proxies/firewalls & Communications through authenticating proxies/firewalls & NATs. Network QoS based RoutingNATs. Network QoS based Routing

– Allows Highest performance transport with Allows Highest performance transport with 1-2 ms overhead 1-2 ms overhead and <1ms timing guaranteesand <1ms timing guarantees

Subscription FormatsSubscription Formats– Subscription can be Strings, Integers, Subscription can be Strings, Integers, XPathXPath queries, queries, Regular Regular

ExpressionsExpressions, , SQLSQL and tag=value pairs. and tag=value pairs. Reliable DeliveryReliable Delivery

– Robust Robust and and exactly-once deliveryexactly-once delivery in presence of failures in presence of failures Ordered DeliveryOrdered Delivery

– Producer OrderProducer Order and and Total OrderTotal Order over a message type. over a message type. Time Time OrderedOrdered delivery using Grid-wide delivery using Grid-wide NTP based absolute timeNTP based absolute time

Recovery & ReplayRecovery & Replay– Recovery from Recovery from failuresfailures and disconnects. and disconnects. Replay Replay of of

events/messages at any time. events/messages at any time. BufferingBuffering services. services. Open Source Open Source http://www.naradabrokering.orghttp://www.naradabrokering.org

http://www.naradabrokering.org/


1010

NaradaBrokering Management NaradaBrokering Management NeedsNeeds NaradaBrokering NaradaBrokering Distributed Messaging System consists of peers Distributed Messaging System consists of peers

(brokers) that collectively form a scalable messaging substrate. (brokers) that collectively form a scalable messaging substrate. Optimizations and configurations include:Optimizations and configurations include:– Where should brokers be placed and how should they be connected, Where should brokers be placed and how should they be connected,

E.g. RING, BUS, TREE, HYPERCUBE etc…, each E.g. RING, BUS, TREE, HYPERCUBE etc…, each TOPOLOGYTOPOLOGY has varying has varying degree of resource utilization, routing, cost and fault-tolerance degree of resource utilization, routing, cost and fault-tolerance characteristics.characteristics.

Static Static topologies or topologies created using static rules may be topologies or topologies created using static rules may be inefficient in some casesinefficient in some cases– E.g., In CAN, Chord a new incoming peer randomly joins nodes in the E.g., In CAN, Chord a new incoming peer randomly joins nodes in the

network. Network distances are not taken into account and hence network. Network distances are not taken into account and hence some lookup queries may span entire diameter of networksome lookup queries may span entire diameter of network

– Runtime metrics provide Runtime metrics provide dynamicdynamic hints on improving routing which hints on improving routing which leads to redeployment of messaging system (possibly) using a leads to redeployment of messaging system (possibly) using a different configuration and topologydifferent configuration and topology

– Can use (dynamically) Can use (dynamically) optimized protocolsoptimized protocols (UDP v TCP v Parallel TCP) (UDP v TCP v Parallel TCP) and go through firewalls but no good way to make choices dynamicallyand go through firewalls but no good way to make choices dynamically

These actions collectively termed as These actions collectively termed as Managing the Messaging MiddlewareManaging the Messaging Middleware


1111

NaradaBrokering Use CasesNaradaBrokering Use Cases Use case I: Audio – Video ConferencingUse case I: Audio – Video Conferencing

(GlobalMMCS project, (GlobalMMCS project, http://www.globalmmcs.orghttp://www.globalmmcs.org) which ) which uses NaradaBrokering as a event uses NaradaBrokering as a event delivery substratedelivery substrate

Consider a scenario where there is a Consider a scenario where there is a teacher and 10,000 students. One teacher and 10,000 students. One would typically form a TREE shaped would typically form a TREE shaped hierarchy of brokershierarchy of brokers

One broker can support up to 400 One broker can support up to 400 simultaneous video clients and 1500 simultaneous video clients and 1500 simultaneous audio clients with simultaneous audio clients with acceptable quality*. So one would need acceptable quality*. So one would need (10000 / 400 (10000 / 400 ≈ ≈ 25 broker nodes). 25 broker nodes).

May also require additional links May also require additional links between brokers for fault-tolerance between brokers for fault-tolerance purposespurposes

Use Case II: Sensor NetworkUse Case II: Sensor Network Both use cases need high QoS streams Both use cases need high QoS streams

of messagesof messages Use Case III: Management SystemUse Case III: Management System itself itself

** “Scalable Service Oriented Architecture for Audio/Video Conferencing”, Ahmet Uyar, Ph.D. Thesis, May 2005

… … …

…

400 participants

400 participants

400 participan

ts

A single participant sends

audio / video

Use Case I

http://www.globalmmcs.org/

June 19, 2006 Community Grids Lab, Bloomington IN :CLADE 2006: 12

Typical use of Grid Messaging in NASA

Datamining Grid

Sensor Grid implementing using NB

NB GIS Grid


1313

Core Features of Management Core Features of Management ArchitectureArchitecture Remote ManagementRemote Management

– Allow management irrespective of the location of the resource Allow management irrespective of the location of the resource (as long as that resource is reachable via some means)(as long as that resource is reachable via some means)

Traverse firewalls and NATsTraverse firewalls and NATs– Firewalls complicate management by disabling access to Firewalls complicate management by disabling access to

some transports and access to internal resourcessome transports and access to internal resources– Utilize tunneling capabilities and multi-protocol support of Utilize tunneling capabilities and multi-protocol support of

messaging infrastructuremessaging infrastructure ExtensibleExtensible

– Management capabilities evolve with time. We use a service Management capabilities evolve with time. We use a service oriented architecture to provide extensibility and oriented architecture to provide extensibility and interoperabilityinteroperability

ScalableScalable– Management architecture should be scale as number of Management architecture should be scale as number of

managees increasesmanagees increases Fault-tolerantFault-tolerant

– Management itself must be fault-tolerantManagement itself must be fault-tolerant. Failure of transports . Failure of transports OR management components should not cause management OR management components should not cause management architecture to fail. architecture to fail.


1414

Management SystemManagement System built in terms built in terms ofof

Bootstrap SystemBootstrap System – Robust itself by Replication – Robust itself by Replication Registry for metadataRegistry for metadata (distributed database) – (distributed database) –

Robust by standard database techniques and Robust by standard database techniques and our system itself for Service Interfacesour system itself for Service Interfaces

NaradaBrokeringNaradaBrokering for robust tunneled messages for robust tunneled messages – NB itself robust using our system– NB itself robust using our system

ManagersManagers – Easy to make robust using our – Easy to make robust using our systemsystem

ManageesManagees – what you are managing – Our – what you are managing – Our system makes robust – There is NO assumption system makes robust – There is NO assumption that Managed system uses NBthat Managed system uses NB


1515

Basic Management Basic Management Architecture IArchitecture I RegistryRegistry

– Stores system state. Stores system state. – Fault-tolerant through replication Fault-tolerant through replication – Could be a global registry OR separate Could be a global registry OR separate

registries for each domain (later slide)registries for each domain (later slide)– Current implementation uses a simple in-Current implementation uses a simple in-

memory system memory system – Will use our WS - Context service as our Will use our WS - Context service as our

registryregistry(Service/Message Interface to in-memory (Service/Message Interface to in-memory JavaSpaces cache and MySQL)JavaSpaces cache and MySQL)

– Note metadata transported by messages but Note metadata transported by messages but we use distributed database to implementwe use distributed database to implement

Messaging NodesMessaging Nodes– NaradaBrokeringNaradaBrokering nodes that form a scalable nodes that form a scalable

messaging substratemessaging substrate– Main purpose is to serve as a message Main purpose is to serve as a message

delivery mechanism between delivery mechanism between ManagersManagers and and Service AdaptersService Adapters (Managees) in presence of (Managees) in presence of varying network conditionsvarying network conditions

Registry

Read / Write

from / to Registry via pre-

determined TOPIC

NB


1616

Basic Management Basic Management Architecture IIArchitecture II

Resources to ManageResources to Manage ( (ManageeManagee))– If the resources DO NOT have a Web If the resources DO NOT have a Web

Service interface, we create a Service interface, we create a Service Service AdapterAdapter (a proxy that provides the Web (a proxy that provides the Web Service interface as a wrapper over the Service interface as a wrapper over the basic management functionality of the basic management functionality of the resource). resource).

– The Service Adapters connect to existing The Service Adapters connect to existing messaging nodes. This mainly leverages messaging nodes. This mainly leverages multi-protocol transport support in the multi-protocol transport support in the messaging substrate. Thus, alternate messaging substrate. Thus, alternate protocols may be used when network protocols may be used when network policies cause connection failurespolicies cause connection failures

ManagersManagers– Active entities that manage the resources. Active entities that manage the resources. – May be multi-threaded to improve May be multi-threaded to improve

scalability (currently under further scalability (currently under further investigation) investigation)

Manager

Registry

…

Read / Write

from / to Registry via pre-

determined TOPIC

ServiceAdapterResource

Managees

NB


1717

ArchitectureArchitectureUse of Messaging NodesUse of Messaging Nodes

Service adapters and Managers communicate Service adapters and Managers communicate through messaging nodesthrough messaging nodes

Direct connection possible, howeverDirect connection possible, however– This assumes that the service adapters are appropriately This assumes that the service adapters are appropriately

accessible from the machines where managers would runaccessible from the machines where managers would run May require special configuration in routers / firewallsMay require special configuration in routers / firewalls

– Typically managers and messaging nodes and registries are Typically managers and messaging nodes and registries are always in the same domain OR a higher level network always in the same domain OR a higher level network domain with respect to service adaptersdomain with respect to service adapters

Messaging Nodes (NaradaBrokering Brokers) Messaging Nodes (NaradaBrokering Brokers) provides provides – A scalable messaging substrateA scalable messaging substrate– Robust delivery of messages Robust delivery of messages – Secure end-to-end deliverySecure end-to-end delivery


1818

ArchitectureArchitectureBootstrapping ProcessBootstrapping Process

The architecture is arranged hierarchically.The architecture is arranged hierarchically.– Resources in different domains can be Resources in different domains can be

managed with separate policies for each managed with separate policies for each domaindomain

A A Bootstrapping serviceBootstrapping service is run in is run in every every domain where the management domain where the management architecture existsarchitecture exists..– Serves to ensure that the child domain Serves to ensure that the child domain

bootstrap process are always up and bootstrap process are always up and running. running. Periodic heartbeats convey status of Periodic heartbeats convey status of

bootstrap servicebootstrap service– Bootstrap service periodically spawns a Bootstrap service periodically spawns a

health-check managerhealth-check manager that checks that checks health of the system (ensures that the health of the system (ensures that the registry and messaging nodes are up registry and messaging nodes are up and running and that there are enough and running and that there are enough managers for managees)managers for managees) Currently 1 manager per manageeCurrently 1 manager per managee

/ROOT

/ROOT/FSU

/ROOT/CGL

RegistryRegistry

HierarchicalBootstrap

Nodes


1919

Architecture: Architecture: User ComponentUser Component Application-specific specification of the characteristics that Application-specific specification of the characteristics that

the resources/services being managed, should maintain. the resources/services being managed, should maintain. – Impacts Managee interface, registry and ManagerImpacts Managee interface, registry and Manager

Generic and Application specific policies are written to the Generic and Application specific policies are written to the registry where it will be picked up by a manager process. registry where it will be picked up by a manager process.

Updates to the characteristics (Updates to the characteristics (WS-Policy WS-Policy in future) are in future) are determined by the user.determined by the user.

EventsEvents generated by the Managees are handled by the generated by the Managees are handled by the manager.manager.– Event processing is determined by policy (future work), Event processing is determined by policy (future work),

E.g. Wait for user’s decision on handling specific E.g. Wait for user’s decision on handling specific conditionsconditions

The event can be processed locally, so execute default The event can be processed locally, so execute default policy, etc…policy, etc…

Note Managers will set up services if registry indicates that Note Managers will set up services if registry indicates that is appropriate; is appropriate; so writing information to registry can be used so writing information to registry can be used to start up a set of servicesto start up a set of services


2020

ArchitectureArchitectureStructure of ManagersStructure of Managers

Manager process starts Manager process starts appropriate appropriate manager threadmanager thread for the manageable resource for the manageable resource in questionin question– Heartbeat threadHeartbeat thread periodically periodically

registers the Manager in registryregisters the Manager in registry– SAM (Service Adapter Manager) SAM (Service Adapter Manager)

Module Thread starts aModule Thread starts a Service/Resource SpecificService/Resource Specific “Resource Manager” that “Resource Manager” that handles the actual management handles the actual management tasktask

– Management system can be Management system can be extended by writing extended by writing ResourceManagers for each type ResourceManagers for each type of of ManageeManagee

Manager

Heartbeat Generator Thread

SAM Module

ResourceManager


2121

PrototypePrototype We illustrate the architecture by managing the distributed messaging We illustrate the architecture by managing the distributed messaging

middleware, NaradaBrokering as illustrated by 3 use casesmiddleware, NaradaBrokering as illustrated by 3 use cases– This example motivated by the presence of large number of This example motivated by the presence of large number of

dynamic peers (brokers) that need configuration and deployment dynamic peers (brokers) that need configuration and deployment in specific topologiesin specific topologies

Use Use WS – ManagementWS – Management (June 2005) parts ( (June 2005) parts (WS – TransferWS – Transfer [Sep 2004], [Sep 2004], WS – EnumerationWS – Enumeration [Sep 2004] and [Sep 2004] and WS – EventingWS – Eventing) (could use ) (could use WS-DMWS-DM))– WS – EnumerationWS – Enumeration implemented but we do not foresee any implemented but we do not foresee any

immediate use in managing the brokering systemimmediate use in managing the brokering system– WS – TransferWS – Transfer provides verbs (GET / PUT / CREATE / DELETE) which provides verbs (GET / PUT / CREATE / DELETE) which

allow us to model setting and querying broker configuration, allow us to model setting and querying broker configuration, instantiating brokers and creating links between them and finally instantiating brokers and creating links between them and finally deleting brokers (tear down broker network) and re-deploy with deleting brokers (tear down broker network) and re-deploy with possibly a different configuration and topologypossibly a different configuration and topology

– WS – EventingWS – Eventing (will be leveraged from the WS – Eventing (will be leveraged from the WS – Eventing capability implemented in OMII)capability implemented in OMII)

WS – AddressingWS – Addressing [Aug 2004] and [Aug 2004] and SOAP v 1.2SOAP v 1.2 used (needed for WS- used (needed for WS-Management)Management)– Used XmlBeans 2.0.0 for manipulating XML in custom container.Used XmlBeans 2.0.0 for manipulating XML in custom container.

WS-ContextWS-Context will replace current registry will replace current registry


2222

Prototype ComponentsPrototype Components Broker Service AdapterBroker Service Adapter

– Note NB illustrates an electronic entity that didn’t start off with Note NB illustrates an electronic entity that didn’t start off with an administrative Service interfacean administrative Service interface

– So add wrapper over the basic NB BrokerNode object that So add wrapper over the basic NB BrokerNode object that provides provides WS – Management front-endWS – Management front-end

– Also provides a Also provides a buffering servicebuffering service to buffer undeliverable to buffer undeliverable responsesresponses These will be retrieved later by a separate Request – Response These will be retrieved later by a separate Request – Response

message exchangemessage exchange Broker Network ManagerBroker Network Manager

– WS – Management client component that is used to WS – Management client component that is used to configure a configure a broker objectbroker object through the Broker Service Adapter through the Broker Service Adapter

– Contains a Request-Response as well as Asynchronous Contains a Request-Response as well as Asynchronous messaging style capabilitiesmessaging style capabilities

– Contains a Contains a topology generatortopology generator component that determines the component that determines the wiring between brokers (links that form a specific topology)wiring between brokers (links that form a specific topology) For the purpose of prototype we simply create a CHAIN topology For the purpose of prototype we simply create a CHAIN topology

where each iwhere each ithth broker is connected to (i-1) broker is connected to (i-1)stst broker broker


2323

Prototype Prototype Resources/Properties Modeled (very specific to Resources/Properties Modeled (very specific to

NaradaBrokering)NaradaBrokering)

Resource URIResource URI OperationsOperations DescriptionDescriptionBROKERBROKER CreateCreate

DeleteDelete

Instantiates the broker Instantiates the broker with current configurationwith current configuration Deletes the broker nodeDeletes the broker node

LINK (Note we manage LINK (Note we manage brokers and streams)brokers and streams)

CreateCreate

DeleteDelete

Creates a link between Creates a link between two brokerstwo brokers Deletes the link between Deletes the link between two brokerstwo brokers

CONFIGURATION, CONFIGURATION,

CONFIGURATION CONFIGURATION PROPERTYPROPERTY

GetGet

PutPut

Retrieves the current Retrieves the current configuration / a single configuration / a single propertyproperty Saves the specified Saves the specified configuration / single configuration / single propertyproperty

NODE ADDRESS,NODE ADDRESS,

GATEWAY ADDRESSGATEWAY ADDRESS

CreateCreate Assigns a NODE / Assigns a NODE / GATEWAY address to the GATEWAY address to the current node if one is not current node if one is not already assignedalready assigned


2424

Benchmarks - IBenchmarks - I Test -I: Deployed a network of Test -I: Deployed a network of 88 brokers on 8 different brokers on 8 different

machines.machines.– Noted the overhead (Create Message + Marshall SOAP + Noted the overhead (Create Message + Marshall SOAP +

Network Latency + Unmarshall SOAP) introduced by the systemNetwork Latency + Unmarshall SOAP) introduced by the system Set Configuration: 73.56 mSecSet Configuration: 73.56 mSec Get Configuration: 61.11 mSecGet Configuration: 61.11 mSec Create Broker: 61.4 mSecCreate Broker: 61.4 mSec Create Link: 88.35 mSecCreate Link: 88.35 mSec Get Node Address: 68.94 mSecGet Node Address: 68.94 mSec Delete Broker: 75.02 mSecDelete Broker: 75.02 mSec

– Used direct HTTP connection (custom written SOAP client / Used direct HTTP connection (custom written SOAP client / server to allow for use of SOAP 1.2 based messages and server to allow for use of SOAP 1.2 based messages and provide compatibility with other software) provide compatibility with other software)

– Currently working on detailed analysis of benchmarks Currently working on detailed analysis of benchmarks with with time probably largely determined by marshalling and un-time probably largely determined by marshalling and un-marshalling messages which are one-way Web Service marshalling messages which are one-way Web Service invocations over TCPinvocations over TCP


2525

Benchmarks - IIBenchmarks - II Test -II: Test -II: Managed brokers were present on remote machinesManaged brokers were present on remote machines

– 3 brokers behind a Home DSL ROUTER3 brokers behind a Home DSL ROUTER– Used a distributed messaging substrate to route messages to appropriate Used a distributed messaging substrate to route messages to appropriate

recipients.recipients. Distributed messaging substrate provides multiple transport support, Distributed messaging substrate provides multiple transport support,

tunneling through firewalls (to enable remote management)tunneling through firewalls (to enable remote management)– Noted the overhead introduced by the systemNoted the overhead introduced by the system

Set Configuration: 172.01 mSecSet Configuration: 172.01 mSec Get Configuration: 178.69 mSecGet Configuration: 178.69 mSec Create Broker: 149.52 mSecCreate Broker: 149.52 mSec Create Link: 143 mSecCreate Link: 143 mSec Get Node Address: 144.34 mSecGet Node Address: 144.34 mSec Delete Broker: 131.41 mSecDelete Broker: 131.41 mSec

– SOAP message is received by a HTTP mapper service and is relayed to the SOAP message is received by a HTTP mapper service and is relayed to the service adapter by publishing a message over a pre-determined topic (with service adapter by publishing a message over a pre-determined topic (with the soap message as event payload). Response is relayed back in a similar the soap message as event payload). Response is relayed back in a similar fashion.fashion.


2626

Benchmarks - IIIBenchmarks - III Test III -Testing Messaging scalabilityTest III -Testing Messaging scalability::

– WS-Management states that if a WS-Management states that if a <wsman:MaxEnvelopeSize><wsman:MaxEnvelopeSize> is specified the value has is specified the value has to be a minimum of 8192 octets.to be a minimum of 8192 octets.

– Performed tests to find how many distinct clients can Performed tests to find how many distinct clients can a single broker support when publishing 8K messagesa single broker support when publishing 8K messages

Both WS-Context and NaradaBrokering can support Both WS-Context and NaradaBrokering can support 300-500 distinct client messages per second 300-500 distinct client messages per second – Axis overhead ignored here (as Axis 2 future)Axis overhead ignored here (as Axis 2 future)– Multi-core Chips like 8 core Sun Niagara much betterMulti-core Chips like 8 core Sun Niagara much better– Perhaps update every 10 secondsPerhaps update every 10 seconds

Today 3000 services per serverToday 3000 services per server– Will increaseWill increase


2727

Axis2 Performance on Multi Core Machines

0

20

40

60

80

100

120

0 500 1000 1500 2000 2500

Messages per Second

Round T

rip T

ime (seconds)

Sun T2000 - 100Bytes Sun T2000 - 400Bytes Sun T2000 - 1000Bytes

Intel Xeon - 100Bytes Intel Xeon - 400Bytes Intel Xeon - 1000Bytes

ms


2828

Axis2 Performance on Different JVMs

0

20

40

60

80

100

120

140

160

0 200 400 600 800 1000 1200 1400

Messages per Second

Rou

nd T

rip T

ime

(Mill

isec

onds

)

j2sdk1.4.2_03+Tomcat 4.1

jdk1.5.0_07+Tomcat 5.5

jrockit-R26.3.0-jdk1.5.0_06+Tomcat 5.5

On Intel Xeon with 400 byte messages

Newer JVMs

BEA JDK1.5

ms


2929

Future Work & Conclusion IFuture Work & Conclusion I This paper gives an This paper gives an overview of architectureoverview of architecture and and

illustrates with a illustrates with a prototypeprototype. . – Prototype focuses on using alternate means of Prototype focuses on using alternate means of

transports to provide different QoS (Quality of Service) transports to provide different QoS (Quality of Service) when certain transports are blocked due to network when certain transports are blocked due to network policies, presence of firewalls or due to NAT devices.policies, presence of firewalls or due to NAT devices.

Work is underway to demonstrate Work is underway to demonstrate fault-tolerance fault-tolerance of cmanagement omponents themselvesof cmanagement omponents themselves (managers, messaging nodes, registry) and how (managers, messaging nodes, registry) and how it affects the overall managementit affects the overall management

We will switch to MySQL/Javaspace We will switch to MySQL/Javaspace implementation of implementation of WS-Context for registryWS-Context for registry – note – note this is compatible with UDDIthis is compatible with UDDI– http://grids.ucs.indiana.edu/ptliupages/publications/SKG06-http://grids.ucs.indiana.edu/ptliupages/publications/SKG06-

Aktas.pdfAktas.pdf


3030

Future Work & Conclusion IIFuture Work & Conclusion II The scheme provides a The scheme provides a Web Service Web Service

management interfacemanagement interface for easy configuration for easy configuration and deployment of middleware components and deployment of middleware components with both general and application specific with both general and application specific features features

The costs obtained are The costs obtained are one-time initialization one-time initialization costscosts and hence acceptable. and hence acceptable.

We provided We provided basic testsbasic tests for scalability purposes for scalability purposes and are currently investigating solutions that and are currently investigating solutions that would improve the scalability in heterogeneous would improve the scalability in heterogeneous and wide-area (cross-continent) resource and wide-area (cross-continent) resource management.management.

Might be useful for Might be useful for debuggingdebugging framework as framework as detects and reports errorsdetects and reports errors

Managing Grid and Web Services and their exchanged messages Authors Harshawardhan Gadgil (his PhD topic), Geoffrey Fox, Shrideep Pallickara, Marlon Pierce.

Documents

management system

management operations

existing management

ws distributed management

network performance

network links

network admins

web services