An Autonomic Service Delivery Platform for Service-Oriented Network Environments by Robert David Callaway Department of Electrical and Computer Engineering College Of Engineering North Carolina State University PhD Preliminary Examination November 2, 2007 Advisory Committee: Dr. Michael Devetsikiotis, Chair Dr. Yannis Viniotis, Co-Chair Dr. Adolfo Rodriguez, Member Dr. Andrew Rindos, Member Dr. Mihail Sichitiu, Member Dr. Sharon Setzer, Graduate School Representative
77
Embed
An Autonomic Service Delivery Platform for Service ... · routing, content transformation, and protocol integration to consumers and providers. By adding application-awareness into
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
An Autonomic Service Delivery Platformfor Service-Oriented Network
Environments
by
Robert David Callaway
Department of Electrical and Computer Engineering
College Of Engineering
North Carolina State University
PhD Preliminary Examination
November 2, 2007
Advisory Committee:
Dr. Michael Devetsikiotis, Chair
Dr. Yannis Viniotis, Co-Chair
Dr. Adolfo Rodriguez, Member
Dr. Andrew Rindos, Member
Dr. Mihail Sichitiu, Member
Dr. Sharon Setzer, Graduate School Representative
ABSTRACT
CALLAWAY, ROBERT DAVID. An Autonomic Service Delivery Platform for Service-
Oriented Network Environments. (Under the direction of Professors Michael Devetsikiotis
and Yannis Viniotis).
Service-oriented architectures (SOA) offer a more effective and flexible approach
to integrating technology with business processes than traditional IT architectures. SOA is
the foundation for both next-generation telecommunications and middleware architectures,
which are rapidly converging together on top of commodity transport services. Services such
as triple/quadruple play, multimedia messaging and presence are enabled by the emerging
service-oriented IP Multimedia Subsystem, and allow telecommunication service providers
to maintain, if not improve, their position in the marketplace. SOA is aggressively leveraged
in next-generation middleware systems as the system model of choice to interconnect service
consumers and providers within and between enterprises.
We leverage previous research in active, overlay, and peer-to-peer networking tech-
nologies, along with recent advances in XML and Web Services, to create the paradigm of
”service-oriented networking” (SON). SON is an emerging architecture that enables net-
work devices to operate at the application layer to provide functions such as service-based
routing, content transformation, and protocol integration to consumers and providers. By
adding application-awareness into the network fabric, SON can act as a next-generation fed-
erated enterprise service bus which provides vast gains in overall performance and efficiency
and enables the integration of heterogeneous environments.
The contributions of this research are threefold: first, we formalize SON as an
architecture and discuss the challenges in building SON devices. Second, we discuss issues
in interconnecting SON devices together to create large-scale service-oriented middleware
and telecommunications systems. In particular, we discuss the concept of federations of en-
terprise service buses, and present two protocols that enable a distributed service registry to
support federation. Finally, we propose an autonomic service delivery platform for service-
oriented network environments. The platform enables a self-optimizing infrastructure that
balances the goals of maximizing the business value derived from processing service requests
and the optimal utilization of IT resources.
Acknowledgments
The author would like to acknowledge financial support from the WebSphere Technology
Institute and RTP Center for Advanced Studies of IBM Software Group, as well as an IBM
2.4 Comparison of Software and Appliance Approaches . . . . . . . . . . . . . . 132.5 Example of Adaptive Admission Control: SEDA Response Time Controller 15
3.1 Example Topology of Multiple ESB Deployments - Hub & Spokes . . . . . 213.2 Example Topology of Multiple ESB Deployments - Peer Business Divisions 213.3 Example Topology of Interconnected Autonomous Federations . . . . . . . 223.4 Message Exchange Between Two ESBs Within a Federation . . . . . . . . . 263.5 Example of Contents of Hello XML Message . . . . . . . . . . . . . . . . . . 273.6 Example of Contents of Database Description XML Message . . . . . . . . 273.7 Example of Contents of Acknowledgement Database Description XML Message 283.8 Example of Contents of Service State Update XML Message . . . . . . . . . 293.9 Flowchart for Routing/Forwarding Service Requests within a Federation of
Enterprise Service Buses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.10 Message Exchange Between Two Autonomous Federations . . . . . . . . . . 323.11 Example of Contents of Open XML Message . . . . . . . . . . . . . . . . . 323.12 Example of Contents of KeepAlive XML Message . . . . . . . . . . . . . . . 323.13 Example of Contents of Update XML Message . . . . . . . . . . . . . . . . 333.14 Example of Contents of Notification XML Message . . . . . . . . . . . . . . 333.15 Message Exchange Between Three Autonomous Federations . . . . . . . . . 343.16 Example of Contents of Open XML Message . . . . . . . . . . . . . . . . . 343.17 Example of Contents of KeepAlive XML Message . . . . . . . . . . . . . . . 353.18 Example of Contents of Update XML Message . . . . . . . . . . . . . . . . 353.19 Example of Contents of Update XML Message . . . . . . . . . . . . . . . . 353.20 Flowchart for Routing/Forwarding Service Requests Between Autnomous
4.1 Example of Service-Oriented Network Topology with Multiple Service Providers 414.2 Plot of Utility Functions for Browse and Order Services . . . . . . . . . . . 534.3 Logical Diagram of Topology of the Service-Oriented Network . . . . . . . . 544.4 Plot of Offered Load for Browse and Order Services . . . . . . . . . . . . . 544.5 Plot of Supported Load for Browse and Order Services . . . . . . . . . . . . 554.6 Plot of Per-Flow Supported for Browse and Order Services . . . . . . . . . 564.7 Total Utility Gained from Browse and Order Services . . . . . . . . . . . . 56
5.1 Various Types of Utility Functions . . . . . . . . . . . . . . . . . . . . . . . 605.2 Using Traffic Prediction Algorithms to Minimze Optimization Calculations 61
1
Chapter 1
Introduction & Motivation
1.1 Telecommunications Services
Telecommunications services are becoming to a commoditized to a large extent;
network service providers are seeing low margins with high investments, and are required to
provide a near-perfect quality-of-service in order to satisfy their customers. Basic economic
theory states that profit and degree of commoditization are inversely proportional to one
another.
The basic landline telephone system in the US is a good example of such a service.
Local calling (basic voice transport service) is a commodity which is experiencing compe-
tition from VoIP providers. Traditional telephone providers have been long dependent on
value added services (such as long distance, caller ID, and voicemail, for example) to gen-
erate additional operational profits, as they allow the provider to differentiate themselves
in the marketplace.
1.2 Integration Services
Due to the pervasive nature of IT in corporations, the problems of heterogeneity
and change are the greatest issues facing IT managers today [1]. Even with wider adoption of
2
open standards, it remains a daunting task to make IT systems communicate across vendor,
protocol, and software differences. The rate of change in available hardware and software
products makes it even more difficult to continue to support a dynamic infrastructure which
is adaptable to business requirements and industry trends.
Furthermore, the pervasive nature of the Internet into global culture is futher
advancing the requirements for businesses to be able to adapt to a quality-sensitive, content-
driven customer base. The ability to interconnect systems in an attempt to maintain a
cohesive product offering is crucial.
1.3 Service-Oriented Architectures
Heterogeneity and change are the problems that are directly addressed by service-
oriented architectures (SOA). Services, the core unit of an SOA, are defined in [1] as “a
course-grained, discoverable software entity that exists as a single instance and interacts
with applications and other services through a loosely coupled, message-based communi-
cation model”. Services are based on the idea that IT infrastructures should be directly
aligned with relevant business processes, rather than more traditional horizontal or vertical
alignment. Services are comprised of the combination of different software components that
together execute a reusable business function.
One key property of services is that they are loosely coupled with one another
within the SOA. Loosely coupled is defined in [2] as having “no tight transactional properties
among the components.” This property is essential to SOA because it removes dependences
on implementation specifics by relying on interaction between services through standardized
interfaces. Services can be implemented in different languages and deployed on different
platforms. The use of standardized interfaces is the key to the enablement of SOA as a
flexible architecture.
Service-oriented architectures were designed to directly address the issues in open
systems [3]. If adopted and implemented correctly, SOA can provide a framework that can
leverage elements of an existing IT infrastructure which will reduce costs and provide a
more flexible and robust environment for the integration of IT and business processes.
3
1.4 Service-Orientation in Next-Generation Networks
The concept of service-orientation is being adopted in several next-generation net-
work architectures. In the IP Multimedia System designed by the 3rd Generation Part-
nership Project, service orientation is key to the interconnection and layering components
of the architecture. In parallel, the International Telecommunications Union has generated
the Next Generation Network standards whose main goal is the convergence of networks
and services. Furthermore, there has been a recent trend in the application/integration
middleware space towards XML-aware networking. This architecture assumes that XML is
now the lingua franca of network communication, and leverages XML-aware devices placed
in the network fabric to perform content-based routing, among many other functions.
Next-generation telecommunications and middleware systems are converging to-
gether under the theme of service-orientation. With this convergence, the properties of
network devices and the larger service-oriented network architecture is an open research
area, that draws from a diverse background of prior work. It is the goal of our research to
summarize the breadth of the research area, and make substantial in-depth contributions
to particular problems that are currently open in the literature.
1.5 Contributions of Our Research
The contributions of our research are as follows:
• Three conference papers
– R. D. Callaway, A. Rodriguez, M. Devetsikiotis, and G. Cuomo. “Challenges in
Service-Oriented Networking”, Proceedings of IEEE GLOBECOM, November
2006.
– R. D. Callaway, M. Devetsikiotis, Y. Viniotis, and A. Rodriguez. “An Auto-
nomic Service Delivery Platform for Service-Oriented Network Environments”,
Submitted to IEEE ICC, May 2008.
– R. D. Callaway, Y. Viniotis, A. Rodriguez, K. Brown, and R. Robinson. “En-
abling Federations of Enterprise Service Buses Using a Distributed Service Reg-
istry”, in preparation for submission.
4
• One journal paper
– R. D. Callaway, M. Devetsikiotis, Y. Viniotis, and A. Rodriguez. “An Auto-
nomic Service Delivery Platform for Service-Oriented Network Environments”,
in preparation for submission to IEEE Transactions on Network and Ser-
vice Management.
• Two patent applications
– Methods and protocols for enabling the dynamic and scalable federation of en-
terprise service buses
– Methods and protocols for enabling the dynamic and hierarchical interconnection
of autonomous federations of enterprise service buses
• One book chapter
– M. Devetsikiotis and R. D. Callaway. “The Role of the Enterprise Service Bus in
IP Multimedia System Architectures”, invited chapter of “The IP Multimedia
Systems Handbook”, edited by S. Ahson and M. Ilyas, CRC Press, 2008
• One invited presentation
– R. D. Callaway and A. Rodriguez. “Service-Oriented Systems in Next Generation
Middleware and Telecommunications Architectures”, Hot Topic Presentation at
1st IEEE Workshop on Enabling the Future Service-Oriented Internet,
November 26th, 2007.
1.6 Outline
The outline of the remainder of this report is as follows:
• In Chapter 2, we formally propose Service-Oriented Networking as an emerging ar-
chitecture. We discuss the challenges both in building SON devices as well as in
interconnecting the devices together to form a true network.
5
• In Chapter 3, we continue the discussion regarding large-scale service-oriented net-
works. We explicitly discuss a use case for SON, federations of enterprise service
buses. We describe how federations are enabled by a distributed service registry, and
provide details and examples of two protocols, based upon Internet routing protocols,
that enable a robust, scalable and dynamic infrastructure.
• In Chapter 4, we present our autonomic service delivery platform. The goal of this
platform is to optimally route requests from service consumers to providers. We
provide details of the underlying utility-based analytic framework, as well as results
from an initial experiment which shows the ability of the framework to optimally route
and throttle load under resource constraints.
• In Chapter 5, we summarize our work to this point and propose items for remaining
work on the autonomic service delivery platform.
6
Chapter 2
Service-Oriented Networking
2.1 Application-Aware Networks
Active and overlay networks have both attempted to provide application-aware
functionality in the network without an open standard for application-to-application com-
munication.
2.1.1 Active Networks
Active networks sought to improve the deployment of emerging networking tech-
nologies and protocols by adding application layer functionality in specific active nodes.
While data would still be passed in packets as in a traditional packet-switched network,
active networks would support “smart” packets, which would carry bytecode, along with
data, to be executed in active nodes. The main assumption underlying active network re-
search is that the network layer is inflexible and can not adapt to the dynamic requirements
of emerging network services. Since standardization of new protocols is often a lengthy
process, active network technology attempted to leverage advances in compilers, operating
systems, and programming languages that would enable running user-supplied code in ac-
tive nodes [4, 5]. Several groups [6, 7, 8] have proposed examples of potential architectures
for the organization of information and program code into the packet headers, showing
7
results in which active networks only suffer a slight degradation in performance versus a
software router.
However, active networks were not widely deployed due to issues with security,
resource allocation, and the substantial cost of deployment. Since packets could contain
arbitrary code to be executed on an active node, precautions must be taken to ensure that
an rogue user could not execute code that would corrupt the operations of other users. It
is essential to manage the computing resources of the node to ensure that programs are fair
to each other. Furthermore, the deployment of active network technology in the network
would require a substantial investment for network operators in order to support this new
architecture.
2.1.2 Overlay Networks
The development and subsequent deployment of active networks showed that en-
abling application-awareness in the network by executing user-supplied code in the network
layer is infeasible. Overlay networks sought to provide application-aware functionality by
pushing the complexity of such algorithms towards the end users of the network. The ma-
jor assumption in overlay networks is that application-aware functionality should not reside
in the network layer due to the issues presented in active networks; rather, application-
awareness should be enabled in the application layer where issues of security and resource
allocation could be more easily addressed. Overlay networks consist of peer nodes that self-
organize into a distributed data structure based on application criteria. Strategically placed
application-level agents serve as intermediaries for forwarding data from a source to a set
of destinations, in effect forming an overlay on top of the underlying IP substrate. Overlay
networks can be used to deploy new protocols, such as multicast [9], or enable application-
aware routing, where messages are forwarded based on application data or state.
2.2 Service-Oriented Networking
The rapid adoption of XML and acceptance of Web Services and SOA have enabled
network components to offload portions of application data processing or decision-making
8
outside of the traditional data-center. Differences in application data-encoding that once
hindered the network’s ability to comprehend true application intent are now described by
XML. Routing has become XML-oriented with the use of functions such as XPath routing
[10] to direct traffic based on XML content. Web Services Security (WS-Security) defines
security criteria within XML Web Services envelopes across service invocations. Further,
additional offload capabilities are now possible, such as XML transformation (XSLT) [11]
to change XML content as it traverses the network and service mediation to enable the
interoperability of Web Services in heterogeneous environments. These have key benefits
to SOA architectures as they enable services to be integrated in a loosely-coupled manner
where implementation details of components are hidden from the requester of the service.
Service-Oriented Networking (SON) is an emerging architecture that enables net-
work devices to operate at the application layer with ESB-like features such as offloading,
protocol integration, and content-based routing. By adding application-awareness into the
network fabric, SON can provide vast gains in overall performance and efficiency and en-
able the integration of heterogeneous environments. We refer to this collection of network-
resident application-level operations as SON functions. Among others, SON functionality
provides three key benefits: service virtualization, locality exploitation and improved man-
ageability.
2.2.1 Benefits of Service-Oriented Networking
Service Virtualization
Service virtualization transparently maps a set of services to the protected back-
end resources which actually provide the service. A SON device can serve as a proxy for
actual services by masking internal resources via XML transformation and routing tech-
niques. The SON device could also be leveraged to manage security and denial-of-service
(DoS) policies for incoming requests.
Locality Exploitation
By deploying certain functions in the network fabric, SON devices can be pro-
visioned and customized to handle unique workloads. For example, these systems can be
9
provisioned with cryptographic hardware assist for SSL or other security functions. Simi-
larly, domain-specific hardware to optimize XML processing can be installed to offset the
cost of processing Web Services or XML transformation functions. Provisioning and cus-
tomizing these SOA servers lead to greater efficiencies and is much more cost-effective than
provisioning the entire enterprise with these capabilities. Lastly, a potential performance
benefit is gained from exploiting locality within co-located SON functions. For example,
consider a function executing an XSLT schema transformation while another is performing
XPath routing. The two functions can communicate to avoid parsing the request twice.
Locality also has benefits at lower levels of the system, such as in cache utilization.
Improved Manageability
Offloading function into the network enables centralized, and therefore simplified,
management of the function and corresponding configuration. For example, style sheets,
security, caching and routing policies can all be centrally managed at SON devices versus
decentralized across a cluster of enterprise servers.
2.2.2 Service-Oriented Networking Functions
Three examples of SON functions include functional offloading, service integration,
and intelligent routing [12].
Functional Offloading
Offloading security-related operations has been a common practice for Internet-
based application environments. This practice applies equally well to document-centric
service-oriented environments. Like the HTTP server, a SON device can be specially provi-
sioned to handle cryptographic functions. This enables the device to optimize the validation
of digital certificates in the context of WS-Security. We illustrate this in Figure 2.1, where
the SON appliance intercepts WS-Security SOAP envelopes, performs the appropriate cryp-
tographic functions, and forwards the requests to the service provider.
A SON device can also perform a firewall-like security function to validate service
requests (for example, against a corresponding WSDL document) before forwarding them
10
ServiceProvider
Encrypted &Signed
SOAP/XML
Decrypted & AuthenticatedSOAP/XML
WS-Security:Decryption & Authentication
ServiceOriginator
SON Appliance
Figure 2.1: Example of Functional Offloading: Decryption and Authentication via WS-Security
to the enterprise server for processing. These checks would ensure that only well-formed
service requests are forwarded. This prevents DoS attacks and ensures enterprise servers
are encountering only valid service requests.
The most efficient form of offloading is in full-function offload where the service re-
quest can be satisfied completely within the SON device. Dynamic service response caching,
a technique that accomplishes this, is most effective for read-mostly interactions where re-
quests do not update back-end states or databases. For example, service requests that
retrieve stock quotes, where ticker values are updated every five minutes, are well suited
for this type of offload. If done correctly, a large proportion of the read traffic can be
completely serviced by the appropriate caching component, thereby reducing the load on
enterprise database servers. A cache policy contains rules that define how results from
specified services requests are cached.
Service Integration
Figure 2.2 illustrates the service integration aspect of the SON device in which a
widget retail store (Widgets, Inc.) is ordering a collection of parts by invoking a service
request back at the home office. The home office has deployed the SON device in the
network fabric that chooses the best parts supplier and forwards the service request to that
supplier. However, in this case, the XML schema of Widgets, Inc. is different depending
on the chosen supplier. The SON device is capable of transforming the original order to
11
WidgetsRUSServiceProvider
Purchase Order in Widgets, Inc.XML Schema
Purchase Order in WidgetsRUSXML Schema
XSLTransformationWidgets, Inc.
ServiceOriginator
SON Appliance
Figure 2.2: Example of Service Integration: Document Transformation via XML SchemaValidation and XSLT
schemas of participating providers, in this case WidgetsRUS. Other widget manufacturers
would likely require different schemas, requiring the SON function to apply the appropriate
XSLT transformation dependant on the supplier.
Since the majority of corporate data today exists in mainframe databases, service
integration also provides the ability to interface with existing legacy systems, giving a
system architect much more flexibility to migrate towards a service-oriented environment.
This increases the number of service consumers that can take advantage of these programs
and data and extends the reach of SON further into the enterprise.
Intelligent Routing
Content-based routing (CBR), like priority-based routing, is driven from policy
documents. The policies typically apply a rule against some part of a service request
(header or content) and derive a token as a result. The token is then used to look up a
corresponding enterprise server address in a routing table. For example, a CBR policy might
be created by combining the port-type and operation-name of a service and mapping it to
a specific enterprise server. In a SON device, CBR can be realized by using XPath-based
expressions to determine the destination of the request as shown in Figure 2.3.
CBR also allows an affinity between a class of services and the enterprise server
that services the request, called service partitioning. Figure 2.3 also shows this service
partitioning pattern. Service partitioning can be used as the foundation to address bottle-
necks that occur in high volume Online Transaction Processing (OLTP) applications that
12
ServiceProviders
Unclassified Requests
XPathRouting
SON Appliance
Figure 2.3: Example of Intelligent Routing: Routing Requests to Logical Partitions usingXPath Routing
intensively read and write data to databases and require the utmost in data consistency
and availability. Examples of such systems include trading, banking, reservation, and online
auctioning systems. While a strategically located SON device enables service partitioning,
the value is actually garnered on the enterprise application servers where the services are
run. For example, service-based applications can now assume that their variation of the
service is not running anywhere else in the enterprise server cluster. The applications can
then aggressively cache interactions without the processing overhead of maintaining data
consistency within that cluster. Service partitioning also enables other optimization tech-
niques such as data batching where insertions, updates, and deletions can be done in bulk
to the database.
2.3 Research Challenges
We believe that service-oriented networking is an exciting new research area that
can have a dramatic impact on the design, performance, integration, and management of
service-oriented environments. Therefore, we believe that significant research is needed in
the following areas in order to create an adaptive and robust SON device which can provide
the benefits of service-oriented networking as we have described in this paper:
13
2.3.1 Implementation Considerations
A tradeoff exists between the performance of implementing SON functions in a net-
work appliance versus software, the extensibility of arbitrary programs versus the hardened
security of an appliance based upon standardized security mechanisms, and the flexibility
of a software solution versus the increased consumability of an appliance. This tradeoff is
depicted in Figure 2.4. Since care must be taken to ensure that the SON function improves
SecurityExtensibility
AppliancePerformance
Software Performance
Consumability
Flexibility
SON Appliance
Figure 2.4: Comparison of Software and Appliance Approaches
the overall performance of the architecture rather than degrading it, we believe that net-
work appliances that host SON functionality can leverage hardened software and specialized
hardware solutions and overcome the limitations experienced in previous attempts to in-
troduce application-awareness into the network fabric. SON functions could be collocated
within a switch or router, as in the Cisco line of AON products [13]. The SON functionality
can also be deployed in a standalone hardened appliance as in several products sold by
DataPower, recently acquired by IBM [14].
A further discussion of issues concerning implementing SON functions in a network
appliance can be found in Section 1.5.
14
2.3.2 Performance
Robustness
The SON device should scale to support a large number of requests to be processed
concurrently. It should be robust to overload conditions, continuing to prioritize and pro-
cess high priority requests and shed low priority requests while operating in an overloaded
regime. Admission control ensures that a server always operates in a stable regime; even
in overloaded conditions, the server can scale and continue to provide differentiated service
to its users. In order for an SON device to provide services that are fair to the requesting
users, a policy should be defined that enumerates the differential treatment that requests
are to receive. This policy should define strategies to prioritize traffic under both normal
and overloaded conditions. Since requests must be classified before they can be prioritized,
it is essential in overloaded conditions that the system can continue to process high pri-
ority requests while possibly shedding lower priority requests. Therefore, fast methods for
classifying incoming requests are needed. The classifications could be based on network
layer information or upon information residing within the XML content. Algorithms for
executing XPath expressions on streaming XML such as QuickXScan [15] could be useful
in such situations.
Resource Allocation
The SON device should be adaptive, changing its underlying execution model to
support different types of software components in order to maximize the efficiency of the
system. We believe that concurrency mechanisms will be a significant component of resource
allocation within a scalable and adaptive SON device. Concurrency mechanisms have a
dramatic effect on the overall performance and efficiency of a device. Internet services are
unique because they require massive concurrency but also block while waiting on unavailable
resources. It is this unique combination of requirements that suggests a hybrid architecture
could be used to exploit the benefits of different concurrency mechanisms. Models such
as the Staged Event-Driven Architecture (SEDA) [16] could prove useful in building an
adaptive resource allocation system for an SON device.
15
SEDA is an architecture that separates functions within applications into stages,
which each have their own thread pool and are connected together as a network of queues
in order to provide the desired application functionality. Admission control is used at each
stage, and adaptive controllers that can modify the thread pool size or the amount of
requests that are processed by each thread (batching) are included. Figure 2.5 (reprinted
from [17]) shows how admission control is performed on requests to a SEDA stage using a
response time controller.
Figure 2.5: Example of Adaptive Admission Control: SEDA Response Time Controller
Performance Optimization
One main benefit of SON is that it can leverage specialized hardware, such as
hardware-accelerated cryptographic or XML processing functionality, to enhance the overall
performance of the device. However, the SON device will contain software components
that will process requests in conjunction with the available hardware devices. Since these
components could block upon the remove invocation of services, it will be important to
ensure an efficient and robust cooperation scheme between these hardware and software
components exists as this scheme will be crucial to the overall stability and performance of
the SON device.
16
2.3.3 Security and Management
Security
As in active networks, SON provides software functionality that will be executed
in the network fabric. However, with the introduction of open standards such as XML
and WS-Security, we believe that SON devices will not suffer from the same security issues
as active networks. The use of XML in network operations raises new research questions
regarding how open standards such as XML and Web Services could be leveraged together
in an SON appliance in order to create a device which is hardened against XML and Web
Services-based DoS attacks. One approach is to leverage well-formedness checking and
XML schema validation against all incoming documents in order to ensure that only valid
requests proceed within the device for further processing.
Manageability
The initial work presented here concentrates on enablement technologies that logis-
tically deliver and deploy SON functions manually; however, we look toward the autonomic
configuration and coordination amongst these functions. Specifically, we anticipate that
enterprise applications of the future will begin to leverage distributed SON design patterns
where large numbers of SON devices coordinate with peers using network-wide application-
specific policies. Manual configurations are not able to scale with these environments,
nor can they adapt the configuration to dynamic network and application conditions. For
example, a large-scale SON deployment could be leveraged to enable application-specific
multicast. SON devices should coordinate with their peers to determine the appropriate
points in the network to perform configuration changes based on prevailing network and
application conditions.
17
Chapter 3
Large-Scale Service-Oriented
Systems
The enterprise service bus (ESB) acts as the integration and communications plat-
form for connecting service consumers and providers. It is often desirable to have multiple
ESB deployments federate with one another to provide a distributed integration platform
that promotes the reuse of services within and across enterprises. However, the existing
solutions to federate ESBs are limited by their inflexibility to change and inability to scale.
In this chapter, we propose the enablement of a federation of enterprise service buses via
a distributed service registry that distributes policy-appropriate service metadata to feder-
ation members. We provide a high-level description of two protocols which maintain the
state of the distributed registry within and between autonomous federations. We argue
that these application of a distributed service registry and the enabling protocols is a novel
application of existing technology that creates a robust, scalable, and flexible federation of
ESBs that is needed in the next generation of large-scale SOA deployments.
18
3.1 Introduction & Motivation
As a critical infrastructural component of service-oriented architectures (SOA),
the enterprise service bus (ESB) acts as the integration and communications platform for
connecting service consumers and providers [18]. As such, the ESB is responsible for, along
with many other functions, the enforcement of policies, routing of service requests, and
performing content and/or transport protocol transformation.
As the number of services in an organization increases, the need for a service
discovery and governance platform arises. The service registry enables consumers to find
available services and providers to advertise available service instances. The registry can
optionally serve as a repository for governance metadata, policy documents, and XML
schemas.
Instantiating the enterprise service bus in a message-oriented middleware product
along with deploying a service registry provides an intuitive solution towards implementing a
small to medium size SOA. However, recent market trends show that SOA is being rapidly
adopted; therefore, strategies for creating more large-scale deployments are needed. A
typical approach to transitioning from a moderate-scale to a large-scale SOA deployment
is to “scale-up”; that is, leave the topology of the architecture fundamentally unaltered
while adding additional resources to the individual architectural components. “Scaling-
out” yields a distributed approach to the large-scale problem that involves altering the
topology of interconnected architectural components. Furthermore, we argue that the rapid
adoption of SOA is causing an increase in the number of business-to-business transactions
between autonomous SOA deployments. Primarily for reasons of governance, these types of
interactions exemplify the need for a large-scale distributed ESB; we refer to such a system
as a federation of enterprise service buses.
In a federation of ESBs, the primary problem is to appropriately disseminate infor-
mation throughout nodes that comprise the ESB to enable policy-driven service discovery
and routing. We propose that a distributed service registry is a scalable and robust ap-
proach to enabling federations of enterprise service buses. The distributed service registry is
hierarchical in nature, and is maintained by two protocols that synchronize relevant service
metadata amongst ESB deployments as appropriate under defined business policies. There
are three main advantages to our proposal over existing approaches:
19
• We enable the federation of ESB deployments within an enterprise in a flexible and
scalable manner.
• The distributed service registry and the supporting protocol allow our solution to
adapt autonomically to dynamic network and service conditions.
• This architecture provides the capability to support on-demand techniques such as
fast failover or priority-based load shedding in an autonomic fashion.
The remainder of the chapter is structured as follows: in the following section, we
review existing approaches to the ESB federation problem. In Section 3.3, we explicitly pro-
pose our architecture that enables the dynamic and scalable federation of ESBs. In Section
3.4, we present an overview of the first of two protocols which maintains the consistency
and availability of service metadata within an autonomous federation. In Section 3.5, we
present the second protocol which is responsible for the maintenance of interconnections of
autonomous federations. In Section 3.6, we provide a brief review of related literature in
the service discovery, naming, and registry areas.
3.2 Current Approaches to ESB Federation
Currently, there are three approaches to addressing the problem of policy-driven
service metadata dissemination: manual configuring interconnections, deploying a broker
ESB, and utilizing a centralized registry across or between enterprises.
3.2.1 Manual Configuration
One way of federating ESBs is by manually configuring functionality within an ESB
that serves as a ”proxy” to other ESBs in the federation. For each service that is managed
by a remote ESB, a mediation must be defined that selects appropriate requests to be
forwarded to the remote ESB, performs necessary content/protocol transformations, and
then forwards the request onto the remote ESB. Matching mediations must exist on remote
ESBs in order to support bidirectional communication in this case. Since this configuration
must be done manually by a systems administrator at each ESB, the configuration of such
20
a solution is tedious and prone to error (for S services and N ESBs, there are possibly
SN proxies to be configured). There is also no mechanism to change the properties of this
mediation based on changes in network or service availability. Manual configuration allows
basic federation of multiple ESBs; however, this is an inflexible and impractical solution for
large scale enterprises.
3.2.2 Broker ESB
Rather than statically defining the routing mediations at each ESB, a separate ESB
called a ”broker” ESB can be deployed whose sole function is to implement the requisite
mediations to support federation. This helps to consolidate the many different mediations
that might exist in the manually configured solution described above into a single ESB.
However, this consolidation is still dependent on a systems administrator to manually define
the mediations required for each service (in this case, the number of proxies to be configured
is minimized to S). Since there is no mechanism to update the mediation metadata based
on dynamic service availability, the broker ESB solution is inflexible. The broker ESB then
becomes the architectural bottleneck, which introduces issues with scalability and fault
tolerance.
3.2.3 Centralized Registry
The final known approach is to deploy a centralized registry for the entire enter-
prise. When ESBs need to route service requests to other ESBs within the SOA, they would
consult the registry at runtime to make a forwarding decision based on the current location
of a service instance, thus addressing the manual configuration concerns raised by the pre-
vious solutions (as with the broker ESB, the number of entries in the centralized registry is
equal to the number of services S). However, centralizing all service metadata and status
into a single registry forces the registry to be the architectural bottleneck in such a federated
system, causing concerns with system performance, scalability, and fault tolerance. The cen-
tralized registry is ideal from the standpoint of the consolidation of service information, but
is infeasible in many realistic business scenarios due to business-to-business interactions,
disparate geographical locations, and limitations imposed by business structures. Today,
21
Figure 3.1: Example Topology of Multiple ESB Deployments - Hub & Spokes
Figure 3.2: Example Topology of Multiple ESB Deployments - Peer Business Divisions
manual configuration of the centralized registry is required to insert/update/delete service
metadata, which limits the flexibility of this solution.
3.3 Federation Architecture
The overarching goal of ESB federation is to provide a logically centralized (at an
appropriate scope) integration platform across different geographic and business boundaries;
that is, the topology formed by the federation of ESB deployments should align directly to
the structure of entities within an enterprise. Examples of federated ESB topologies that
align with common business structures are presented in [19].
Figure 3.1 shows the logical topology of a hub/spoke federated ESB. This type
of topology directly aligns with the Store/Branch business structure described in [19], and
forces all service routing to be done through the hub ESB deployment.
Figure 3.2 shows the logical topology of a directly-connected federated ESB. In this
22
Figure 3.3: Example Topology of Interconnected Autonomous Federations
topology, all ESB deployments are connected directly to one another, so that service requests
which are routed within the federation pass directly from the source ESB to destination ESB.
This type of topology directly aligns with the Multiple Geographies & Multiple Business
Divisions business structures described in [19].
A natural extension of the intra-federation topology is interconnecting multiple
federations, as shown in Figure 3.3. There is a practical need for interconnected federa-
tions; the need arises, for example, in business-to-business environments, in which separate
enterprises have to interact to provide a service to each other or to create a composite ser-
vice to be offered to an external customer. The same need arises even within a single but
large enterprise (e.g., in an e-government setting), when the enterprise itself is organized as
multiple, autonomous, and heterogeneous federations of ESBs.
A key concept in our proposal is the notion that the amount of service registry
23
data that is shared with a federation member is configurable via policy; we refer to this
concept as policy-based service advertisement. For example, in the hub & spoke case, it is
desirable for a spoke to share appropriate service information (as defined by policy) with
the hub ESB, and share no service information with any other spoke ESB. Policy-based
service advertisement allows different members of the federation to have different views of
hosted services at a particular federation member. We envision that certain services should
only be exposed to certain federation members, and that it may be desirable to allow or
disallow the advertisement of particular services. While certainly related, we believe that
the appropriate distribution of policy documents is an orthogonal problem to the one we
are addressing and is therefore outside the scope of this manuscript.
Our method to achieve a dynamic and scalable federation of enterprise service
buses is based upon the concept of a distributed service registry. Federation members
create a distributed service registry by sending policy-based service advertisements to peer
members. Each federation member will have its own (possibly unique) converged view of all
routable service endpoints in the federation, which it will use in making routing/forwarding
decisions. This model contrasts with the centralized registry solution described in Section
II; notably, the distributed nature of the registry allows it to overcome the scalability and
robustness concerns that exist with a centralized solution. In order to allow the federa-
tion members to distribute service state amongst themselves, protocols are needed that
implement the policy-based service advertisements in an automated fashion.
3.3.1 Related Work
There are several previous attempts that have been made towards developing fed-
erated service discovery architectures that are based on service registries. The authors of
[20] propose defining a topology for collaborative service discovery, and subsequently floods
the topology with the discovery query, and all nodes respond to the client with the results.
A similar proposal is presented in [21], where UX servers are part of a federation, and a
minimum spanning tree is found in order to flood queries if an initial lookup returns null.
However in both of these proposals, flooding is required and such systems are known not
to be scalable.
24
There are several proposals for distributed service discovery based upon distributed
hash tables (DHT) and P2P technologies. [22] proposes the integration of distributed hash
tables with UDDI registries to enable a larger distributed registry; however, they do not
consider governance issues in a cross-domain system such as theirs in the proposal. The
authors of [23] present the use of DHT as a way to enable a scalable service discovery
platform for Grid environments. In [24], a P2P network is used to interconnect registries
using a layered approach. Clients issue queries about available services (which meet their
desired semantics and QoS) to a gateway layer, that is responsible for the translation of
the query into the ontologies of the different types of ontologies supported at the different
registries within the federation, and then passes the query off to a routing layer which
transmits the query onto the appropriate registry. Finally, [25] focuses on the discovery
component of the service composition problem; they utilize Pastry as a P2P service overlay
to find services that can be used in a larger composition. A proposal for using a pub/sub
network as a way for different registries to learn about advertisements and updates to
distributed service information is presented in [26]. Finally, [27] integrates a specific proposal
for storing information about existing UDDI installs inside DNS, and then using DNS
& UDDI together to enable a distributed registry. They do not consider the caching or
replication of registry data for lookups, nor governance of cross-domain situations.
Perhaps closest to our proposal are the following three manuscripts. [28] provides
an architecture similar to ours, but requires use of the UDDI protocol, and does not discuss
the use of policy, convergence of their protocols, or restrictions made on topology. [29]
provides a solution to the multiple domain discovery problem, though it is arguably not
a scalable one due to a possible single-point-of-failure in the service broker, as well as the
dependency of full replication of registry state in the broker. The authors of [30] explicitly
consider a cross-domain service discovery, and use a P2P approach to enable lookups of
services across different domains.
Also relevant to the discussion is the concept of service naming. Service naming
refers to the ability to uniquely identify and address service instances in a service-oriented
architecture. The proposal presented in [31] involves removing the tight coupling between
naming and location that exists in the Internet today; they propose by adding two layers
(service ID and endpoint ID), an architecture that readily accepts mobility of services, data,
25
and hosts can be created. Their naming architecture is flat in nature, and propose using
DHTs to deal with scalability issues in such a system. Similar in nature to this proposal is
the work proposed in [32]. They present a two-layered naming scheme for service lookup and
routing. However, their naming scheme is based on fixed length delimiters and is therefore
less flexible than an XML-based scheme such as our own. A good overview of the service
naming literature is presented in [33].
3.4 Building An Autonomous Federation
Our proposed routing/management protocol for maintaining a distributed service
registry within a single autonomous federation is similar in nature to the Open Shortest
Path First routing protocol [34]. It is also built atop the Web Services Distributed Manage-
ment (WSDM) framework. We envision that a reliable messaging infrastructure, such as
WS-ReliableMessaging or WSRM would be utilized to ensure delivery of messages between
federation members. Also, we expect that a security mechanism, such as mutually authen-
ticated SSL, would be used to ensure communication only occurs between actual federation
members.
The intra-federation routing/management protocol has four main message types:
• Hello: This message is used to establish a connection with peers in the federation; it
also provides a mechanism to detect if a peer is currently reachable or not so that the
distributed registry can be updated appropriately.
• Database Description: Used as an acknowledgement of the Hello message, this
message is used to share the sender’s current view of the topology with the receiver;
it also contains the set of all appropriate exportable service information between the
peers.
• Service State Request: This message is sent to a peer if a federation member needs
information about a particular service.
• Service State Update: This message is sent as a response to a Service State Request
message with relevant information about the requested service, or in a ”push” model
to send updates to service metadata to federation members.
26
Figure 3.4: Message Exchange Between Two ESBs Within a Federation
In the text below and in Figures 3.4-3.9, we provide an example which describes
the semantics of the protocol (and provides examples of message format, etc), and how the
protocol can be utilized to establish and maintain the distributed service registry within an
autonomous federation.
Once peering relationships are extracted from the ESB topology (which is defined
by a system architect), and assuming appropriate policies exist to drive the policy-based
service advertisement function, our protocol can begin running at each federation member.
When an ESB member joins the federation, it sends a Hello message to all other federation
members to which it has a peering relationship - this can be seen in Figures 3.4 & 3.5.
When a federation member receives a Hello message, it consults its policies to
determine what subset of its service registry information it should share with the sender of
the Hello message. Once it has made this decision, it responds to the joining member with
a Database Description message, as seen in Figure 3.6, which contains the appropriate
service information.
The joining member acknowledges the receipt of the Database Description mes-
sage by sending a Database Description message which lists the shared services in the peering