Technische Universität Wien DIPLOMARBEIT … · Technische Universität Wien DIPLOMARBEIT An Evaluation of Enterprise Application Integration Solutions and the Role of Web Services

Technische Universität Wien

DIPLOMARBEIT

An Evaluat ion of Enterpr ise Applicat ion Integrat ion Solut ions and the Role of Web Services

ausgeführt am Institut für

Verteilte Systeme

der Technischen Universität Wien

unter der Anleitung von Ao.Univ.Prof. Dr. Schahram Dustdar

durch

Ulrich Endlich

Langobardenstrasse 44, 1220 Wien Wien, am 21.03.2004

Abstract

The worldwide globalization and the rapidly changing consumer demands challenge the e-Business market. Flexible business processes are the basis to remain competitive. Thus, the way of thinking about information comes from business and not from technology. This causes a growing dependency on information systems entailing larger and more complex IT infrastructures. However, such IT infrastructures consist of many isolated solutions whereby new applications are often newly developed although the required functionalities are already provided by legacy systems. In order to find a key, integration becomes more and more important. Unfortunately, systems are usually integrated via proprietary point-to-point connections to implement one specific process without any adaptability which causes a multiplicity of interconnections. Therefore, integration becomes a major aspect within development issues nowadays which evokes a new buzzword - enterprise application integration, shortly EAI.

The main idea behind an EAI system is that external resources are integrated only once and in between there is a standardized communication-platform to enable seamless interconnections. This allows the definition of business processes across various systems based on their common platform. Nowadays, there are two different competing approaches available to implement such systems. On the one hand, there are dedicated EAI systems that integrate external resources by implementing all their standards. On the other hand, there are Web services that require from integrated applications to implement standards defined within the Web services framework to establish a common communication platform.

Current EAI systems available on the market offer a large set of features that go beyond simple application integration. This involves graphical modeling, generation and execution of business processes across various systems as well as many features to simplify integration tasks. That means simple integration of external resources with the help of specific adaptors, data transformation and mapping capabilities, monitoring facilities and web-based administration features for operational tasks. Furthermore, EAI systems are built on a reliable, scalable, and solid infrastructure to fulfill all requirements of essential business processes.

The Web services framework contains a lot of standards and specifications that can be composed to implement many features of an integration platform. As Web services are designed for distributed systems, most of their mechanisms address aspects of integration. However, many recently published specifications are still in evaluation phase and will need a lot of time to be generally acknowledged. Only if these specifications develop into mature and globally supported standards, Web services will provide a complete enterprise application integration solution.

Although, both technologies are competing, there is a certain convergence between them. This is intensified through the changing of the paradigm from client-server architectures to service-oriented ones. They are based on common communication mechanisms and implement their functionalities by the composition of services. Therefore, services have to implement certain standards to discover, describe and execute themselves to enable a unified and platform-independent infrastructure. Recently published Web services standards implement these functionalities, wherefore current EAI-systems can be called into question. However, an essential contribution for this new architecture isn’t available within the Web services framework, namely the integration of already existing applications that aren’t Web services enabled. Their integration is only possible through their adaptation which is often an unsolvable challenge. This disadvantage is used by traditional EAI systems as this is exactly their benefit. They integrate external resources without the need to modify them. Furthermore, they are able to wrap up Web services functionality around these systems to leverage service-oriented infrastructures. Thus, a certain cooperation and complementation of both technologies is demanded in future.

Zusammenfassung

Die weltweite Globalisierung und die hohen Anforderungen an Flexibilität und Geschwindigkeit stellen neue Herausforderungen an den e-Business-Markt. Geschäftsprozesse müssen rasch den variierenden Kundenwünschen angepasst werden, um im Wettbewerb bestehen zu können. Daher ist die Ausrichtung heutiger Informationssysteme an Geschäftsprozesse anstatt deren Anpassung an die zur Verfügung stehenden Technologien, in den Vordergrund getreten. Da heutige Geschäftsprozesse sehr stark von Informationssystemen abhängen, werden diese immer komplexer, um den Anforderungen gerecht zu werden. Dadurch ist eine Landschaft von vielen heterogenen unabhängigen Systemen entstanden. Neue Anwendungen werden, obwohl Teile der Funktionalitäten oftmals schon in verschiedenen Systemen vorhanden sind, von Grund auf neu entwickelt, wodurch Funktionalitäten und Datenbestände oftmals redundant vorliegen. Ein Fakt, der nicht gerade für Wettbewerbsfähigkeit sorgt. Um dieser Tendenz entgegenzuwirken werden die einzelnen Systeme und Funktionalitäten integriert. Diese Integration basiert aber meistens auf dedizierten Punkt-zu-Punkt Verbindungen, die eine Wiederverwendung der Schnittstelle nicht ermöglichen, wodurch eine Vielzahl von Schnittstellen und Verbindungen entstehen, die einer aufwändigen und kostspieligen Wartung bedürfen. Integration spielt daher heutzutage eine wichtige Rolle in der Entwicklung von Anwendungen, wodurch sich ein neues Schlagwort verbreitet hat – Enterprise Application Integration, kurz EAI.

Das Grundkonzept besteht darin, Systeme einmal zu integrieren und dazwischen eine gemeinsame standardisierte Plattform zu schaffen über die die integrierten Systeme untereinander verbunden sind. Dies ermöglicht Geschäftsprozesse auf dieser Ebene zu definieren, und dadurch übergreifend über die verschiedenen Systeme abzubilden. Um dies zu bewerkstelligen, gibt es zwei verschiedene Ansätze, die zueinander im engen Wettbewerb stehen. Einerseits gibt es proprietäre EAI Systeme, die externe Systeme integrieren indem sie sich deren Schnittstellen über spezielle Adaptoren anpassen und andererseits etablieren sich die Web Services Standards, die den Applikationen ihre Standards aufdrängen, um eine gemeinsame Kommunikationsbasis zu schaffen.

Die heute am Markt verfügbaren EAI Systeme besitzen eine Vielzahl von Funktionen, die weit über einfache Applikationsintegration hinausgehen. Diese umfassen das grafische Modellieren und Umsetzen von Geschäftsprozessen über Systemgrenzen hinaus und weitestgehende Unterstützung bei der Implementierung von Integrationslösungen mit all ihren Aspekten, um deren Implementierung signifikant zu vereinfachen. Dazu zählen die einfache Integration externer Systeme mit speziellen Adaptoren, Möglichkeiten für Datentransformationen, Monitoringfähigkeiten zur Überwachung ablaufender Geschäftsprozesse und web-basiertes Systemmanagement. Außerdem unterliegt nahezu jedem EAI-System eine zuverlässige, skalierbare und stabile Infrastruktur, um die hohen Anforderungen an essentielle Geschäftsprozesse zu gewährleisten

Das Web Services Framework besteht aus einer Vielzahl von Standards und Spezifikationen, die gemeinsam eine große Anzahl von Funktionen einer Integrationsplattform ermöglichen. Die meisten der vorgestellten Mechanismen adressieren Integrationsaspekte verteilter Systeme. Allerdings sind einige dieser Spezifikationen erst in einem Evaluierungsstadium und noch weit davon entfernt, allgemein anerkannt zu werden. Erst wenn sich daraus ausgereifte und global unterstützte Standards entwickelt haben, werden Web Services vollständige unternehmensweite Integrationslösungen bieten.

Obwohl die beiden Technologien oft als im Wettbewerb stehend angesehen werden, ist die Konvergenz zwischen diesen Technologien nicht zu verleugnen. Zusätzlich wird diese durch das Aufkommen des Paradigmenwechsels von Client-Server-Architekturen zu Service-Orientierten Architekturen verstärkt. Diese Bauen auf einem gemeinsamen Kommunikationsmechanismus auf und bilden Funktionalitäten durch Kompositionen einzelner so genannter Services ab. Dies verlangt aber den Services eine Einhaltung gewisser Standards wie deren Findung, Beschreibung und Benutzung ab, um eine einheitliche und plattformunabhängige Infrastruktur zu ermöglichen. Neue Web Services Standards versuchen genau diese Anforderungen abzubilden, weshalb aktuelle EAI-Systeme manchmal in Frage gestellt werden. Einen wesentlichen Beitrag zu dieser neuen Architektur können Web Services aber nicht anbieten, nämlich die Integration bereits bestehender Anwendungen, die noch keine Web Services Schnittstellen besitzen. Deren Einbindung kann nur durch deren Adaptierung gewährleistet werden, was allerdings oftmals aufgrund von veralteten Technologien und Systemen eine fast unlösbare Herausforderung darstellt. Diesen Nachteil nützen EAI–Systeme zu Ihrem Vorteil, indem sie die Einbindung solcher Systeme ermöglichen, ohne diese modifizieren zu müssen. Weiters können den somit integrierten Funktionalitäten Web Service Standards aufgesetzt werden, um den ganzheitlichen service-orientierten Ansatz zu ermöglichen. Daher wird die Zukunft wohl verstärkt eine gewisse Kooperation und ein gemeinsames Auftreten dieser Technologien verlangen.

Contents

Abstract........................................................................................................................... II

Zusammenfassung ........................................................................................................IV

Contents .........................................................................................................................VI

Index of illustrations.....................................................................................................IX

1 Problem Description ................................................................................................ 1 1.1 Introduction .............................................................................................................................. 1 1.2 Problem Definition and Motivation ........................................................................................ 3

2 Review of the State of the art .................................................................................. 4 2.1 Levels of Integration................................................................................................................. 4 2.1.1 Platform or Infrastructure Integration......................................................................................... 4 2.1.2 Data integration .......................................................................................................................... 4 2.1.3 Application Integration............................................................................................................... 5 2.1.4 Business Process integration ...................................................................................................... 6 2.2 Middleware ............................................................................................................................... 7 2.2.1 Message Queuing ....................................................................................................................... 7 2.2.2 Publish-and-subscribe................................................................................................................. 8 2.3 Enterprise Application Integration - EAI .............................................................................. 9 2.4 EAI Architectures..................................................................................................................... 9 2.4.1 Hub-and-spoke ........................................................................................................................... 9 2.4.2 Federated .................................................................................................................................. 11 2.4.3 Bus topology............................................................................................................................. 11 2.5 EAI Components .................................................................................................................... 13 2.5.1 Connectors................................................................................................................................ 13 2.5.2 Messaging Middleware ............................................................................................................ 14 2.5.3 Data mapping and transformation ............................................................................................ 14 2.5.4 Process management ................................................................................................................ 14 2.5.5 Workflow tasks......................................................................................................................... 15 2.5.6 Administration and monitoring ................................................................................................ 16 2.6 Web services............................................................................................................................ 17 2.6.1 Service-oriented architectures .................................................................................................. 17 2.6.2 A technical overview................................................................................................................ 18 2.6.3 WSDL....................................................................................................................................... 20 2.6.4 SOAP........................................................................................................................................ 21 2.6.5 UDDI........................................................................................................................................ 21 2.6.6 Business Process Execution Language for Web services ......................................................... 21 2.6.7 Web services Composite Application Framework ................................................................... 24 2.6.8 Web services Security .............................................................................................................. 25 2.6.9 Web services Notification ........................................................................................................ 26 2.7 Web services and Integration ................................................................................................ 27

3 Evaluation of EAI products................................................................................... 30 3.1 BEA WebLogic Platform ....................................................................................................... 30 3.1.1 Installation and configuration................................................................................................... 31 3.1.2 WebLogic Integration overview............................................................................................... 31 3.1.3 WebLogic Workshop................................................................................................................ 32 3.1.4 WebLogic Control Architecture ............................................................................................... 33 3.1.5 Business Process Management................................................................................................. 34 3.1.6 Data Transformation................................................................................................................. 36 3.1.7 Message Broker ........................................................................................................................ 37 3.1.8 Application Integration............................................................................................................. 37 3.1.9 Human collaboration ................................................................................................................ 37 3.1.10 Management and Administration ............................................................................................. 38 3.1.11 Conclusion................................................................................................................................ 39 3.2 Microsoft BizTalk 2002.......................................................................................................... 40 3.2.1 Conceptual overview................................................................................................................ 40 3.2.2 Installation and configuration................................................................................................... 41 3.2.3 BizTalk Orchestration Services ................................................................................................ 41 3.2.4 BizTalk Messaging Services .................................................................................................... 43 3.2.5 Management and Administration ............................................................................................. 47 3.2.6 Conclusion................................................................................................................................ 48 3.3 webMethods integration platform ........................................................................................ 50 3.3.1 webMethods integration platform overview............................................................................. 50 3.3.2 Platform architecture ................................................................................................................ 51 3.3.3 Installation and configuration................................................................................................... 52 3.3.4 webMethods Developer............................................................................................................ 52 3.3.5 Workflow Modeler ................................................................................................................... 53 3.3.6 Data Transformation................................................................................................................. 54 3.3.7 Message Broker ........................................................................................................................ 55 3.3.8 Adapters ................................................................................................................................... 55 3.3.9 Human workflow...................................................................................................................... 56 3.3.10 Management and Administration ............................................................................................. 56 3.3.11 Conclusion................................................................................................................................ 58 3.4 Product taxonomy .................................................................................................................. 58

4 Case study ............................................................................................................... 61 4.1 The problem............................................................................................................................ 61 4.1.1 The environment....................................................................................................................... 61 4.1.2 The claim of the management .................................................................................................. 61 4.1.3 Analysis of a flawed process .................................................................................................... 61 4.1.4 The challenge ........................................................................................................................... 62 4.2 Solution strategies................................................................................................................... 63 4.2.1 Implementation of the EAI based solution ............................................................................... 63 4.2.2 A solution approach based on Web services ............................................................................ 63 4.2.3 Web services within EAI projects ............................................................................................ 64 4.3 The solution based on the webMethods Integration platform............................................ 65 4.3.1 Description ............................................................................................................................... 65

4.3.2 Context ..................................................................................................................................... 65 4.3.3 Scope ........................................................................................................................................ 65 4.3.4 Risks ......................................................................................................................................... 65 4.4 System Description ................................................................................................................. 66 4.4.1 Functional Requirements.......................................................................................................... 66 4.4.2 Design constraints .................................................................................................................... 66 4.4.3 Hardware environment ............................................................................................................. 66 4.4.4 Software environment .............................................................................................................. 67 4.4.5 Network environment............................................................................................................... 67 4.5 Logical Design......................................................................................................................... 67 4.5.1 Scenario: Shop exchange request ............................................................................................. 68 4.5.2 Scenario: Create new debtor..................................................................................................... 70 4.5.3 Scenario: Notify administrator group ....................................................................................... 71 4.5.4 Scenario: Acknowledgment of requests ................................................................................... 71 4.5.5 Scenario: Modify Partner data.................................................................................................. 72 4.5.6 Scenario: Create export document............................................................................................ 73 4.5.7 Scenario: Send success mail ..................................................................................................... 73 4.6 The business process diagram ............................................................................................... 74 4.7 Detailed Design ....................................................................................................................... 75 4.7.1 Scenario: Shop exchange request ............................................................................................. 75 4.7.2 Scenario: Create new debtor..................................................................................................... 76 4.7.3 Scenario: Notify administrator group ....................................................................................... 78 4.7.4 Scenario: Acknowledgement of requests.................................................................................. 78 4.7.5 Scenario: Modify Partner data.................................................................................................. 80 4.7.6 Scenario: Create export document............................................................................................ 82 4.7.7 Scenario: Send success mail ..................................................................................................... 84 4.8 Summary ................................................................................................................................. 84

5 Conclusion and future work.................................................................................. 85

6 References ............................................................................................................... 86

Index of illustrations

Figure 2.1 Data integration level ................................................................................................................. 4 Figure 2.2 Application Integration with API’s ............................................................................................ 5 Figure 2.3 A simple business process model ............................................................................................... 6 Figure 2.4 Message Queuing ....................................................................................................................... 7 Figure 2.5 A publish/subscribe system ........................................................................................................ 8 Figure 2.6 The “hub-and-spoke”-architecture ........................................................................................... 10 Figure 2.7 The “federated”-architecture .................................................................................................... 11 Figure 2.8 The bus-architecture ................................................................................................................. 12 Figure 2.9 Cooperation of EAI components .............................................................................................. 13 Figure 2.10 Process management .............................................................................................................. 15 Figure 2.11 Basic service-oriented service interaction .............................................................................. 17 Figure 2.12 Web services layers ................................................................................................................ 18 Figure 2.13 Basic Web services interaction............................................................................................... 19 Figure 2.14 BPEL process interactions with partners................................................................................ 22 Figure 2.15 WS-Context overview ............................................................................................................ 24 Figure 2.16 A possible service-oriented architecture based on Web services and EAI ............................. 28 Figure 3.1 BEA WebLogic Platform architecture [53].............................................................................. 30 Figure 3.2 BEA WebLogic Workshop....................................................................................................... 32 Figure 3.3 A clipping of a business process model in Workshop .............................................................. 34 Figure 3.4 BEA WebLogic XQuery Data Mapper..................................................................................... 36 Figure 3.5 Monitoring................................................................................................................................ 38 Figure 3.6 Administration.......................................................................................................................... 39 Figure 3.7 Microsoft BizTalk Server architecture ..................................................................................... 40 Figure 3.8 Workflow (left) and Implementation (right) view.................................................................... 42 Figure 3.9 Integration of SAP into BizTalk............................................................................................... 44 Figure 3.10 BizTalk Editor ........................................................................................................................ 45 Figure 3.11 BizTalk Mapper...................................................................................................................... 45 Figure 3.12 Message flow between receive functions, channels and messaging ports .............................. 46 Figure 3.13 The XLANG Event Monitor................................................................................................... 47 Figure 3.14 BizTalk Document Tracking Query Builder .......................................................................... 48 Figure 3.15 A business process in BizTalk Orchestration Designer.......................................................... 49 Figure 3.16 webMethods architecture - an example .................................................................................. 51 Figure 3.17 Workflow Modeler ................................................................................................................. 53 Figure 3.18 Graphical data mapping.......................................................................................................... 54 Figure 3.19 webMethods Administrator .................................................................................................... 56 Figure 3.20 webMethods Monitor ............................................................................................................. 57 Figure 4.1 The business process today ...................................................................................................... 62 Figure 4.2 Logical Design Use Case diagram............................................................................................ 67 Figure 4.3 Shop partner exchange request form ........................................................................................ 69 Figure 4.4 Web service request activity diagram....................................................................................... 69 Figure 4.5 Create new debtor activity diagram.......................................................................................... 70 Figure 4.6 Send notification mail to administrators activity diagram........................................................ 71 Figure 4.7 Acknowledge or decline requests activity diagram .................................................................. 71

Figure 4.8 Modify partner data activity diagram ....................................................................................... 72 Figure 4.9 Create export document activity diagram................................................................................. 73 Figure 4.10 Send success mail activity diagram........................................................................................ 73 Figure 4.11 Business process model .......................................................................................................... 74

Problem Description

Page 1

1 Problem Description

1.1 Introduction

Nowadays, corporate dependencies on information technologies are growing and getting more and more complex. Since IT provides the knowledge backbone of an enterprise, these systems are highly valuable for companies. As the market and therefore business processes are changing rapidly, systems have to be migrated or newly developed to provide the functionalities necessary for today’s e-business strategies. Often, these functionalities are already provided by legacy systems or are distributed across various systems caused by departmental implementation of IT systems. Therefore, a seamless interconnection for information exchange and collaboration across various resources is getting more important as it provides a fast and simple way to meet new requirements. Most of these connections are built proprietary and are usually only appropriate for one particular process which results in a maze of interconnections and increasing maintenance costs. Hence, the consolidation and integration of disparate systems into a unified IT-infrastructure brings along several advantages, for example, business process automation, human collaboration, integration of legacy systems into new technologies, real time data transformations, etc. Furthermore, integration is required to deal with new business-to-business strategies, growing intranets as well as mergers and acquisitions.

Theses days, two different approaches for achieving such a unified information-system are under discussion. On the one hand, there are Enterprise Application Integration (EAI) systems. They support the integration of business processes and data across disparate applications. EAI tools are expensive in the acquisition, but the return on invest is fast as they provide the ability to respond, adapt and collaborate quickly and relatively inexpensively with innovations. On the other hand, the popularity of the internet and the growing e-business market causes the standardization of communication protocols. On top of these standards are the well known Web services. They facilitate communication between various systems in different environments. Web services protocols are all driven by the most recent used message format standard - XML. All these standards offer messaging between heterogeneous applications and permit their integration. EAI systems and Web services technologies follow up the same approach, but what are the differences. In my opinion, EAI systems are containing a lot of valuable out-of-the-box features, which give them a competitive edge.

Problem Description

Page 2

The organization of this thesis is as follows: • In the first chapter you will find an introduction about what this thesis is about. It

explains the development from the beginning to the actual situation and the problems arising from it. Then it discusses the motivation and goal of this thesis.

• Chapter two is a review of the state of the art. It explains the underlying EAI and Web services technologies by starting with an overview of the concepts behind. Moreover, it describes the details of them and finally it brings a conclusion about the role of Web services within EAI.

• The third chapter discusses three EAI solutions from different vendors available on the market. It introduces their architecture and the different approaches to solve enterprise wide integration problems.

• Chapter four introduces a case study conducted within an Austrian corporation. It starts with the description of a business process as it is used nowadays. Then the environment and the basic conditions, which are dependent on the existing infrastructure, are illustrated. Afterwards, it describes the desired solution and discusses solution approaches based on EAI tools and on Web services. Furthermore, the roles of Web services within integration projects are discussed.

• The fifth chapter gives a conclusion of this thesis and presents some future work. • Finally, chapter six lists the references and literature used for this work.

Problem Description

Page 3

1.2 Problem Definition and Motivation

EAI provides a common integration framework, enabling companies to integrate business processes, workflows and data across various applications. But in times of standardized communication mechanisms like Web services, which are built-in by nearly every new application, are EAI-Tools still needed? In my opinion, they are. The strengths of EAI-tools still remain. EAI acts as a middleware between systems, and have the capability to enable communication between those systems, providing transaction integrity, message format translation and offering complex workflow mechanisms.

In contrast to EAI, Web services provide a large set of standards which can be combined to attain the same functionality as EAI systems have to offer. They connect systems in a standardized way, define the communication protocols, and allow for possibilities to implement transactions, reliable messaging and flow control. This master’s thesis will evaluate the strengths of EAI-tools and discuss the capabilities Web services are offering for integration tasks.

Review of the State of the art

2 Review of the State of the art

The objective of this chapter is the description of the discussed concepts and techniques. It starts with an overview of the involved technologies followed by a detailed evaluation about common EAI frameworks. The second part describes Web services and their effort to support integration. The end of this chapter points out that the Web services technology cannot fulfill all the EAI requirements, however it can be used to generate the adaptors which are subject to a traditional EAI solution and help to integrate new Web services enabled applications fast and inexpensively.

2.1 Levels of Integration

Before discussing EAI principles, some of the basics of integration are introduced. There are different types of integration levels provided by EAI systems [1] explaining the various dimensions of integration tasks [2][3].

2.1.1 Platform or Infrastructure Integration The lowest level is responsible for the integration of the underlying infrastructure.

This includes heterogeneous software and hardware, like operating systems, database management systems, server and network. Therefore, processes and tools are required that allow these systems to communicate, so that data can be sent between different applications and systems, for example, to pass information reliably from an NT machine to a UNIX machine.

2.1.2 Data integration Data integration is essential in most integration projects. It is responsible for making

data accessible from one application to another. Therefore, this level has to provide data transport and data transformation services. Data integration is a low-level integration which means that knowledge of the application logic is not required. The applications only share data, but they don’t understand each other, so this integration avoids the changing of application logic. The data transfer circumvents application logic by directly reading and writing data from the data store used by the applications (see Figure 2.1).

Figure 2.1 Data integration level

Application 1 Application 2

Data Store 1 Data Store 2 Data integration

Page 4


Usually, this doesn’t sound demanding as standard databases are often used and their interfaces are well known. Although, this seems to be manageable, often transactional consistency is ensured within application code, instead of the database management system facilities being used. So a data integrator has access to inconsistent data too, which can result in unwanted conditions. Another problem is the usage of proprietary data formats, which can be easily misinterpreted. Additionally, applications often use metadata, transparent for the user, for advanced handling and transaction control. Ignoring these data can also cause errors, in both, the source and the destination application.

Also, syntactic and semantic data incompatibility makes the integration more complicated [4]. Syntactic incompatibility means different data formats and structures. For example, the representation of floating point data may be a character string or a binary type. Syntactical conversions often remain simple, but if semantic transformations are necessary, it can become very complex, for example, if there are several data sources combined and transformed into a new record format.While this integration level is one of the easiest and cheapest to implement in contrast to other integration forms, all these problems illustrate the various challenges of data integration.Well-known data integration applications are data warehouses, which populate the extracted data into a common database.

2.1.3 Application Integration The goal of this integration level is to provide application-to-application integration

of cross-platform applications over a network [5]. This can be achieved via various techniques. One of these techniques includes the usage of application program interfaces (API’s) that exist within the applications. Using these interfaces, applications can be bind together, letting them share application logic and application data (see Figure 2.2). Their limitations are that the API specific functions are predefined by the developer and vary widely in number and quality. If a required function isn’t accessible via an API, the changing of the application cannot be avoided which means that therefore the integration and maintenance costs will rapidly increase.

Figure 2.2 Application Integration with API’s

Application integration can also be realized through Web services. They follow the

same approach as API’s by offering application logic through a predefined interface, but they make use of standard technologies to simplify the integration task. Another approach to get in contact with an application is the usage of common shared objects. This is often referred to as method-level integration and means the sharing of application logic with the help of, for example, distributed objects, based on e.g.,

Check and modify data in database

Page 5

CORBA or COM+. Various companies have been practicing this reuse of application logic for years, but this reuse finds its way only in custom applications.

Application 1

API call API call

Application 2


Hence, the usage of existing interfaces and methods avoids the changing of applications and minimizes the integration costs. However, some commercial and n

2.1.4 The highest level integrates business processes involving diverse enterprise business

s nderlying integration levels. It is about the e

business pro

business process integration. This is done by a process broker or a workflow engine, which is responsible for the execution of th

early every custom applications don’t provide any type of interface that allows their integration. This increases the cost and the risk to adapt these applications.

Business Process integration

ystems. Its realization requires all uxchange of enterprise information through the semantics of a business process view

[6], including process definition, process modelling, process execution and process management tasks [7]. After the process definition has been done by a business analyst, the process modelling comes into operation. It describes the flow of information inside a business process. This includes the drawing of the process interactions, the description of the data flow and the mapping of these blocks to implementation resources (see Figure 2.3). As the process model is a high-level view of the process, it hides all the specific data mappings and implementation details of lower integration levels.

Application Application Application

Step 2a

Step 2b

Step 3 Step 1

Human interaction

Figure 2.3 A simple cess model

The process execution is the major task of

e individual process steps. Therefore, it manages the control flow of the business process. Additionally, the workflow engine must also be able to track a step-by-step progression of any running business process instance to deal with failed or delayed steps. Process integration doesn’t finish after definition and execution. It also provides the management of the business process. The process management monitors business process activities and tracks detailed process statistics to optimize business processes subsequently.

Page 6


2.2 Middleware

Middleware tools are necessary for the realization of the different integration levels, which were discussed in the previous section. They provide the connectivity between network services, applications and business processes in a distributed computing system [13][14][15][16]. Middleware makes application development easier by hiding low-level programming details and providing common programming abstractions. It creates a platform and network transparency by hiding the heterogeneity and the distribution of the underlying hardware and operating systems [8][9][10][11].

Message-oriented middleware (MOM) is designed specifically to handle the complexities of exchanging information within a distributed environment based on guaranteed message delivery [12]. Therefore, it has to provide consistent, reliable and secure transport mechanisms. Analogue to the different integration issues, different messaging technologies are being used. There are two types of communication – synchronous and asynchronous.

Synchronous interaction is similar to typical client/server transactions. The message is sent to a recipient, and then a response is awaited. The sending application is blocked during the whole process of message delivery. Both, consumer and producer of a message are tightly coupled and must run simultaneously for successful message delivery.

The asynchronous mechanism decouples sender and recipient of a message. The producer sends the message and is able to continue processing. The sent messages are queued by a separate process until the recipient is available. Asynchronous middleware is usually a reasonable implementation choice in loosely coupled distributed architectures. Two messaging architectures, message queuing and publish/subscribe, have been established in EAI solutions at present and are illustrated in the following sections.

2.2.1 Message Queuing Message Queuing is a technology, in which messages are appended asynchronously

into message queues by the sender. The consumer receives the messages from these message queues (see Figure 2.4).

Figure 2.4 Message Queuing

MQ System Producer Consumer

Page 7

Producer Consumer

Producer Queue 1 Consumer

Queue 2


There can be multiple producers as well as multiple consumers attached to a single queue [17][18][19]. Messages can be successfully submitted to a queue even if the consumers for that queue are not running or are unreachable. The MOM infrastructure ensures that messages are reliably delivered, for example, by making queues persistent. The delivery order of the messages is generally first-in first-out (FIFO) or priority-based, thus, the consumer synchronously pulls messages from the queues. Since producers and consumers aren't directly interconnected, the producer is able to continue processing. Some of the popular products are IBM’s WebSphere MQ [20] and Microsoft Message Queuing (MSMQ) [21].

2.2.2 Publish-and-subscribe In publish-and-subscribe communication the producer (publisher) has to publish

messages without explicitly specifying recipients. Similarly, the consumers (subscribers) have to receive only those messages that they are interested in without the knowledge of the publisher. Hence, the publisher and the subscriber are said to be loosely-coupled [22][23].

In this event-based mechanism the publisher sends a message (an event) to an event service and all the previously subscribed systems receive the data immediately without additional request. Therefore, a subscriber has to subscribe to events he is interested in, to be notified subsequently of matching events. The event service is often called message broker, which is in fact an optimized publish/subscribe server that on receipt of any new message figures out which subscribers are interested in the message and how to deliver it to them (see Figure 2.5). New subscribers and new publishers can be added to the system anytime without interruption.

Figure 2.5 A publish/subscribe system

These systems are classified by the scheme of subscription. The usually used

schemes are topic-based and content-based publish/subscribe systems. In topic-based systems the publisher associates a topic or subject with his message and the subscriber subscribes to specific topics to get connected. Since the topic-based scheme is very static most systems allow wildcards within topic names to subscribe and publish to several topics [24]. Furthermore, hierarchical addressing is used in which topics are hierarchically related so that all sub-topics of a subscribed node are automatically subscribed, too.

Page 8

Message Broker

Publisher

Subscriber Subscribe

Subscriber Publish

Publisher Notify

Subscriber


Usually message brokers introduce channel-based publish/subscribe which is a mapping of individual topics to distinct channels. Content-based systems offer subscribers to express a query against the content of messages published. This is done by specifying constraints and filters using a subscription language to gain maximum expressiveness for determining the match between an event and a subscription. The disadvantage of content-based systems is that they require sophisticated protocols and complex matching processes which results in protocol overhead and slowness.

2.3 Enterprise Application Integration - EAI

The vast majority of corporations use several generations of different systems and applications developed over many years to fulfill their needs. These systems are all providing some value to the enterprise and are building the backbone of the corporation [25]. Most of them can’t be simply replaced through new or other systems, as huge investments would be necessary. The integration of so called legacy systems into the non-rigid application architecture of the corporation is obliged. Also economic reasons entail that an enterprise must be able to react fast and flexibly to new customer requirements, without large costs. All these functionalities are offered by Enterprise Application Integration systems [26][27]. The main objective of an EAI system is to integrate heterogeneous systems to make them acting like one to realize complex business processes [28]. They promise an easy, flexible, cost-effective and future-proofed integration [29]. There are a lot of definitions available for EAI, but none is universally valid, but EAI stands for a technology, which enables automated communication and interoperability between different applications and business processes within the corporation and between trading partners [30][31].

2.4 EAI Architectures

Basically there are three different types of EAI architectures available. The first one is the “hub-and-spoke“-architecture, the second one is called “federated". The third architecture is based on a bus topology. A short introduction will show the different paradigms behind them, and their assets and drawbacks.

2.4.1 Hub-and-spoke Here EAI acts centralized between the connected applications. In this commonly

used architecture the EAI tool works like an information hub. As shown in Figure 2.6 every application has to be connected only once to provide integration with the centralized hub, which reduces the integration complexity. Applications interacting with others through the EAI hub, without direct connections between them. EAI performs the data transformations and controls the information flow to realize centralized automation of business processes.

Page 9


Application 2

EAI Tool

Page 10

Figure 2.6 The “hub-and-spoke”-architecture This widespread architecture leverages many value added services:

• Minimal amount of non-invasive interfaces Only two interfaces are required to connect one application to every other linked system. One is found within the application and its complement is offered by the EAI system. Because almost every application provides an interface, a non-invasive integration is possible.

• Synchronous/asynchronous communication Depending on the connected application the EAI tool provides different types of communication protocols.

• Business Process oriented The control of all integrated systems by the hub facilitates the business process oriented approach, because such processes can be easily mapped, managed and controlled by one centralized workflow engine.

• Facilitates publish/subscribe models Such models are requiring a common entity to publish events to, and to subscribe to these events. The central hub meets these requirements.

• Provides reuse and less implementation complexity Because there is only one place where integration occurs, workflows are defined, and data transformations take place, parts of these components can be easily reused and grant fast and flexible implementations.

• Centralized management/administration It provides a consistent interface management and allows a unique central administration to control and maintain the whole system.

On the other hand this centralization constitutes, like other centralized architectures,

a single point of failure. It also defines the bottleneck of the complete system and is to blame for causing performance problems. The integrated systems’ reliability depend on the EAI hub. Therefore, reliability and performance have to be achieved through cost intensive redundancy and clustering.


Application 4


2.4.2 Federated The federated architecture distinguishes itself that there is no centralized EAI-server.

Data transformations and workflow controlling is directly embedded within the applications via a module. Integration between systems has to be realized through direct linking (see Figure 2.7). The communication between theses systems uses a bus topology with product-dependent message formats.

Figure 2.7 The “federated”-architecture “Federation” circumvents the hub-and-spoke drawbacks:

Application 2

EAI Module

Page 11

• Avoidance of a single-point-of-failure increases the reliability of the system • No performance bottleneck due to the lack of a central component • no huge investments in hardware clusters

This architecture is suited only for smaller integration projects for the very reason

that they don’t follow the idea behind enterprise integration as there are still point-to-point connections. So most of the available products are underlying the other architectures or consisting of a combination of them.

2.4.3 Bus topology Some of the vendors are using a bus topology to realize communication between

different systems and the EAI system. A drawback of this architecture is, that there has to be a module installed on every integrated system. This component is responsible for the linkage between the integrated system and the bus, which connects all integrated applications. One central EAI server manages all the workflows and data transformations (see Figure 2.8).

Application 1

EAI Module

Application 3

EAI Module

EAI Module

Application 4



Page 12

Figure 2.8 The bus-architecture The advantages of this topology are as follows:

• Scalability The bus topology is a very scalable platform, where the amount of integrated systems has no impact on the performance.

• High performance Often a publish/subscribe model is implemented to realize real time communication. When one system publishes an event all subscribed applications retrieve this event nearly simultaneously.

• Distributed architecture Eliminates bottlenecks and single points of failure and offers possibilities to realize fault-tolerance and load-balancing

• Centralized management One central server to manage and control all the distributed processes

BUS

Application 3

EAI Server

Application 4


2.5 EAI Components

Behind the scenes there are more components embedded in the integration system. Applications or legacy systems are linked to the EAI tool by so called connectors or adaptors. The communication to the integration server takes place via a middleware. The integration server performs data transformations between applications and processes them following a previously defined business process (see Figure 2.9).

Figure 2.9 Cooperation of EAI components

Page 13

2.5.1 Connectors Connections between applications and the EAI framework are built with connectors,

which are normalizing the access and the interaction. The connection can be configured through a set of parameters, depending on the adaptor and the type of the integrated system. Communication with the system is realized through the application dependent API. Such an API provides a set of well defined methods to get in contact with the application data and functionality. The adaptor acts with the application using this API like a usual application client. The EAI tool behind the connector is transparent for the application. The adaptor transforms the data format used within the API into a tool specific generic data format to realize data exchange between different systems. Typically, a block of data used for API in- and output is not self-describing. It is incorporated into the application code [32]. To build a generic format it is transformed by the adaptor, which knows the API data format to realize communication, into a structured, self-describing format. The eXtensible Markup Language (XML) [33] is preferred. It was designed to describe, structure and carry data, and is nowadays established as a common, standardized language. Most EAI tools are delivered with up to 150 different standard adaptors for typical applications like Ftp, Email, SAP, Oracle, MS SQL, Unix File system, NT File system, Web services, and so on, implementing the specific APIs. These adapters assist the connection establishment from the EAI run-time system to the related application [34].

Process management

D

ata

tran

sfor

mat

ions

Application

Processing engine

Connector

Messaging Middleware

Administration/Development

Data Bus (XML)

API


Development of custom adapters for applications not supported out-of-the-box is also possible within every EAI system. Therefore, proprietary development kits are provided by the vendors to simplify such an implementation and provide common adapter functionality, such as sending and receiving data, configuration and monitoring of adaptors.

2.5.2 Messaging Middleware The messaging middleware is responsible for the data integration. It securely

transfers the data between the adaptors and the integration server to provide data integrity. The middleware provides synchronous and asynchronous communication and make point-to-point, publish-and-subscribe and message queuing mechanisms available. Most systems are using asynchronous messaging queuing or publish-and-subscribe interaction as described in section 2.2.

2.5.3 Data mapping and transformation When data from an integrated application is sent, for example, in XML format, it has

to be mapped and transformed into a format, the endpoint understands. This processing is either done directly within the connector, inside the messaging middleware or in the processing engine. XML data can be easily adapted to the requirements with the help of the Extended Stylesheet Language Transformations (XSLT) [35]. These transformations take place within the EAI system and are transparent for the operator. Nearly every EAI tool provides a graphical user interface which makes the data manipulation very easy, by showing the available input data elements and the mandatory output data fields. Then these fields are connected via drag and drop to map them. It is also possible to add additional transformations. For example if one has to collect three input fields into one output field or to change the date format. Typical transformations are collected in libraries and are often delivered by EAI vendors within the product, but every product allows building complex transformations by writing code and adding it to the library for reuse. Some of the tools provide GUI-driven mapping and transformations which minimizes the need of programming a single line of code in the majority of cases, and don’t require understanding of the underlying techniques and standards (XML, XSLT, Java …)

2.5.4 Process management A core feature of EAI systems is the process management service. It defines and

controls the automatic interaction of complex transactions based on business processes between different systems (see Figure 2.10). If the definition of a business process has finished, it can be modelled in the EAI system with the help of graphical modelling tools, but all of them allow editing the generated code and implementing more sophisticated procedures. The solutions vary in level of supported details, but all of them follow the divide and conquer paradigm. They allow granulating processes and dividing them in sub processes. This simplifies the complexity of processes and allows distributed process modelling, which means that everyone is only responsible for a certain part. One supervisor, who doesn’t have to understand the sub process details, has to build the main process controlling all the integrated sub processes.

Page 14


Another benefit of this granularity is the reusability of such sub processes; for example the sub process “send an email” is a typical process which needs only to be defined in a generic format once, so that it can be reused in all other processes. Transaction controlling is also an important characteristic [36]. A transaction represents a set of steps forming a logical unit. Either all of these steps are being performed or none of them. If one step fails, all previously performed steps have to be revoked. Other features offered by most vendors are automatic exception handling, logging and recovery mechanisms.

Figure 2.10 Process management

Processing engine

Process

Application 1 Application 2 1

Page 15

2.5.5 Workflow tasks Business process automation reduces the level of human involvement, but many

processes require human interaction. This could be realized with additional workflow capabilities. Most tools on the market are offering such workflow features. Some tools are integrating these directly into the process modeller, and some are providing them through external tools, delivered within the software bundle. This allows the realization of a seamless integration of human control into business processes. Task specific data from back end applications can be directly provided to the human controlled front end, to address situations requiring human judgment. It is also often necessary to manually handle exceptions and errors, and to model processes involving approval and review tasks.

Service 1

Service 2

Service 1

3

Service 2 4 5

6

2


2.5.6 Administration and monitoring Defining and executing business processes involving some integrated systems

doesn’t finish the EAI activities. In order to respond rapidly to the needs of the business, processes need to be managed, optimized and adapted. Though, an important part is the administration and monitoring facility of an EAI tool to fulfill the requirements. It makes business visible and helps to analyze systems to optimize processes and find bottlenecks within the infrastructure. Also the involvement of many fault-prone sub systems requires a centralized powerful monitoring tool to locate sources of error immediately. Nearly every tool provides these features via a web-based console for fast client and location independent access. The monitoring data is often stored in a database for later reviewing of processes and their data.

Page 16


2.6 Web services

An introduction into Web services technology [37][38][39] will show their functionalities and domains and their contribution to EAI. Often Web services are praised as the new integration standard. The following sections are going to discuss their approach to realize integration solutions and compare them to pure EAI systems.

2.6.1 Service-oriented architectures A service-oriented architecture (SOA) is basically a set of services [40]. These

services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity. Therefore, the connection of services to each other is required. Service-oriented architectures are not a new model for architecting software and IT infrastructure. The first service-oriented architectures were DCOM and Object Request Brokers (ORBs) made up of the CORBA specification, but recently established standards and ubiquitous network technologies provide the technology support for this trend.

A service itself is a function that is well-defined, self-contained, and does not depend on the context or state of other services. The basic service interaction within a service-oriented architecture is illustrated in Figure 2.11. A consumer service sends a service request message to a service provider. The service provider returns a response message to the consumer.

Figure 2.11 Basic service-oriented service interaction Within SOAs services are the building blocks which can be composed to create new

applications. So, the composition of multiple services becomes the main issue of the application development process. A service composition combines services to achieve business requirements or to provide new service functions which may themselves be composed to provide business functionality in a recursive matter. Such a service composition provides the seamless integration of applications within the enterprise and between trading partners (business-to-business). The SOA makes interoperability and agility a major issue with system design. Furthermore, it assumes that the system has integration capabilities from the beginning, rather than adding them subsequently.

This emerging architecture defines a set of requirements that distinguishes it from other computing paradigms. To support the SOA, the services must provide standards-based definitions of communication protocols, mechanisms for service descriptions, discovery mechanisms, composition methods and a set of quality of service protocols. The Web services framework intends to be such a standards-based realization of the service-oriented computing paradigm

Page 17

Service provider

Service consumer

Request

Response


The initial trio of Web services specifications, SOAP, WSDL and UDDI, provided open XML-based mechanisms for application interoperability, service description, and service discovery. For a detailed look at these specifications and how they fit together, see [42]. Nowadays, the Web services framework contains a lot of standards to fulfill the requirements of a service-oriented architecture. They are discussed in the following sections.

2.6.2 A technical overview The Web services framework consists of a set of standards [41]. Their objective is to

make applications accessible via standardized internet protocols using XML for exchange of messages, describing services, discover and orchestrate them to enable a service-oriented architecture (see Figure 2.12).

Figure 2.12 Web services layers

As they only define standards and not their implementation, they are completly

platform and programming language independent. Only the usage of the ubiquitous internet protocols HTTP and SMTP are assumed. XML message formats encourage the paradigm of Web services, providing a standard for exchanging platform-independent data to avoid marshalling and unmarshalling processes [42].

Page 18

Composite Application Framework

BPEL4WS

UDDI

CoordinationTransactions

SOAP

XML

Context

Orchestration and

Composition Layer

Se

curi

ty

Discovery Layer

Description LayerWSDL

Messaging Layer

Encoding Layer


The core infrastructure is based on three standards. The first is the Web services Description Language (WSDL) [43], which describes the services provided by a Web services. The next is the Simple Object Access Protocol (SOAP) [44], a protocol used to exchange messages. Finally, there is the Universal Data Description Interface (UDDI) [45]. The UDDI registry is used to publish and discover available services.

Figure 2.13 Basic Web services interaction

The basic Web services interaction is shown in Figure 2.13. A service requester and

a service provider interact via SOAP, exchanging messages in XML format, based on the WSDL service description. This description was previously published by the provider to a service registry (UDDI) and discovered by the requester.

Discovery Agency (UDDI)

Page 19

Service Requester

(Client)

Publish a service Find/search a service

(SOAP/HTTP) (SOAP/HTTP)

Service Provider Bind to a service (WSDL Service description)

(SOAP/HTTP)


2.6.3 WSDL The Web services Description Language defines the interface of the provided

services using XML as the underlying format. Both sides, the requester and the provider, need to know the same service description to interact accurate. The sender needs it to format the message correctly and the receiver needs it to understand the received message properly.

The WSDL service description contains a service specific part and a section with protocol details. It contains the following elements: • Types

Defining data types used within a messages • Message

These are abstract elements exchanged between sender and receiver aggregating typed data items

• Port Type Port Types are a collection of operations to define interactions supported by an endpoint.

• Operation They specify a message exchange pattern. Therefore it contains Input, Output and Fault elements

• Input, Output They describe messages sent to and received from the client.

• Fault This error messages are returned if there is a problem processing a message.

• Binding It defines the communication protocol and how the interaction occurs

• Port A port is a combination of a binding and a network location defining a single endpoint

• Services Combines a set of related Port elements

WSDL descriptions can be mapped to the application behind in any form. The

concrete implementation of the service itself is completely separated from the description, so it can be implemented using any standards or any proprietary solution. WSDL specifications are usually generated with the help of tools used to implement the service.

Page 20


2.6.4 SOAP The SOAP protocol standardizes a XML-based data exchange format for messaging

and remote procedure call (RPC) purposes in a distributed environment working on the request/response model. The client sends a valid SOAP XML request to a server, who parses and validates it after its reception. If it is valid (WSDL conform) the server processes the request invoking the requested service and send a SOAP response back to the client. SOAP operates on the existing transport protocols HTTP or SMTP and has a simple XML structure containing an envelope element as root with two child nodes, the optional header and the body element. The header encloses header block elements containing metadata regarding the method call, for example for authentication extensions. The body contains the actual message in a format appropriate to the used service described with WSDL.

2.6.5 UDDI An important section in service oriented architectures is the discovery of provided

services. In the Web services architecture this part is currently realized with the help of UDDI. This specification defines a way for providers to publish their services and enables their discovery. It provides a simple framework for describing any kind of Web services. The schema defines four core types of information: business information, service information, binding information, and information about specifications for services. This information is stored within yellow-pages like directory called the UDDI registry. These registries can be public for global access or private for internal services. UDDI also offers an API specification defining how to gain programmatic access to the information allowing for example rich discovery automation or tools interacting directly with the registry. This API is also built on the SOAP specification.

2.6.6 Business Process Execution Language for Web services The Business Process Execution Language for Web services (BPEL4WS or BPEL,

for short) [46] defines a formal specification of business processes and business interaction models. The XML-based language is designed to enable task-sharing for a distributed computing environment. It describes the interactions between the process and its partners, these are the services the process interacts with, as well as the state and the logic necessary for coordination of multiple Web services to realize more complex functionality. The interactions itself are based on XML Schema, SOAP and WSDL. The BPEL was written by developers from BEA Systems, IBM, and Microsoft and it combines and replaces IBM's Web Services Flow Language (WSFL) and Microsoft's XLANG specification. It is nowadays the standard orchestration language to compose multiple services into loosely-coupled business flows as all major players in the industry participate now.

Page 21


Like any Web services, a BPEL process defines a set of WSDL interfaces that enable it to exchange messages with its partners. The interaction with them is based on the invocation of the operations supported by the partners and the receiving of messages through the process service interface. An example of such interactions is illustrated in Figure 2.14. In general, the interaction within a process between the composition and its partners is a peer-to-peer conversation in which each party sends messages to the public interfaces of the other to invoke its operations.

Figure 2.14 BPEL process interactions with partners A BPEL process consists of the following main elements to represent business

processes. First, there is the definition of the partners by declaring the WSDL interfaces which will take place in the interaction with each partner. All interfaces provided by the partners as well as by the process are included. BPEL compositions are platform-independent as only abstract interfaces are used for partner definitions to allow standard access via SOAP over HTTP, as well as, for example, JMS and J2EE protocols. Second, activities define how messages are exchanged with each partner. These activities can be separated into primitive activities and structured ones. Primitive activities are the invoke activity, used to sent a message to a partner, the receive activity, for being invoked by a client, and the reply activity which is the response of an operation. Some further primitive activities, for example, signaling faults and terminating, are available. Structured activities are a combination of the primitive activities to facilitate more complex algorithms. This also includes the ability to define step sequences, loops conditional branches, and parallel execution. These structured activities can be recursively combined to achieve high flexibility and enable reusability. Another functionality provided by BPEL is the correlation mechanism. This allows the identification of stateful instances of business processes, based on key fields within the data messages to correlate messages from a partner to the associated conversation. Partners and Web services are in contrast to traditional object systems, in which process instances are referenced using an explicit instance identifier, dynamically bound based on these service references. These correlation fields are

BPEL process

Page 22

Partner 1

Process WSDL

Partner 2 WSDL

Partner 2

Partner 1 WSDL


initialized when the creation of an instance takes place, for example, through a received message. Moreover, each interaction may be based on different correlation sets at different times for maximum adaptability. Furthermore, BPEL establish scopes within business processes to arrange interactions to transaction units. These scopes can be nested to enable transaction and error handling. The corresponding protocol is defined in the WS-Transaction specification. While scopes within BPEL imply that a scope and all its nested scopes are within a single process hosted by a single BPEL engine, the agreement protocol in WS-Transaction does not assume this. Therefore, a future enhancement of BPEL may support distributed transactions across processes and execution engines. Finally, BPEL enables through fault and compensation handlers considerable error handling capabilities. The fault handlers are similar to typical try-catch blocks, for example, in Java. The compensation handlers facilitate rollback functionality within BPEL. Both handlers are used within the context of scopes for enhanced error handling.

Business processes specified in BPEL are fully executable and portable between BPEL-conformant environments. A BPEL business process interoperates with the Web services of its partners, whether or not these Web services are implemented based on BPEL.

Page 23


2.6.7 Web services Composite Application Framework The Web services Composite Application Framework (WS-CAF) [47] is an open

framework that supports the coordination of multiple Web services to enable various transaction processing models and architectures that are used in combination within applications. Although this is a self contained set of specifications its primary aim is to complement orchestration with BPEL. The WS-CAF is composed of three standards, starting from WS-Context (WS-CTX) for simple context management, second, WS-Coordination (WS-CF) for additional context management features and message delivery guarantees, and finally WS-Transactions (WS-TXM) for recovery protocols to deliver a large set of functionalities. These standards can be combined in various ways to fulfill the requirements.

WS-Context The WS-Context [48] specification provides a mechanism for message correlation in

stateful interactions among related Web services. It defines the context, the scope of context sharing, and basic rules for context management.

WS-CTX defines activities that relate multiple Web services through shared context. An activity is created, executes, and then completes. It may execute over long periods of time to enable long-running processes. Activities itself are implemented as Web services, too. In order for an activity to span a number of Web services, it propagates the context as additional information to each participating service to associate multiple Web services with the same activity. For reliable management a Web services Context Service maintains a repository and tracks shared contexts in Web services interactions (see Figure 2.15).

Figure 2.15 WS-Context overview

WS- Coordination The WS-Coordination [49] describes an extensible framework for providing

protocols that coordinate the actions of distributed applications. The WS-Coordination framework is based on a coordination service (or coordinator) that contains context services to create context, activity services to propagate context to other services and registration services to register to coordination protocols. This specification enables transactional processing in a heterogeneous environment and defines context structures and their propagation. The framework provides coordination protocols to enable activities for short-lived operations as well as for complex long-lived processes.

WS-Context

Web services Context

Resource Web services

Web services

Page 24


The CoordinationContext is used to pass coordination information to partners involved in a coordination service. Its elements are located inside the message. The Activation services define a CreateCoordinationContext service which is used to create a coordination context that supports a specific coordination type in heterogeneous environments. Such coordination types are specified, for example, with WS-Transaction that is a companion specification. The Registration service allows a service to register for the activity to participate when a coordination context was received before.

WS- Transactions Transactional reliability is an often critical requirement on integration tasks. Reliably

means that if any part of a process fails, no part of the process is processed and the already processed work can be undone. The WS-Transactions [50] defines two coordination types to enable transactions within Web services: Atomic Transaction (AT) and Business Activity (BA). Atomic transactions are used to coordinate activities having a short duration executed within trusted domains. They are called atomic because either all or nothing within the transaction is executed. Business activities are used to coordinate long running activities where business logic is used to handle exceptions. As these activities are long running the involved resources cannot be locked until the business activity has finished. Instead, these activities are applied immediately and are permanent.

In order to implement these coordination types, a distributed transaction coordinator acts as an intermediary, tracking all interactions with the underlying services and coordinates the acceptance or "commit" of the transaction. The Web services transaction coordinator itself is a Web service that receives communications via SOAP messages to enable platform independence. The WS-Transaction specifications explain how Web services implementations describe common operations such as "begin," "commit," and "rollback" in a transactional context. Web services applications can include both atomic transactions and business activities. Furthermore, WS-Transaction specifies protocols for coordinating distributed atomic transactions which may be a future extension of BPEL to support activities of different processes.

2.6.8 Web services Security Web services Security (WS-Security) [51] is a construction kit for various security

models including Kerberos, PKI and SSL. The specification enhances SOAP with methods for security token propagation, message integrity and message confidentiality. WS-Security supports various security token formats to be extensible as no specific type of security token is required. It provides mechanisms to associate security tokens with messages to enable various security models as well as mechanisms to describe the characteristics of the credentials that are included with a message. Additionally, binary security tokens, for example, for the encoding of X.509 certificates and Kerberos tickets, are addressed in WS-Security. WS-Security doesn’t provide a complete security solution. It provides building blocks that can be used with other Web services extensions to achieve security.

Page 25


2.6.9 Web services Notification A recently introduced family of Web service specifications is the WS-Notification,

that defines a topic-based publish/subscribe interaction pattern (see Chapter 2.2.2) for Web services [52]. WS-Notification composes with the WS-Resource framework specifications [53], which introduce the relationship between stateful resources and Web services, to avoid duplication of functions within the Web services framework.

The major issue of WS-Notification is the introduction of mechanisms and interfaces for clients to subscribe to topics of interest. The associated WS-Resource framework provides the features to represent and structure notifications whereas WS-Notification extends the WS-Resource framework by introducing asynchronous notification mechanisms of resource property changes for interested partners. The WS-Resource framework allows WS-Resources to be declared, created, accessed, monitored for change, and destroyed via conventional Web services mechanisms. Therefore, the WS-Resource framework, introduces five specifications that defines the WS-Resource approach. First there are the WS-ResourceLifetime mechanisms for the destruction of WS-Resources either immediately or scheduled. Second, the WS-ResourceProperties enable the examination and modification of WS-Resource properties via message exchanges. Third, the WS-RenewableReferences enables updates of WS-Addressing endpoint references when they become invalid. Fourth, there is the Ws-Servicegroups to define by-reference collections of Web services. Finally, WS-BaseFaults enables standardized fault reporting facilities within the WS-Resource framework.

The WS-Notification itself consists of three specifications to extend the WS-Resource framework with notification mechanisms, for example, to implement publish/subscribe patterns. The WS-BaseNotification, describes the basic concepts for NotificationProducers and NotificationConsumers to enable a subscriber to register interest in receiving messages from a producer. Such a notification message concerns changes in the state or the value of a resource property of the producer. Therefore, a NotificationConsumer sends a subscribe message to register interest on one or more topics, so that the subscriber receives a WS-Resource-qualified endpoint reference to a subscribed WSResource. A WS-Resource and the WS-ResourceProperties and WSResourceLifetime are used to manage the relationship between the subscriber and the producer. The WS-Topics part introduces the XML specification of topics. A topic represents a description of notification messages to enable convenient message subscriptions. These topics are hierarchically organized to enable topics to be split to more specific child topics. Finally, the WS-BrokeredNotification introduces a NotificationBroker which is an intermediary service to manage consumer and producers across the system.

Page 26


2.7 Web services and Integration

Web services can provide some integration solutions, but they cannot offer all the features provided by EAI systems. Web services are completely based on standards that are platform independent, so they can be implemented in every environment. In contrast to most standards, the core Web services standards are supported by all major vendors around the world, so their implementation is supported by a wide variety of products. Only recently released specifications are causing conflicts between two vendor camps.

The platform independence enables the lowest level of integration – platform integration. The next level, data integration, cannot be sufficiently accomplished with Web services, because the used standards require the definition of the exchanged data types. Hence, the data transformation problem itself isn’t solved, but it is shifted into the Web services implementation, which is responsible for the correct mapping between the application data and the data provided by the service. Most existing proprietary service implementations do not work together out-of-the-box. Web services help to build a bridge between those products by providing a common framework through which heterogeneous platforms can be integrated, when these systems are extended with Web services functionality. Thus, the benefit of Web services is the realization of application integration only for Web services enabled applications. The integration of these applications can be straightforwardly achieved as Web services provide a standardized way for method invocation.

The BPEL enables an implementation independent description of business processes integration across Web services enabled resources. This allows the composition of available services to achieve business goals. Nowadays, BPEL is widely supported within business process modeling products for exchange of abstract business processes. However, the implementation itself isn’t often based on Web services because business functionalities within applications aren’t exposed as Web services in many current cases. Besides, BPEL doesn’t support distributed transactions, which are in enterprise wide integration tasks, a common requirement to achieve scalability.

The recently published Web service extensions provide the mechanisms to build a steady EAI solution. A combination of these features enables, for example, the implementation of a asynchronous, loosely coupled, secure and reliable infrastructure which is a necessary constraint for solid integrations. However, they are still specifications in evaluation and review state. Primarily, a widely support for these enhancements is needed. Then, all these WS-* specifications can be combined to enable an enterprise service-oriented architecture with sophisticated integration capabilities.

However, some features offered by EAI solutions still represent a challenge for Web services nowadays. These include data transformation and mapping. Furthermore, the monitoring of activities within the environment does not exist due to the absence of measurement, reporting and management specifications. And last, human workflows are missing, as there are no abstractions for people, roles, or inboxes in BPEL or other specifications.

Page 27


Page 28

Another drawback of Web services is that they offer their integration potential only through the implementation of the Web services interface on top of applications. They are often preloaded in new applications, but a legacy system needs a costly modification to obtain integration. Whereas EAI tools adjust to the applications with the help of adapters, Web services impose their own standards on the applications. Therefore, with EAI tools a modification of the applications is not required. These facts point out why Web services are sometimes called EAI light as they don’t offer all characteristics of EAI technologies nowadays. Newer versions of these standards will improve Web services technology and push their pervasiveness.

Overall, the emerging service-oriented architectures and integration technologies are convergent. Future computing technologies are highly affected by service-oriented architectures, but a transition to this new architecture isn’t done within a little while, due to the strong dependency on well-established legacy systems. So a combination of these technologies will influence IT infrastructures in the near future.

A possible combination is the following. New implemented services are instantly based on Web services to leverage service-orientation. Already existing applications are integrated within a system that wraps up service-orientation without changing them. This can be done with almost every EAI tool on the market, as they support the exposition of internal services, based on proprietary standards, as Web services. Finally, such a combined system builds the basic infrastructure of a service-oriented architecture (see Figure 2.16).

Figure 2.16 A possible service-oriented architecture based on Web services and EAI

Tools A summary of all characteristics of state of the art application integration systems are

illustrated in the compendium below.

EAI-Tool

Integrated legacy applications

New Web services enabled

applications

Web services Available

New Web services enabled

applications

BPEL for service orchestration


Page 29

The following table (Table 2.1) illustrates a compendium of the EAI technologies discussed in this chapter. Component Available characteristics Messaging middleware to provide communication between the connected systems

Synchronous communication for time-shared interaction

Asynchronous communication for time-decoupled message exchange

Message queuing for order-preserved delivery

Publish-subscribe mechanism for loosely coupled systems

Application integration to connect the EAI run-time system to the integrated applications

“Technical generic adapters” like CORBA, JDBC, RPC

Proprietary or standard based adapters using API

Web Services usage

Data transformation Available Transformation types

Support for Data Mapping

Support of standards documents (i.e. EDI, ...)

Process integration Distributed process execution Transactional processes Graphical tool and/or Business Process Modeling Language

System architecture for a scalable, reliable infrastructure to handle all integrated systems

Centralized system Decentralized system

Scalability (modularization)

Administration andmonitoring features

Message tracking and generating

to manage the system

Debugging

Adapter administration

Statistics (Long-time monitoring)

Table 2.1 Compendium of EAI technologies

http://dict.leo.org/?p=lURE.&search=characteristic

Evaluation of EAI products

Page 30

3 Evaluation of EAI products

The evaluation of different EAI products available on the market demonstrates their strength and weaknesses. All of them assure that they can meet all requirements for a complex EAI system, but there are many different approaches to solve them. The following sections provide a discussion of the underlying architectures, the used integration solutions, the provided adaptors, available data transformation and data mapping. Furthermore, it discusses the modelling process and the management and administration features put into action. These tools were analyzed with the data of the case study (see Chapter 4).

3.1 BEA WebLogic Platform

BEA WebLogic Platform 8.1 provides a new approach for enterprise integration. It combines the two different worlds of application development and application integration in one unified platform based on the J2EE foundation of WebLogic Server. The WebLogic Integration is now a part of the single WebLogic Workshop development environment. The new WebLogic Platform contains several parts, previously offered as different products, like the BEA WebLogic Portal, the BEA WebLogic Server, the BEA WebLogic Workshop, BEA WebLogic JRockit and the WebLogic Integration. The WebLogic Server builds the fundament of the platform, the Workshop provides the development environment, JRockit is the shared Java Virtual Machine and the Portal offers the possibility to build custom portals. EAI is realized inside the WebLogic Integration component. The figure below shows a Platform overall architecture chart to see the cooperation of all platform components.

Figure 3.1 BEA WebLogic Platform architecture [53]


Page 31

3.1.1 Installation and configuration The recent BEA WebLogic platform 8.1 is only a beta version and therefore only

available for MS Windows. The final version will also run on Sun Solaris, HP-UX and Red Hat Linux. The java-based installer is for evaluation purposes available for download on BEA’s homepage. It can be installed in express or in custom mode. The express setup does all the work and installs all the necessary program files. This is adequate for evaluation purposes, but for a production environment the custom setup, which allows a detailed selection of the installable components, is preferable. After the installation has finished, some additional steps have to be performed for completion. A WebLogic domain has to be created and configured. A domain encapsulates a set of resources into a single manageable unit. A set of domain templates is available to simplify this process. It can be reconfigured every time with the help of a web-based configuration console. Now the basis platform is ready to run.

3.1.2 WebLogic Integration overview For EAI tasks the WebLogic Integration module, included in the platform,

implements a set of relevant business integration components, like data transformation, application connectivity, message-brokering, business process management and user interaction. Furthermore, there exists a management console for administration and monitoring purposes. The following chapters are discussing the components to achieve these capabilities.


Page 32

3.1.3 WebLogic Workshop WebLogic Workshop is an Integrated Development Environment (IDE) framework

which is used for all implementation work inside the platform (see Figure 3.2). Therefore, it embeds all necessary tools. Development in WebLogic Workshop starts by the creation of an application. It’s like a project containing all the files and properties belonging to the application. Furthermore, such applications can be packaged for deployment. There are different types of files to add to a project. For example, usual java source files, schema definitions, page flow files for JSP-pages with process logic and workflow files. Some file types provide a source code view as well as a graphical view. So, for example, java page flows or business processes can be designed graphically, while in background the corresponding source code, for later execution, is generated automatically. The flow logic is represented by XML documents and the performed operations are implemented in java. In Workshop, switching between the design view and the related source code is possible at any time. A modification of the design immediately changes the source and vice versa. This also offers the possibility to add own code directly within the source view for additional functionality.

Figure 3.2 BEA WebLogic Workshop


Page 33

3.1.4 WebLogic Control Architecture The Workshop introduces a Control Architecture to get in contact with each type of

resource to provide a consistent mechanism interacting with resources across the platform. A Control is a high-level view, abstracting specific details, of internal or external resources. They offer a unified interface to get in touch with them. This simplifies the usage of internal or external resources as they provide functions, with input and output parameters, hiding all underlying settings. Hence, an external resource can be integrated by the help of a control providing only the functions needed by the developer. All settings required for communication and invocation of these functions are hidden. These settings can be configured by an expert of the integrated resource with the help of a web-based configuration console. WebLogic is based on an extensible architecture to develop custom controls and to make them available anywhere within the environment. There are also compositions of controls possible, for example, combining controls that invoke themselves and providing a higher abstraction level. WebLogic comes with a set of controls for integration tasks to get in contact with external resources. However, these controls aren’t restricted to integration problems as the Control abstraction incorporates them into the entire platform. Here is a short description of them:

• File

The File-Control allows integrating read/write file access into a workflow, also from large files. Every type of file event is automatically published to the Message Broker for further processing if required.

• E-Mail With this control e-mails can be sent and also incoming mails can be monitored and published immediately to the Message Broker

• Database This integrates read and write from and to a database into a workflow. This is a very generic control, so it doesn’t provide any further functionality. For that reason, a set of additional specialized adapters is favored.

• JMS With the JMS-control a standard messaging protocol is provided.

• Application View Control Provides direct access to back end resources such as packaged applications and adapters that offer a control interface.

• Transformation Control This Control provides sophisticated data transformations. So transformations can be separated into one or more controls and are independent from the data routing.


Page 34

3.1.5 Business Process Management The WebLogic business process management subdivides the enterprise into a set of

business services accessed via Controls that can be modelled up to orchestrate a business process. This includes internal systems, external resources and users. The workflow tool offers synchronous and asynchronous communication, as well as state-full and stateless processes for high flexibility. Workflows are created with a graphic workflow editing tool, which is integrated into the WebLogic Workshop development environment, to straightforward build a descriptive representation of business processes (see Figure 3.3).

Figure 3.3 A clipping of a business process model in Workshop


Page 35

A workflow consists of any of the following nodes: • Client Request

These nodes provide a way for a client to make a request to a workflow. A method name has to be defined for invocation that can be exposed, for example, as a Web services.

• Client Response Such nodes allow a workflow to send messages to clients.

• Control Send Control Send nodes represent points in workflows at which they send messages to resources via controls.

• Control Receive Control Receive nodes are for receiving asynchronous messages from controls. A workflow waits at a Control Receive node until it receives a message from the specified control.

• Control Send and Return This type of node handles synchronous exchange of messages between workflows and resources via controls.

• Perform Perform nodes allows to write custom Java code. They invoke the specified methods.

• Decision Decision constructs allows branching within workflows based on evaluation of one or more conditions.

• While-Do Activities in a While-Do loop are performed repeatedly while a specific condition is true.

• Parallel Parallel nodes allow execution of activities in parallel.

• Finish A finish node exits a workflow.

Every node has properties for input and output parameters which are assigned to the

executed operation. The parameters sent to and retrieved from nodes have to be defined as global variables before, which makes theme available across the whole process. While workflows are graphically designed, the corresponding java source code, which is used for the process execution, is being written automatically to a workflow file. Toggling to the code view is possible at any time. In that view a few lines of code can be added directly to accomplish more complex tasks if necessary. The generated workflow file consists of an XML part for the flow and java code for the implemented operations.


Page 36

3.1.6 Data Transformation WebLogic Integration introduces an XQuery processor for transformations [55]. It

also provides the commonly used XSLT language, which was initially designed to convert XML data to another presentation format. XQuery is an SQL-stylish language and defines what to do with the data.

These XQuery transformations are created with a visual data mapping tool (see Figure 3.4). A library with standard transformations is offered to simplify more complex transformations tasks. As mapping processes are very error-prone a testing mode for debugging of transformations is embedded. In this mode test data can be entered manually and with a single click the transformed data is displayed to check if the XQuery works as desired.

Figure 3.4 BEA WebLogic XQuery Data Mapper

Transformations can be called directly inside a control node to map and modify workflow data corresponding to the required control parameters. Furthermore, these components can be packaged as controls to inherit all advantages from the Control Architecture, for example, platform-wide reusability. These possibilities allow a seamless integration of transformations within the workflow.


Page 37

3.1.7 Message Broker WebLogic implements the communication between business processes via a

Message Broker. This is a channel-based publish and subscribe mechanism. This means, that a publisher broadcasts his messages to a channel and each subscribed process is immediately activated. Processes publish and subscribe to channels statically or dynamically via controls. Subscriptions can trigger a workflow or can be used to block a process. Additionally, an XQuery filter for message selection on subscriptions is available to filter messages based on document types or document content. In addition to business processes that can publish messages to channels, event generators are supported, which can publish events from external sources to message broker channels, for example from a file control or any adapter.

3.1.8 Application Integration The integration of back-end systems and legacy applications is provided by a set of

J2EE Connector Architecture (JCA) [56] adapters. A growing set of standard adapters for major business applications is available nowadays. The installation and configuration of these adapters has to be done separately. Some of the major adapters have their own documentation [57] [58], which indicates that this task isn’t as simple as it looks at first glance. It takes a lot of time to set up an adapter properly, but the resulting application view simplifies the integration of applications into business processes. BEA introduces these application views as an abstraction layer between the integrated application and the usage of their functionality inside WebLogic. An application expert can configure the services and events exposed by an adapter via web to make them available through an Application View Control in the Workflow Modeller. A non application expert can now use and maintain these services of the integrated application via a standard component, the Application View Control.

In addition, an Adapter Development Kit (ADK) supports the development of JCA-based custom adapters. This ADK provides a framework to develop, test and distribute self-made adapters to get in contact with proprietary legacy systems.

3.1.9 Human collaboration BEA’s Worklist system enables end users to collaborate with business processes. It

allows the creation, manipulation and management of tasks. Tasks interact with one or more processes and are assigned to users or roles. Therefore, two extensible Worklist controls are available to realize all operations on tasks. The Worklist framework provides a standard web-based user interface as well as a custom user interface to develop corporate design conform JSP-pages. An included Worklist wizard helps to implement collaboration tasks by creating standard functionality. However, without any coding the integration of human resources isn’t feasible.


Page 38

3.1.10 Management and Administration The web-based WebLogic Integration Management console offers workflow

management and monitoring as well as system administration. Therefore, two separated data stores are used. Run-time data about integration engines, business process states and message history is stored into the online database and via configured policies archived to the offline store for later analytics. The workflow management allows specifying workflow properties like name, description and tracking level whereas the monitoring displays workflow instance statistics. The configurable tracking level on individual workflow types allows setting up the stored data amount regarding the individual needs for workflow monitoring. The instance statistics includes the state of the processes, like Running, Suspended, Frozen, or Completed, the actual variable names and values, the elapsed time and pending activities (see Figure 3.5). Depending on the actual state suspending, resuming or terminating of instances is possible.

As all the monitoring data is separately stored, data manipulation via the console isn’t provided which restricts debugging and error diagnostics.

Figure 3.5 Monitoring

The Message Broker monitoring provides the viewing and managing of channels, filters and publication and subscriptions properties as well as summary statistics. General system, user and security settings are also available via the console (see Figure 3.6).


Page 39

Figure 3.6 Administration

3.1.11 Conclusion BEA’s solution is not only an EAI system. Based on the BEA WebLogic Application

Server the platform is a complete Integrated Development Environment (IDE) extended for application integration and business process modelling and execution tasks. This is reflected by the implementation of a business process, because most of the time code is written. Also the integrated business process designer is still in its infancy, so it is often very confusing to find the accurate process step. However, the cooperation of the design and the implementation part work very well. The simple toggling between these two views enhances the development in every realization step. This “all-in-one paradigm” has some great benefits, but it is also its drawback, because the whole implementation part has to be accomplished by skilled developers. I think that an EAI tool itself is complex enough to handle, so that a determination between an application and an integration system has its claim. Anyway, WebLogic’s strength is the control abstraction which defines a level between the resource behind the control and its usage in the code, for example, an application view. After the definition of an application view is done, the integration within a workflow or an application by a developer is very easy. Also the separation between the definition of an external application view via web, and its usage within a project allows that application experts can define their application views and make them available for the developers who do not need to know exactly how the external resource works. This allows easy integration of external resources, a main task in EAI projects.


Page 40

3.2 Microsoft BizTalk 2002

3.2.1 Conceptual overview Microsoft’s integration platform is based on the BizTalk Server 2002 [59] and is

tightly integrated with the Microsoft Windows operating system. So BizTalk makes use of many Microsoft Windows features as well as other Microsoft products. This includes Windows services like the Internet Information Server and the Message Queuing Service, the MS XML Package, the MS SOAP package, the MS .NET framework and the MS SQL database for data storage and MS Visio for process definitions.

BizTalk itself provides data mapping and transformation capabilities as well as a development and execution environment for the managing of business processes between various systems. BizTalk is separated in two core components: the BizTalk Messaging and the BizTalk Orchestration. The Figure 3.5 shows a high-level overview of the architecture.

Figure 3.7 Microsoft BizTalk Server architecture

The BizTalk components can be distributed over several servers to achieve high

performance and scalability. Further operating system features are used to implement security and to monitor and log server activity for administration purposes.

BizTalk Orchestration

BizTalk W

eb S

ervi

ces

CO

M/C

OM

+

Messaging Services

MSM

Q

SMT

P

HT

TP

AIC

n

AIC

1

File

…


Page 41

3.2.2 Installation and configuration As the Microsoft BizTalk Server 2002 is operating system dependent it is only

available for Microsoft Windows 2000. Before it can be installed a lot of prerequisites are necessary. So at least a Windows 2000 Professional Server with Service Pack 1, a Microsoft SQL Server and Microsoft Visio 2002 must be available and properly configured. Also a lot of additionally components for Windows must be installed to enable full functionality of BizTalk. After all the needed components are available the installation of BizTalk itself can be done.

An installation wizard guides one through the setup of the BizTalk Server itself. For an express installation only the database name and the database user that is used by BizTalk Server, has to be defined, but the amount of required packages lengthens the whole process to several hours. Finally, some further configurations have to be completed to successfully run a BizTalk Server instance.

3.2.3 BizTalk Orchestration Services The first component in Microsoft’s integration solution is BizTalk Orchestration. Its

services are responsible for the implementation, execution and administration of business processes. The BizTalk Orchestration Designer, the process modeler application for BizTalk, offers the possibility to create XLANG [60] schedule drawings. These drawings are the graphical representations of a business process and consist of a workflow view, an implementation view (see Figure 3.8) and a data flow view. The Designer is a graphical design environment based on Microsoft Visio. As Visio is a diagramming tool, a business process is designed similar to a flow chart. Therefore the following objects can be inserted in the workflow view to design a business process: • Action

The Action object represents a process that has an input and an output port for sending and retrieving messages. The process itself is defined in the implementation phase.

• Decision Decision shapes evaluates rules sequentially and redirect one input port to one or more output ports.

• While This object repeats the encased steps until the rule evaluates to true.

• Fork and Join The Fork shape introduces parallelism into a business process with one input and up to 64 output ports, and the Join shape synchronizes parallel flows to one output port.

• Transaction The Transaction shape represents a collection of actions that are either all executed successfully or none of them.

• Abort The Abort shape terminates execution within a transaction.

• End The End object defines the completion of one flow.


Page 42

The business process is designed in the workflow view without considering any implementation details. They are specified in the implementation view. Therefore, the Orchestration Designer offers some implementation shapes which represent the tasks that had to be performed by each action step drawn in the workflow view.

On one hand there are the BizTalk Messaging, the COM Component and the Message Queuing shape, which can be set up via a dialog to interact directly with the corresponding elements within the Messaging Services without writing any code. The exchanged document data defined within these services can be used immediately inside the process.

On the other hand there is a scripting component shape for the creation of lightweight components in one of the common scripting languages, like Microsoft VBScript, to integrate business logic into the business process. As BizTalk doesn’t provide any development environment the implementation of such components has to be done with external tools.

Furthermore, these scripting components are the only possibility to implement human interaction directly within BizTalk, because there is no human workflow component to embed human interaction into business processes, for example, for confirmation issues, inside BizTalk. More complex human interaction like multi-level approval and review processes may only be realized with the help of external applications.

Inside the data flow view the relationship between document fields on different documents are specified. Therefore, all available fields within the process can be connected via drag and drop. This is just a simple data flow definition. Data mappings and transformations have to be done outside the process definition.

Figure 3.8 Workflow (left) and Implementation (right) view


Page 43

Completed process definitions have to be compiled into an executable business-process file called an XLANG schedule. This is a XML based description of the logical sequencing of the steps and their implementation by components or services. These XLANG schedules are controlled by a service called XLANG Scheduler Engine, which is responsible for the instantiation and execution of one or more schedules. It provides the ability to store and retrieve all state information in a MS SQL Server database. This allows the execution of complex transactions that run as long as weeks or months without having to run a process or to restore process information after failures. These features leverage the reliability and the scalability of the system.

3.2.4 BizTalk Messaging Services The BizTalk Messaging Services are responsible for reliable and secure document

exchange. They represent the message-oriented middleware of the BizTalk platform. These services receive messages from defined sources and route them to one or more destinations. In between the received data is transformed into a generic data object in XML format. Afterwards, this XML data object is mapped into a format related to its destination. To obtain documents from sources there are some receive functions available to monitor data posted at specific locations: • Files

Applications can simply create files containing specific data without the need to know anything about BizTalk. The BizTalk Server picks them up and processes them depending on the respective file content. For such a file reception the configuration of the server, the directory to poll for and the relevant file types is necessary. This allows an asynchronous loosely coupled process execution which leverages the reliability, because each system works independent and is unaffected by the temporary unavailability of other systems due to system or network failures. When the failed system is available again the process execution can advance.

• Message Queuing receive functions The Microsoft Message Queuing (MSMQ) works similar to the mechanism introduced in section 2.2.1 and is integrated in the Windows operating systems family. These queues are sending their messages directly to the BizTalk messaging service. MSMQ provides synchronous and asynchronous, reliable and secure message delivery.

• Internet receive functions BizTalk has the possibility to receive documents through e-mail or from Web sites to interact with the Messaging Services.

• Application integration functions These allow the receiving of documents from COM objects and from application integration components (AIC), Microsoft’s Adaptor solution. Microsoft itself offers only adaptors for SAP, WebSphere MQ, SQL Server and Web services, but a lot of standard adaptors are available from third-party companies, like MS Office, Peoplesoft, Remedy, Siebel, BroadVision, and so on. Also an adaptor developer kit is available to implement proprietary solutions, if required.


Such an application integration component creates COM objects which are able to interact with the BizTalk messaging services. As an example of these components the functionality of the SAP DCOM connector is shortly introduced here. The SAP DCOM component receives messages from an outgoing BizTalk message queue and response messages to an ingoing one. The DCOM Connector creates COM components which communicate directly with a SAP system. Therefore, the DCOM Connector is first configured to get in contact with SAP and then the required SAP remote function calls are selected. Finally, the Connector Object Builder is used to create the executable COM object. This can be called directly by BizTalk to communicate with a message queue. The flow of a message between these systems is shown in Figure 3.9.

Fig A

quethedelincpro

WfordocdocAlstakEdi

Ttheof IDo

COM component

Page 44

ure 3.9 Integration of SAP into BizTalk

ll receive functions are sending the received data into BizTalk Messaging Service ues, which are named channels. A channel is responsible for the identification of document source and it defines the steps performed by the server before data is ivered to its destination. Therefore, a channel contains a set of properties. This ludes information about sources, destinations, logging, as well as data translation perties. hen the Messaging Services receive a document, it is translated into a generic

mat. For these data translations BizTalk provides a set of data parsers for industry ument standards like EDI that translates non-XML documents into generic XML uments. The registration and usage of custom parser components is possible, too. o a verification of the document against a document definition specification (XML) es place to check its validity. These specifications can be edited within the BizTalk tor. he BizTalk Editor is a graphical tool to create and edit document specifications in form of XML schemes (see Figure 3.10). They can be designed manually, but a set ready-to-use document specifications for popular document types, like EDI, SAP c, and so on, can be reused too.

Response

Request BizTalk Server

SAP System

Message

COM component

Message queue 2 queue 1


Page 45

Figure 3.10 BizTalk Editor

These data format specifications are used by the graphical BizTalk Mapper to create

transformations from input to output specifications. It displays both specifications side-by-side and allows drawing lines between the corresponding fields (see Figure 3.11). For more complex transformations so called “functoids” are available. More than 60 of these “functoids” are included within BizTalk Mapper for standard string, date/time, logical, and other transformation operations. A script functoid for implementing transformation logic with Microsoft VBScript and the possibility to add new functoids to the library are provided, too. The tool internally generates XSLT (XSL Transformation) scripts to represent these mapping rules. Non-valid documents aren’t processed any further and are placed into a suspended message queue for further analysis.

Figure 3.11 BizTalk Mapper


Page 46

BizTalk provides a set of transport services to transmit the documents from its sender to its receiver. These transport services include a set of protocols (HTTP, HTTPS and SMTP) and application integration components to realize communications. The endpoint of a channel - its destination - is named messaging port (see Figure 3.12).

Messaging ports describe how documents are transported and secured to their destination. Destinations are defined like document sources and use the previously introduced transport services. A messaging port can also deliver messages to the BizTalk Orchestration Services, for further processing in business processes, as well as to applications respectively to their application integration components. A set of messaging ports can be grouped into distribution lists to send one source document to several different receivers.

Figure 3.12 Message flow between receive functions, channels and messaging ports

The reliability of document delivery can be achieved with the help of some

Messaging Services properties. For example there are receiving receipts, number of retries and time between retries adjustable. Also the BizTalk framework-compliant envelopes, which provide reliable messaging features, are supported. For persistency reasons all the data and metadata is stored into a MS SQL Server database. This includes all the configuration settings and all generic XML document specifications.

Furthermore, BizTalk provides the security mechanisms offered through Microsoft Windows 2000 and Microsoft SQL Server security. It supports encryption and digital signatures for transmission and decryption and signature verification for the documents that it receives. It also supports a public-key infrastructure and the Kerberos [59] protocol.

File receive function

Channel (Transformations

generic XML document

Messaging P t

source document format

destination document format Other receive


Page 47

3.2.5 Management and Administration The managing and monitoring of running processes is offered by the XLANG Event

Monitor. This application is based on Microsoft Windows 2000 Events where all the events generated by the XLANG scheduler are stored. It displays currently running and completed business processes and their states, for example, completed, erroneous or suspended (see Figure 3.13). These events can also be displayed with the generic Microsoft Event Viewer.

Figure 3.13 The XLANG Event Monitor

A click on a process instance displays additional information for reviewing all the

already performed steps and the consumed documents. Also, the possibility to run, resume, suspend or terminate a process instance is offered for manual intervention. Unfortunately, there isn’t any web application available for process monitoring which restricts its usage to the server which runs the XLANG scheduler service.

For data monitoring purposes a stand-alone web application, the BizTalk Document

Tracking, is available. Every document exchange that is send via the Messaging Services is automatically recorded in a tracking database. By default metadata for an interchange, such as source and destination information, document type, date and time parameters, are recorded. However, additional fields can be configured to be captured if required, for example to keep copies of electronic business transactions to fulfill legal requirements. The stored information can be accessed via the web interface (see Figure 3.14). A set of standard queries to filter messages based on date range, message types, source and destination is available by default. The Advanced Query group in Document Tracking allows building expression-based queries to select fields previously configured to be captured in the database. The result of the query shows only the standard information, but the result can be expanded to get access to all the information that is available, including process state, the related schedule and all database fields. This web application is more or less a web front-end of a database query tool without any included logic. Thus, it only displays data that is stored in the database, but any kind of data manipulation or statistical analyzing isn’t offered.


Page 48

Figure 3.14 BizTalk Document Tracking Query Builder

The administration and configuration of BizTalk is done with a MS Windows

console. BizTalk’s administration model splits into four main areas, the server, application, programmatic and database administration. The server administration area is responsible for the configuration and management of servers and server groups. In enterprise-wide solutions it is highly recommended that key components are distributed across multiple servers to optimize performance. The definition of server groups, a collection of individual servers, allows managing and configuring them centrally. For communication within server groups shared queues are used. Within the server administration all the tasks for defining servers, server groups, shared queues and receive functions are done to centrally handle server farms. Application administration includes the management of the scheduler application, the COM+ applications and the Orchestration persistence database. The programmatic administration manages the queues and the XLANG scheduler system manager which is responsible for XLANG schedule instances. The database management allows setting up the connectivity between BizTalk Server and Microsoft SQL Server. Databases can be created, configured, restored and/or removed manually within the server groups.

3.2.6 Conclusion Microsoft’s BizTalk Server was initially built for data mapping and transformation

tasks to allow “talking” between trading partners. The BizTalk Server 2002 has been extended to a complete EAI-tool, but with many weaknesses. Thus, the handling is spread over many applications and often the features of the operating systems are reused. So it is very confusing to manage a business process from its design phase through the development up to the running operation.


Page 49

The Orchestration Designer, a flattened version of Microsoft Visio, has also its shortcomings. For example, there is the predefined sheet size on which the process has to be designed, which makes it very confusing if there are many steps involved within a process (see Figure 3.15). Also the reuse of frequently used process steps, like sending mails, as a consequence of lacking sub process objects, isn’t supported.

Figure 3.15 A business process in BizTalk Orchestration Designer

However, BizTalk’s origin also brings along some benefits. The routing of

documents between source and destinations, the mapping of document formats and the transformation of data is highly sophisticated. The BizTalk Editor facilitates the creation of proprietary document specifications for data mappings and the BizTalk Mapper is an easy to use graphical data mapping and transformation design tool. The integrated “functoids”, blocks for transformation functions, are a useful enhancement. Not only can the underlying script be modified with a single click, but also their usage is self-explanatory and simple. All things considered, BizTalk’s strengths are the Messaging Services for business to business integration in the means of data exchange and transformations, but the Orchestration services aren’t fully developed yet.


Page 50

3.3 webMethods integration platform

3.3.1 webMethods integration platform overview The webMethods [62] integration platform provides an infrastructure to enable end-

to-end integration across the enterprise. Since its beginning, webMethods platform is based on a service-oriented architecture which provides an abstraction layer, a so called service layer, between the client and the business logic being invoked. Thus, the platform allows the creation of services by integrating and Web services-enabling existing enterprise applications, data-bases, and other resources without any coding or development. A set of graphical development tools is provided to fulfil all these tasks. Also human interaction, created by the webMethods workflow tool, can be implemented as a service. Such services can be incorporated to model, execute and manage enterprise wide business processes. Additionally, webMethods delivers a broad range of security capabilities built into the integration platform, like public key infrastructure, digital signatures, X.509 and SSL. In addition, third party security capabilities can be added to address particular security needs. For managing and monitoring purposes webMethods offers a web-based view of the enterprise via the webMethods Administrator and the webMethods Monitor.


Page 51

3.3.2 Platform architecture webMethods has a flexible architecture model to meet various project requirements.

At its core, webMethods is a federated, broker based system, but other configurations, like a bus or a hub-and-spoke system, are possible. The common hub-and-spoke architecture provides centralized execution and management and is easy to develop. In situations where scalability and high performance loads as well as a distributed architecture is required, the webMethods integration platform supports connecting multiple hubs through a federated hub-and-spoke architecture or to a logical bus architecture. These multiple configuration options give a maximum flexibility in deployment depending on the project requirements. To achieve this flexibility the platform architecture introduces a decentralized execution model that enables code execution in adapters as well as in servers to maximize performance and scalability.

The foundation for webMethods' platform capabilities is the run-time architecture (see Figure 3.16). It provides the application connectivity and enables the reliable and secure communication between the different components. It is also responsible for data transformations and the business process execution to put application integration into action. It consists of three components that are relevant for the execution of a business process: the Adapters, the Integration Server and the Message Broker.

Figure 3.16 webMethods architecture - an example

The Integration Server implements the logic defined in the modeled business process. It represents an extended version of an adapter providing more capabilities than a single adapter. In contrast to the fully-distributed approach with independent adapters, the Integration Server features the housing of multiple adapters for tighter coupling of logic for faster and more complex data transformations and mappings, for example, obtaining a single view of data offered by multiple applications within a single process. The Integration Server leverages the flexible architecture delivered by webMethods as it provides more options for the design and deployment of an EAI solution.

Application Application

AdapterAdapter

Runtime Architecture Integration Server

Adapter Adapter

Application Application

Message Broker


Page 52

3.3.3 Installation and configuration The webMethods integration platform is available for MS Windows, AIX, AS/400,

Linux, HP-UX, Solaris and Mac OS X systems. On most systems an installation wizard is available to simplify the setup process. Depending on the selected components some settings have to be configured. An installation script file is being created by the wizard, which allows installing webMethods immediately via command line with the same selections and settings, for example, on another server. Afterwards, the necessary files are downloaded from the webMethods website during the installation process. The Integration Server is now ready to run. Further post-configurations are accomplished with the webMethods Administrator.

3.3.4 webMethods Developer The webMethods Developer is a graphical development tool for the implementation

of integration logic. It provides an integrated development environment to create, modify and test integration solutions. The implemented logic consists of several elements which are combined within a package. A package is like a container that contains all project related elements within a folder hierarchy for structuring. Besides, packages are manageable units that can be copied, replicated, archived or distributed within the platform for all deployment purposes. Elements include services, document types, schemes and event triggers. The most important elements are the services. They are an abstraction of business logic units, like function calls, with a specified set of input and output parameters. In between any type of function can be implemented, for example, a request to an integrated application, a Web services request, a java program or something akin. A large set of standard services for typical tasks is delivered with the platform. This includes services for file I/O, data manipulation, FTP, SMTP, HTTP, Web services and so on. Depending on the selected element the Developer displays the related editor on the right side. With the Developer tool all implementation tasks, like data mappings, data transformations, schema definitions etc., are accomplished. Only the business process modeling and the human workflow creation are done in separate tools.


Page 53

3.3.5 Workflow Modeler The graphical tool for the design and implementation of the business processes is the

webMethods Modeler. A business process is a multi-step interaction among various services. Within the Modeler the interactions and relationships of a sequence of activities in a process are described by using common process-modeling constructs (see Figure 3.17). The Modeler realizes the top-down approach which allows the modeling of a process without having to understand the technical aspects of the actual services being involved. Process models designed with the Modeler describe a sequence of steps and specify how information flows among them. In general, a process model is composed of steps, transitions, and documents.

A step represents a unit of work that is carried out by a service on an Integration Server or within an adapter. It can also represent an out-of-process activity that is performed by an external component that consumes or produces documents at execution time, for example, publish or subscribe to events. A transition represents the flow of control and connects one step to another with the possibility to define conditional transitions such as rule based transitions, a branch or a join. Documents represent the data consumed or produced by a step. A sequence of steps can be grouped into one sub process step. This enables a better structuring of complex workflows and the reusability of common sub processes, like sending a notification email.

Once the process design is completed, the underlying control logic for the individual steps has to be generated. Therefore, the Modeler connects to the participating Integration Servers that execute one or more steps in the process and generates run-time logic for each step. A new package is created, containing all process steps described as services.

Figure 3.17 Workflow Modeler


Page 54

3.3.6 Data Transformation Data transformation and mapping within the webMethods Integration Platform is

offered through the use of a graphical, drag-and-drop based editor inside the Developer. Document schemes and their contents can be visualized and document mappings and transformations can be defined with drag-and-drop techniques. The editor allows to “inline” transformation functions between mappings. The built-in portfolio of transformation functions such as date/time conversions, string manipulations, math functions, and so on can be extended with new transformation components written in Java. Alternatively, further data transformations can be implemented by XSLT and by custom programs in C/C++, COM, but their integration is a little bit more complex. The mappings and transformations can be tested directly within the editor to verify their functionality immediately.

Figure 3.18 Graphical data mapping


Page 55

3.3.7 Message Broker The webMethods Broker is the foundation for the scalable messaging backbone of

the webMethods Integration Platform. Networks of brokers manage the secure and reliable flow of information between systems and components connected to the platform, and perform the essential queuing, routing, and filtering required for integration. All common communication models are supplied to enable efficient information exchange between connected applications. There is the publish/subscribe pattern for asynchronous, sender and receiver decoupled communication and the synchronous request/reply pattern, similar to conventional client/server interactions, where a request is sent and a reply is expected. The third communication form, the conversational pattern, allows peer-to-peer communication between the connected components.

Additionally to the message broker webMethods supports many standard communication mechanisms like HTTP, HTTPS, SMTP/IMAP and FTP.

3.3.8 Adapters webMethods’ adapters provide connectivity to resources by seamlessly linking and

integrating them to the webMethods Integration Platform with minimal effort. After their installation the visual configuration of the adapters, performed inside the Administrator, enables fast integration of external resources. When a resource is connected successfully all available operations are displayed for browsing. These operations can be exposed as a service inside a package providing automatically the correct input and output parameters without any coding and without the need to understand low-level application details. Additionally, after the installation of an adapter, a package is available that contains a set of standard services according to the integrated system. Often, this is sufficient as these sets are generous equipped.

webMethods provides a wide spectrum of adapters for applications like SAP, Oracle, Peoplesoft, and so on. There are also technical adapters, for integrating resources via ODBC, JDBC, J2EE, .NET, LDAP, SMTP …, available. Additional connectivity to common messaging middleware, including WebSphere MQ, MSMQ, and JMS completes the spectrum. The included Adapter Development Kit enables the development of custom adapters for applications not supported by an out-of-the-box adapter.

Moreover, webMethods, an early pioneer in Web services, introduces Enterprise Web services, which are a combination of common Web services functionality and the webMethods platform features for integration and business process management. This allows a seamless integration of Web services enabled applications. Furthermore, any business process can be instantly exposed as a Web services or can call any existing Web services.


Page 56

3.3.9 Human workflow webMethods Workflow allows to integrate human resources into business processes.

A graphical environment provides the capabilities to build user interfaces interacting with business processes and their data without the need to write any line of code. Workflow generates all of the code necessary to run the complete workflow process either as a java program or as a JSP-based web page. Also the managing of deployment and run-time versioning as well as the activation of workflow processes is implemented in Workflow.

3.3.10 Management and Administration For management and administration purposes webMethods offers two separate web-

based tools: the webMethods Administrator and the webMethods Monitor. The first provides the configuration and administration for all available webMethods servers with a single, browser-based interface in a portal like manner (see Figure 3.19). Therefore, it displays a set of links to the administrative interfaces for the various servers on the platform. Each server is configurable regarding the requirements. For example, per default an Integration Server is logging into flat files, but it can be configured to log directly into a database instead. This is necessary if the webMethods Monitor is used for business process managing and monitoring purposes.

Figure 3.19 webMethods Administrator


Page 57

webMethods Monitor is an administrative tool to examine business processes activities, services, and documents that the integration platform is processing or has finished processing (see Figure 3.20). Besides viewing status information about processes, services, and documents, Monitor can also be used to perform control tasks such as suspending or resuming business processes or editing and resubmitting a document in process instances, for example, for testing and debugging process models. webMethods Monitor retrieves information about processes, services, and documents by querying the audit logs. The audit logs maintain a permanent record of certain types of activity on the integration platform.

Also a complete history of previously performed process instances is available for analyzing. If required, every step and document data can be permanently stored within the database for later examination.

Figure 3.20 webMethods Monitor

In addition to these management features, webMethods introduces the Open

Management Interface (OMI) [63]. OMI is a programmatic interface to the systems management functionality associated with a business integration platform. It was jointly developed by webMethods and Hewlett-Packard and is based on the open standards of XML, SOAP, and HTTP. This allows managing the integration platform with an existing OMI-compliant IT systems management tool to provide a view of the entire corporate IT infrastructure. webMethods itself offers the webMethods Manager, which is an OMI based console to manage webMethods’ integration platform.


Page 58

3.3.11 Conclusion The maturity of WebMethods integration platform 6.1 is not only due to the release

number. The product suite establishes also some concepts that are a considerable facility in managing and operating an EAI system. The utilized service oriented architecture needs some time for getting used to, but with a little bit of practice integration tasks can be implemented with ease. This service abstraction level hides all the functionality done by the component behind, whatever logic, system or resource is applied, for example, data transformation or application integration. It only requires a set of input parameters and delivers a set of output parameters – the result. This concept allows reusing services without exact knowledge of their implementation and their used resources. This enables rapid development of business processes that are using already implemented functionality. The service abstraction implicates another advantage – the seamless integration of Web services, because all services can be exposed as Web services and vice versa.

Another highlight is the webMethods Modeler, a sophisticated business process design tool with a pleasing graphical user interface which enables the straightforward realization even of complex business processes. The web-based administration and monitoring of the system offers the possibility to configure and manage all adapters and business processes via one uniform web application.

3.4 Product taxonomy

The three introduced EAI systems follow different approaches to solve integration problems. Therefore, they are based on different technologies, introduced in the second chapter. The following taxonomy of the products shows a general survey of the underlying technologies and architectures (compare with Table 2.1).


Page 59

webMethods BEA WebLogic MS BizTalk Messaging middleware to provide communication between the connectedsystems

A broker based messaging system allows to communicate between system components in asynchronous and synchronous communication patterns like publish-subscribe and request-reply

BEA’s message broker only supports the publish-subscribe mechanism

BizTalk’s messaging service is based on the publish-subscribe pattern

Application integration to connect the EAI run-time system to theintegrated applications

webMethods uses its own adapter format to integrate applications. Technical adapters are also available as well as an adapter development kit. A Web service connector allows to call external Web services and all internal services can be exposed as Web services for other systems

BEA uses JCA-compliant adapters and provides an Adapter development kit. Available are application adapters as well as technical adapters (JDBC, CORBA, …). Tightly integrated Web services support for inbound as well as outbound Web services completes the connectivity features

MS uses his proprietary application integration components (AIC), which are often offered from third-party companies. An adapter development kit is also available. One adapter is responsible for the use of external Web services. Internal services can't be exposed as Web services

Data transformation The graphical mapping tool uses a proprietary transformation language. All services inside the platform can be used as transformers between input and output values. Additional modules, like EDI or SAP, are delivered with a lot of standard document types.

BEA provides XQuery or XSLT language data mapping features via a graphical user interface between XML and non-XML data. Internally XML data is used. Standard data formats like EDI are available through the appropriate adapters

Graphical data mappings are represented with the help of XSLT, as all the documents passing through are converted into XML data.


Page 60

Process integration webMethods provides an enhanced modelling tool and supports the export of models in the BPEL4WS language. For each process step the component which executes it can be defined to enable distributed process execution. Transactions are also supported.

Business processes are designed via a graphical user interface and are represented in java. Processes are bound to the server where they are run.

BizTalk Orchestration uses XLANG, a process language, for the creation of processes via the graphical tool MS Visio. Processes can communicate via the messaging system between distributed components, but direct support for distributed processes isn't available. XLANG also supports transactions.

System architecture for a scalable, reliableinfrastructure to handleall integrated systems

The platform is based on distributed components, so that every scalable architecture is possible, depending on the requirements

BEA’s centralized server is based on the hub-and-spoke architecture and can be clustered.

BizTalk is a decentralized modular system which can be spread over many servers for scalability

Administration andmonitoring features

The platform administration and the adapters are configured via a web based interface. This also allows monitoring running processes as well as completed ones, with the possibility to modify data and resubmit it, for example, for debugging.

to manage the system

A monitor for runtime data as well as for long-time monitoring is available but without the ability to change data, for example, for debugging. The same interface provides Administration and configuration features.

Administration and the process monitoring is performed with operating system tools. So there is a restrictive usage for them. Process monitoring displays only steps without the data. Data is monitored via the DocumentTracker which observes the Messaging system activities.

Table 3.1 Product taxonomy

Case study

4 Case study

In this chapter a case study will show the strength and weaknesses of the EAI tools described in the last chapter and show the role of Web services within an integration solution. This study is performed in a large Austrian corporation with a complex and sophisticated IT infrastructure. The implemented process is very similar to an ongoing business process to get prominent and comparable results.

4.1 The problem

4.1.1 The environment The IT environment within the corporation consists of many applications for

different areas according to professional demands. There are SAP R/3-Systems, a Portal-Software, various Oracle and MS SQL-Databases, some niche solutions and a lot of proprietary applications executed on different operating systems which are all connected via a local area network. In addition, many business standards had to be maintained, for example, for B2B data exchange [64]. Because of the size of enterprise many fields of responsibility are separated into several departments, which require excellent internal communication for spanning business processes. Although, this is assured by sophisticated communication technologies this slows down the response time.

4.1.2 The claim of the management As the business changes day by day flexible business processes have to be installed.

Today many processes are involving several systems, requiring several human interactions, as well as complying with some business standards. So, the management wants to speed up the running time of these processes. Therefore, a case study is accomplished to obtain a proposal for an adequate solution.

4.1.3 Analysis of a flawed process The analysed process performs a store partner change, which simply means that old

partner data will be exchanged through data of the new partner. This seems to be simple, but in fact it is composed of numerous steps and involves many systems which are all fault prone. The process starts when a store partner sells his store to another legal entity. After that, all the transactions and the charging have to take place with the new partner. Thus a shop partner exchange in all invoked systems is necessary to handle all transactions correctly. Nowadays, the process runs as follows (also see Figure 4.1):

The partner request is sent via fax, email or by phone from the old partner to a predefined authority. This authority validates the retrieved request and creates manually a new debtor in the SAP system, if all the necessary data is available and correct, then the request is forwarded to another department, which is responsible for store supply. They serve the infrastructure which is required for the automated supply, and they are authorized to modify data within the supply-system. They check the

Page 61

Case study

request against the database and modify the data directly within the system. Then an XML file is generated via a manually executed script, including all the modified data, which is then sent to the corresponding logistic partner. Afterwards a response to the old partner informs him about the result of his request. Nowadays these steps need a few days to be performed and this transitional period causes some difficulties.

Figure 4.1 The business process today

Page 62

4.1.4 The challenge This business process shall now be optimized to speed up the cycle time with a

minimum of costs. The entire manually performed validation and modification tasks shall be automated, so that only an acknowledgment from the authority is required to perform all the steps, described above. The trigger for the process shall be realized through Web services. Administrators should be able to monitor the process in runtime and also a backup of the process activities shall be performed for later analysis and optimization.

Store partner exchange Manual data Validate requestrequest entry into SAP

Forward data to another department

Generate XML file for logistic partner

File Server Oracle DB

Check and modify old Send result to

and new partner data requestor

Case study

4.2 Solution strategies

The task description implies that any form of business process management is required. Furthermore, the involvement of heterogeneous distributed systems necessitates a middleware solution for information exchange and - as human interaction takes place - also a workflow tool will participate. As the challenge is, in fact, an integration problem, the usage of an EAI tool seems to be useful.

4.2.1 Implementation of the EAI based solution The task requires a centralized system which controls the whole business process and

collaborates with all other resources used in this process, like a SAP system or an authorized person. An EAI tool represents an appropriate solution, as it provides all (or most of) these features. A solution based on MS BizTalk implies that the human interaction part has to be implemented separately. With BEAs platform the configuration of the adapters is very complicated, and also its centralized architecture prevents enterprise wide integration solutions. So, only the webMethods integration platform is left, which seems to be the best companion for this task. It fulfills all the needs and supports the development by a set of graphical tools.

4.2.2 A solution approach based on Web services Another solution approach is based on Web services technology. As standards are

free and implemented in many products, they are an attractive alternative to expensive EAI tools. For proofing their capabilities, the following conceptual considerations are discussed:

The implementation of the example business case can be divided into four different parts. The first is the process triggering via a Web services request. This is, according to the specification, a classical Web services task and is solved with the help of Web services technologies. The second part is the business process and its flow control.

This can be handled with the BPEL4WS, but the logging of process activities and their later analysis is missing within this specification. The third part is the communication to other systems like SAP and databases. More and more systems are offering Web services interfaces, but EAI tools supply the usage of these systems as known. After a connection between the EAI system and the resource has been created, one can handle it, as if it were locally. For example, this allows to simple use ongoing SQL-statements. Last but not least, there is the human interaction part. Web services do not provide any facilities to implement human interaction at the moment. So, some of the tasks in our example can be realized with the help of Web services, but not all of them are convenient. As Web services can’t solve the problem in an adequate manner due to required additional coding to implement the missing functionalities this solution is abolished.

Page 63

Case study

4.2.3 Web services within EAI projects As described in the previous section Web services can’t be considered in every

integration project. Additionally to the reasons exemplified above, there are some common drawbacks. At first there is the performance of Web services. The overhead produced by SOAP messages can be a barrier. If there is a mission critical process sending a lot of messages with only a few bytes of information via a Web services, every message includes the whole amount of bytes of the SOAP envelope. Also the runtime schema validation for SOAP messages, which is not required when you use simple XML data, decreases performance. The combination of numerous Web services standards and specifications enables a secure, reliable and transactional business process execution, but with the drawback of high additional implementation expense and plenty of overhead data. Even if these disadvantages are irrelevant, the use of Web services isn’t always the best solution. For example, if you are implementing a point-to-point integration, and there are no plans for sharing that connection with other systems, there is no need to use Web services. Moreover, if the system you're trying to connect to have no or inadequate out-of-the-box Web services support, it may be easier to connect to it using an adapter or the systems native API.

So as we can see, Web services still don’t solve all enterprise integration problems, but their primary aim – providing services with the help of internet technologies – brings them a legitimate existence, even in integration projects. Due to the fact that there is only an agreement on the interface, not on the technology behind, Web services are useful for integration projects involving another organization or an external trading partner. This means that public information is minimized and corporate security is maximized while already known internet technologies are used.

Page 64

Case study

4.3 The solution based on the webMethods Integration platform

4.3.1 Description Electronically received shop partner exchange requests shall be automatically

processed. A manual acknowledgment of the request for confirmation is necessary for further processing the request. An acknowledged request modifies data in all shop-related systems immediately and notifications inform all responsible persons about successful exchanges. Furthermore, the logistic partner gets a file containing the information about the shop partner exchange for his charging system.

4.3.2 Context This is an initial system pilot accomplished within a case study to show the strengths

and weaknesses of EAI tools. For gaining comparable results the implemented business process is very similar to a realistic scenario. All integrated systems are already in test or productive use and the implementation has to deal with the existing environment as no changes are possible.

4.3.3 Scope Describe all project and implementation details concerning the integration tasks

required for achieving the target issues by using an EAI tool.

4.3.4 Risks As this project is a case study, there are no critical risks to mention. The only things

to consider are that all involved systems have to be available most of the time and that their functionalities are remaining unchanged. So they have to be handled carefully without influencing their consistency.

Page 65

Case study

4.4 System Description

The initial design has been created with the help of members of the customer department, which best know the demands, and a group of technicians which are responsible for the involved systems. As they are all professionals in their domains all requirements could be specified promptly. They are described in detail in the next sections.

4.4.1 Functional Requirements The following high level requirements from the business perspective have

implication in technical design. These requirements are critical to business and at any cost should be addressed in the technical design.

Requirement Method for implementation

1 Requests should be confirmed within 24 hours (except weekend)

A group of users has to be defined for acknowledging requests in time

2 Request validation have to take place for consistency reasons

Check request data and verify the content against the database

3 The business process has to be adaptable so that any new system that requires shop partner data can be easily integrated inside the process

Model the process with small units so that they can be replaced and new steps can be added

4.4.2 Design constraints The following constraints of the design have to be met in the technical

implementation.

Constraint Impact 1 SAP IDoc format The SAP specific IDoc format has to

be heeded

2 Logistic partner provides a data format specification that should be used.

Mapping of the data for the file extraction has to be designed according to this specification

4.4.3 Hardware environment For the hosting of the EAI tool a single server with 2 CPUs and 2GB RAM is

available. Because the project is in pilot stadium no additional hardware is available, for example, for clustering. The clients, for development and the confirmation tasks inside the process, are common desktop PCs. The hardware of all other systems isn’t relevant for this study and therefore not further specified.

Page 66

Case study

4.4.4 Software environment Most of the participating computers are based on Microsoft Windows operating

systems. The EAI server runs on a Windows 2000 Professional platform and all desktop PCs are running Windows XP. The EAI server also has a Microsoft SQL database installed to use, for example, as database for tracking issues. The corporate mail delivery system is based on Microsoft Exchange Server. The interface to the WWW is realized through an Apache Web Server with a Tomcat Application Server – both running in an AIX environment. The database servers are also based on AIX and running an Oracle 8i RDBMS. The SAP R/3 system, version 4.6b, runs on the open source operating system Linux.

4.4.5 Network environment The involved systems are all connected via the corporate local area network at a

speed of. 100Mb and an adequately dimensioned uplink to an internet service provider. An internal DNS server is responsible for all host names and a firewall secures the LAN from intruders from the WWW.

4.5 Logical Design

The logical design provides interface-level information of all the performed steps as well as some low-level implementation details. It describes all of the inputs, outputs, documents and sequences that are involved in this business process. Therefore, a logical grouping into services based on the business process takes place. At first, a Use case diagram illustrates the involved services (see Figure 5.1).

Figure 4.2 Logical Design Use Case diagram

Page 67

Case study

The following table describes the steps from the previous diagram: No. Use case Name / Scenario Name Description 1 Shop exchange request A Web services based triggering

of the business process and a validation of the form data

2 Create a new debitor A new debitor in the SAP R/3 system is created via a remote function call and the new debitor number is received via another call.

3 Notify administrator group A notification informs an administrator group of the newly arrived shop exchange request

4 Acknowledge Request Manual acknowledgement of an authorized person is required to confirm a shop exchange request.

5 Modify Partner data The shop database is updated with the new partner data while the old data is deleted.

6 Create export document A XML document, containing the partner data, for exchange with an external logistic partner is created.

7 Send success mail After all systems are updated, the new shop partner receives a mail informing about the successful shop partner exchange

4.5.1 Scenario: Shop exchange request

Scenario description The business process is triggered via a Web service. This Web service can be called

via a web page which is available only for successful authenticated shop partners at the corporate portal (see Figure 5.2). This authorization is required, as it is the only group with permission for shop partner exchanges. A completely filled and submitted form initiate’s a new business process instance. In the beginning, the received data is validated (A2). An invalid request terminates the business process after sending the error to the calling browser (A3b). If the submitted data is valid the requestor receives a return message (A3a) and the request is further processed.

Page 68

Case study

Figure 4.3 Shop partner exchange request form

Activity diagram

Figure 4.4 Web service request activity diagram

A1: Store partner change request

Page 69

[Error]

[All OK]

A3b: Return error message

A3a: Return success message

A2: Check input parameter

Case study

4.5.2 Scenario: Create new debtor

em for correct charging. has to be created inside the SAP system. A remote function

c

urns the debtor number, has to be invoked (A5a). Both function c

y diagram

Scenario description Each shop partner has to be a debtor in the SAP R/3 syst

Therefore, the new debtorall offered by the system provides this functionality (A4). The return value of this

function only contains an error number if an error happens which is sent to the administrators (A5b).

For further processing the new debtor number is required. So, another remote function call, which retalls are retried three times if they not succeed, to get around network failures,

timeouts or other errors. If an error happens a mail is sent to the system administrators (A5b).

Activit

Figure 4.5 Create new debtor activity diagram

A4: Create new debtorRetry 3 times

[Error]

[All OK]

A5b: Send error message to system administrators

A5a ew debtor number : Receive n

[Error]

[All OK]

Retry 3 times

Page 70

Case study

4.5.3 Scenario: Notify administrator group

Scenario description A shop partner exchange has to be reviewed and acknowledged by an authorized

person. Thus, a notification via mail reports the new request to an administration group (A6). It is sufficient when one person handles the request, but for substitution and backup reasons more than one person is notified. For security reasons the mail contains no request data. Only a link which directs to the correct page at the corporate portal is offered.

Activity diagram

Figure 4.6 Send notification mail to administrators activity diagram

A6: Send notification mail to the administrator group

4.5.4 Scenario: Acknowledgment of requests

Scenario description The acknowledgment or rejection of a request is done by authorized persons with the

help of a standard browser at the corporate portal site. All available data from the previous steps is displayed at the web page to facilitate the correct handling. The user decides if a request is acknowledged or declined by clicking the appropriate button (A7). If a request is declined the partner is informed by an email (A8).

Activity diagram

Figure 4.7 Acknowledge or decline requests activity diagram

Page 71

[Declined]

[Acknowledged]

A7: Human interaction for acknowledgment

A8: Send mail to declined partner

Case study

4.5.5 Scenario: Modify Partner data

Scenario description In the shop database information about the shops as well as the partners is stored.

Acknowledged requests entail a modification of these data. A requirement for the management of partner data is an existing and valid shop. Therefore, the existence of the related shop and its activity status has to be verified (A9). Also, the old partner data has to be available inside the database. Then the old partner is disabled and a new partner record is inserted inside the database (A10). Finally, the relationship between the shop and the new partner has to be registered (A11). Errors are handled as usual (A12).

Activity diagram

Figure 4.8 Modify partner data activity diagram

Page 72

[Error]

[All OK]

A12: Send error message to system administrators

A9: Check store and old partner

A10: Clean up old partner data and create new partner

[All OK]

A11: Modify store data

[Error]

Case study

4.5.6 Scenario: Create export document

Scenario description All shop partner data is also used by an external logistics company. This company

has to be informed about new partners or modifications of existing ones. The information is provided within XML documents, which are made available for the partner via an FTP server. These files contain all data changes and additions and are downloaded automatically by the logistics company twice a day. So, for each shop partner exchange a file has to be created containing all information of the new partner in a predefined document format.

Activity diagram

Figure 4.9 Create export document activity diagram

A13: Create XML file for export

4.5.7 Scenario: Send success mail

Scenario description When all previous scenarios are executed successfully, a mail to the new partner is

sent to inform him about the successful shop partner exchange. This is the final step in the business process.

Activity diagram

Figure 4.10 Send success mail activity diagram

A14: Send success-mail to new partner

Page 73

Case study

4.6 The business process diagram

The following illustration shows the resulting business process diagram created with the webMethods Modeler (Figure 4.11).

Figure 4.11 Business process model

Page 74

Case study

4.7 Detailed Design

4.7.1 Scenario: Shop exchange request

Processing details The data validation is done manually by implementing a few lines of java code. The

code snippet returns a string reflecting the correctness of the data or the error message. During this first step, only the syntactical correctness is verified but not the semantic one. Valid requests are producing a document containing the request data that is sent to the next process step. The document embeds the data conform to the schema specified in the “Documents” section below.

Input details Data Type / Name Description String ShopNr Unique number of the shop String OldDebitorNr Debitor number of the old partner Date OldValidfrom “Valid from” date of the old partner Date OldValidto “Valid to” date of the old partner String Name Name of the new partner String Address Address of the new partner String City City of the new partner String PostalCode Postal code of the new partner String Email Mail address of the new partner Date Valid from “Valid from” date of the new partner Date Valid to “Valid to” date of the new partner

Output details Data Name / Type Description String validationResult Contains the return code of the

validation task. Either the correctness of the submitted request data or an error message

Common Components Web services are standard components provided by the EAI system. Only the

definition of the required input data and the resulting output has to be done. Then all of the required files, also including the Web form, are created automatically.

Custom Components The validation of the received data has to be implemented manually. The code

skeleton for data processing is already created, so only a few lines of code implementing the validation logic are necessary. The logic executes some validation tasks and creates the appropriate output document as well as the return code.

Page 75

Case study

Documents This scenario produces documents embedding the request data for further processing

conform to the following document type definition: <schema targetNamespace="http://www.example.com/IPO"

xmlns="http://www.w3.org/2001/XMLSchema"

xmlns:xn="http://www.example.com/IPO">

<element name="requestData">

<complexType>

<sequence>

<element name="ShopNr" type="string"/>

<element name="OldDebitorNr" type="string"/>

<element name="OldValidFrom" type="date"/>

<element name="OldValidTo" type="date"/>

<element name="Name" type="string"/>

<element name="Address" type="string"/>

<element name="City" type="string"/>

<element name="PostalCode" type="string"/>

<element name="Email" type="string"/>

<element name="ValidFrom" type="date"/>

<element name="ValidTo" type="date"/>

</sequence>

</complexType>

</element>

</schema>

Error Handling The validation routine’s return code contains the error message in case of an error.

4.7.2 Scenario: Create new debtor

Processing details Initially, the SAP R/3 system has to be integrated in the EAI system to interact

seamlessly with it. Therefore, the SAP R/3 adapter is installed and configured with the appropriate settings. These settings include hostname, system number, client, user and password data. With the correct settings a connection can be established which allows the browsing of all available public remote function calls. By selecting the two necessary functions and exposing them as services inside the webMethods platform, their functionality is immediately available inside the Developer with the correct input and output parameters. The first function, IDOC_INPUT_DEBITOR, expects an IDOC as input document. This IDOC has a specific format and so all the data from the request has to be mapped to the corresponding fields. Furthermore, a set of fixed values has to be added to create a valid IDOC. By passing the email address to a second remote function, the corresponding debtor number is returned.

Page 76

Case study

Input details Data Name / Type Description String Name Name of the new partner String Address Address of the new partner String City City of the new partner String PostalCode Postal code of the new partner String Email Mail address of the new partner

Output details Data Name / Type Description String DebitorNr The newly created debtor number

Common Components The installed SAP R/3 adapter provides a set of standard services and predefined

document standards from SAP. This includes the used IDOC format schema and a service for executing remote function calls, which offers the possibility to call exposed function services with all necessary environmental settings, for example, the login data.

The data mappings as well as the addition of the fixed values are done with the help of the available graphical mapping tool.

If an error happens an email is sent via a built-in SMTP-service.

Custom Components None

Documents The IDOC document has the following data fields: Data Name Description MSGFN Message function

Fixed Value (=CREATE) BEGRU Authorization group

Fixed Value (=2) LAND1 Country Key

Fixed Value (=AT) NAME1 Name of the new partner

Char(50) ORT01 City of the new partner

Char(30) PSTLZ PostalCode of the new partner

Char(6) STRAS Address of the new partner

Char(80) EMAIL Mail Address of the new partner

Char(80) Page 77

Case study

Error Handling If a remote function call fails, it will be repeated three times. If an error happens, a

mail to the system administrators is sent.

4.7.3 Scenario: Notify administrator group

Processing details A mail is sent to a list of receivers containing a standardized mail text including the

link to the webpage for acknowledging the arrived requests.

Input details Data Name / Type Description StringList Receivers List of mail addresses Document requestData

Output details None.

Common Components For sending an email, a built-in service is available. It requires some parameters like

SMTP host and port for delivery and the mail subject and body. As these are all fixed values, they are inserted directly.

Custom Components None

Documents None

Error Handling None. This step just sends an email without any error handling.

4.7.4 Scenario: Acknowledgement of requests

Processing details Every request generates a new task that has to be completed by hand. Each task is

supplied with all the collected data that is displayed at the web page. Then it is assigned to a predefined group of responsible users. At the personalized web page the user sees all his pending tasks. By selecting one, the data is displayed and two buttons for task processing are offered. One button is for the acknowledgment of the request, and one for the rejection. As all the required task functionality is available out-of-the-box, the implementation of this scenario is very simple.

Page 78

Case study

Unless the user makes a decision, the business process continues processing based on the return value: • Declined requests are causing a mail to the requestor to inform him about the

rejection including the message inserted by the administrator. • Acknowledged requests are further processed (see section 5.3.5).

Input details Data Type / Name Description String ShopNr Unique number of the shop String OldDebitorNr Debitor number of the old partner Date OldValidfrom “Valid from” date of the old partner Date OldValidto “Valid to” date of the old partner String Name Name of the new partner String Address Address of the new partner String City City of the new partner String PostalCode Postal code of the new partner String Email Mail address of the new partner Date Valid from “Valid from” date of the new partner Date Valid to “Valid to” date of the new partner String DebtorNumber The just created debtor number of the

new partner

Output details Data Name / Type Description Boolean reqAck Reflects the decision of the user String message Allows the user to insert text to describe

his decision

Common Components The built-in workflow component is used to implement this scenario. This

component provides methods for task creation with the possibility to hand over data, user or group assignment, and methods for handling task completions.

Custom Components Only the automatically created web-page is modified, to fulfil the corporate design

guidelines within an HTML editor.

Documents None.

Error Handling The workflow component itself has provides some standard error handling routines.

For example, if the task creation, assignment or completion fails the administrator of the workflow model is notified. Improved error handling isn’t implemented.

Page 79

Case study

4.7.5 Scenario: Modify Partner data

Processing details The implementation of the steps inside this scenario are done with the help of the

JDBC adapter. So, initially a new JDBC connection has to be established by configuring a new connection with the appropriate settings (hostname, username, password ...). Then new JDBC adapter services can be created. This includes selectSQL, insertSQL and updateSQL services. So the first services are two SQL select statements to read the shop and partner data and verify their validity. The following two select statements are used to retrieve the desired results.

select oldpartnerid from partner

where supplierid = 3

and siteid = ShopNr

and debitornumber = OldDebtorNr

and validfrom <= OldValidfrom

and validto is null;

select count(1) from shop

where supplierid = 3

and siteid = ShopNr

and validfrom <= OldValidfrom

and (validto is null or validto >= OldValidto)

and partnered = oldpartnerid;

When both services return one record the data is valid and the scenario is further

processed. Otherwise the result variable is set to false, the database administrators are notified via mail about the invalid data and the scenario finishes. When all requirements are fulfilled, the old partner has to be disabled and the new one inserted. Before, the new partner data can be inserted, a new partner ID has to be obtained. As these ID’s are an ascending sequence the function “nextval” can be used to retrieve this value.

update partner set validto = OldValidto

where partnerid = oldpartnerid;

newpartnerid = partnerid.nextval;

insert into partner

(partnerid, supplierid, siteid, validfrom, validto,

partnername, partneraddress, city, postalcode,

debtornumber, partnermail, createdat, currencycode)

values

(newpartnerid, 3, ShopNr, ValidFrom, ValidTo, Name,

Address, City, PostalCode, DebtorNr, Email, sysdate, ’EUR’);

Page 80

Case study

Finally the shop data has to be updated with the new partner ID.

update shop set partnerid = newpartnerid

where siteid = ShopNr;

Input details Data Type / Name Description String ShopNr Unique number of the shop String OldDebtorNr Debtor number of the old partner Date OldValidfrom “Valid from” date of the old partner Date OldValidto “Valid to” date of the old partner String Name Name of the new partner String Address Address of the new partner String City City of the new partner String PostalCode Postal code of the new partner String Email Mail address of the new partner Date Valid from “Valid from” date of the new partner Date Valid to “Valid to” date of the new partner String DebtorNumber The debtor number of the new partner

Output details Data Name / Type Description Boolean result Contains the return code of the task.

Common Components The included JDBC adapter provides all required functionalities out of the box. It can

create services for each SQL statement type (select, update, insert …). All tasks inside this scenario can be processed with the usage of this component, except the mail sending. This is done with the help of the mail-client component.

Custom Components None.

Documents None.

Error Handling False database content and every other error cause a mail to the database

administrators containing the error message and the shop and partner data.

Page 81

Case study

4.7.6 Scenario: Create export document

Processing details The available data is transformed into an XML document with the help of the

mapping component. Then the created output is written into a file with the help of the file adapter simple by adding the mapping output to the file input parameter. Depending on the values available inside the XML file the logistic partner

Input details Data Type / Name Description String ShopNr Unique number of the shop String OldDebitorNr Debitor number of the old partner Date OldValidfrom “Valid from” date of the old partner Date OldValidto “Valid to” date of the old partner String Name Name of the new partner String Address Address of the new partner String City City of the new partner String PostalCode Postal code of the new partner String Email Mail address of the new partner Date Valid from “Valid from” date of the new partner Date Valid to “Valid to” date of the new partner String DebtorNumber The just created debtor number of the

new partner


Common Components The flat-file adapter offers services for creating, reading, modifying and deleting files

at all reachable hosts. After all the data is available in correct format, a file at the specified host, containing the XML-data, is created.

Custom Components Custom data mapping is used to create the XML document containing the data in a

valid format.

Page 82

Case study

Documents The export document conforms to the following schema:

<schema targetNamespace="http://www.example.com/IPO"

xmlns="http://www.w3.org/2001/XMLSchema"

xmlns:xn="http://www.example.com/IPO">

<element name="partnerData">

<element name="oldPartnerData">

<optional>

<element name="oldDebtorNr" type="string"/>

</optional>

</element>

<element name="newPartnerData">

<complexType>

<sequence>

<element name="DebtorNr" type="string"/>

<optional> <element name="ShopNr" type="string"/>

</optional> <optional> <element name="Name" type="string"/>

</optional> <optional> <element name="Address" type="string"/>

</optional> <optional> <element name="City" type="string"/>

</optional> <optional> <element name="PostalCode" type="string"/>

</optional> <optional> <element name="Email" type="string"/>

</optional>

<optional> <element name="ValidFrom" type="date"/>

</optional> <optional>

<element name="ValidTo" type="date"/>

<optional> </sequence>

</complexType>

</element>

</element>

</schema>

Page 83

Case study

Error Handling None.

4.7.7 Scenario: Send success mail

Processing details A new mail is created and sent to the new partner. All the data that was inserted into

the various systems is provided inside this message.

Input details Data Type / Name Description String ShopNr Unique number of the shop String Name Name of the new partner String Address Address of the new partner String City City of the new partner String PostalCode Postal code of the new partner String Email Mail address of the new partner Date Valid from “Valid from” date of the new partner Date Valid to “Valid to” date of the new partner String DebtorNumber The debtor number of the new partner


Common Components The mail client component is used to send the mail. It provides a service called

sendMail. This service has input parameters for mail server, receivers, subject and body to send a mail.

Custom Components None.

Documents None.

Error Handling The common component throws an exception if the mail server host cannot be

reached. No other errors are handled.

4.8 Summary

Finally, the resulting process is compiled to an executable process. Afterwards, the process was successfully tested by the involved team while it was monitored by the administrator of the system. The whole implementation was done in about 2 weeks. There were no lines of code written within the design phase. All requirements were accomplished with out-of-the-box features.

Page 84

Conclusion and future work

5 Conclusion and future work

There are lots of different Enterprise Application Integration systems available on the market and everyone is following his own approach to realize integration, but they all provide more than integration of different systems by connecting them to allow simple data exchange. Nowadays, EAI solutions are offering complex data transformations, mapping and routing technologies, implemented without writing any line of code, coupled with a business process engine as control instance to automate business processes including many systems as well as human interaction. In addition, mature monitoring and administration features to simplify management and operation of large EAI infrastructures are also included. So their benefit isn’t only the avoidance of interface programming. It’s the added value a process gets through controlling, advanced monitoring, guaranteed delivery and security features which significantly reduces operational costs. High sophisticated infrastructures at the lowest level of the systems allow scalable, reliable and secure architectures. This enables powerful and comprehensive business process integration, with the disadvantage of the high acquisition costs of such systems. Another advantage of EAI systems is that the development is done with the help of graphical tools, mostly without writing fault-prone lines of code. So the main design is concentrated on the definition of the business processes, which aren’t often as clear as they should be. Furthermore, their illustration and activity monitoring helps to understand the sequences among various systems and allows business process optimization.

Alternatively, Web services provide a large set of standards and specifications which can be used to implement an EAI system. Therefore, several available standards can be combined to achieve the main goals of enterprise application integration. There is the BPEL to implement a business process design and execution environment, the WS Communication Framework Architecture and the WS-Notification to enable a reliable messaging infrastructure and the WS-Security to implement miscellaneous security mechanisms. However, they cannot guarantee any quality of service as they are only a set of standards, based on internet technologies, and no products. They can enable integration but they aren’t EAI, yet.

Maybe this will change in a few years, because the evolution of Web services didn’t finished - it has just begun. New drafts and specifications will arise that add more and more features to Web services. Furthermore, Web services will have a lasting effect on applications as most future developments will implement Web services, with the result that application integration will give way to service integration. However, not only Web services advance, but also EAI systems. Future EAI-solutions will be based on the latest published Web services standards and will be enhanced by functionalities not represented in the Web services stack to enable reliable, secure, scalable and manageable enterprise application integration.

In future both technologies will probably grow together for the very reason that the trend from client-server architectures to service oriented ones requires integration and a standardization of the interacting services to facilitate the high demands on applications. So, it remains enthralling in the integration sector and maybe both technologies will not compete but complement each other in future.

Page 85

References

6 References

[1] Leah Gormly (2001), EAI Overview, http://eai.ittoolbox.com/browse.asp?c=EAIPeerPublishing&r=%2Fpub%2Feai%5Foverview%2Ehtm

[2] CIOL (2002), Integration levels in EAI,

http://www.ciol.com/content/search/showarticle1.asp?artid=40712 [3] Michael Crouch (2003), Enterprise Application Integration,

http://www.mbconsulting-inc.com/Downloads/EAI%20-%20Shaping%20the%20Reality%20Today.pdf

[4] http://www.bijonline.com/Article.asp?ArticleID=631&DepartmentId=6David

McGoveran (2003), Data Integration Part I-IX, http://www.bijonline.com/Department.asp?DepartmentID=6

[5] David Linthicum (2003), Defining B2B Application Integration,

http://www.sims.berkeley.edu/academics/courses/is290-4/s03/readings/linthicum.pdf [6] Andre Yee (2001), Demystifying Business Process Integration,

http://www.synchrobit.com/SynchroFlOW/Demystifying.doc [7] David Linthicum (2000), B2B Process Integration,

http://www.bijonline.com/PDF/B2B%20Process%20Integration%20-%20Linthicum.pdf

[8] Sacha Krakowiak (2003), What is Middleware, http://middleware.objectweb.org/ [9] Internet2 (2003), Overview of Middleware, http://middleware.internet2.edu/overview/ [10] Network Working Group (2000), A Report of a Workshop on Middleware,

http://www.ietf.org/rfc/rfc2768.txt [11] Nenad Medvidovic (2003), Middleware Technologies,

http://sunset.usc.edu/~neno/cs477_2003/April8.pdf [12] NWC (1995), A typology of middleware,

http://www.nwc.com/netdesign/cdmwtypo.htm#mom [13] Michael Stonebraker (2002), Too Much Middleware, SIGMOD Record, Vol. 31, No. I,

March 2002 [14] Wolfgang Emmerich (2000), Software Engineering and Middleware: A Roadmap,

ACM Press 2000 Page 86

mailto:[email protected]

http://eai.ittoolbox.com/browse.asp?c=EAIPeerPublishing&r=%2Fpub%2Feai%5Foverview%2Ehtm

http://eai.ittoolbox.com/browse.asp?c=EAIPeerPublishing&r=%2Fpub%2Feai%5Foverview%2Ehtm

mailto:[email protected]

http://www.ciol.com/content/search/showarticle1.asp?artid=40712

http://www.mbconsulting-inc.com/Downloads/EAI - Shaping the Reality Today.pdf

http://www.mbconsulting-inc.com/Downloads/EAI - Shaping the Reality Today.pdf

http://www.bijonline.com/Article.asp?ArticleID=631&DepartmentId=6

http://www.bijonline.com/Department.asp?DepartmentID=6

http://www.sims.berkeley.edu/academics/courses/is290-4/s03/readings/linthicum.pdf

http://www.synchrobit.com/SynchroFlOW/Demystifying.doc

http://www.bijonline.com/PDF/B2B Process Integration - Linthicum.pdf

http://www.bijonline.com/PDF/B2B Process Integration - Linthicum.pdf

http://middleware.objectweb.org/

http://middleware.internet2.edu/overview/

http://www.ietf.org/rfc/rfc2768.txt

http://sunset.usc.edu/~neno/cs477_2003/April8.pdf

References

[15] Philip A. Bernstein (1996), Middleware: A Model for Distributed System Services,

Communications of the ACM February 1996/Vol. 39, No. 2 [16] David Ritter (1998), The Middleware Muddle, SIGMOD Record, Vol. 27, No. 4,

December 1998 [17] Sun Microsystems (2002), Sun ONE Message Queue Administrator's Guide,

http://docs.sun.com/source/816-5922-10/index.html [18] IBM, IBM WebSphere Message Queues,

http://www-3.ibm.com/software/ts/mqseries/messaging/about/ [19] Dieter Gawlick (2002), Message queuing for business integration,

http://technet.oracle.com/products/aq/pdf/Gawlick.pdf [20] IBM, IBM WebSphere MQ, http://www-3.ibm.com/software/integration/wmq/

[21] Microsoft) Microsoft Message Queuing (MSMQ) Center,

http://www.microsoft.com/msmq [22] Steve Bolam, Introduction to MQSeries Publish/Subscribe, http://www-

3.ibm.com/software/integration/support/supportpacs/individual/supportpacs/ma0dintr.pdf

[23] Patrick Eugster, Pascal Felber, Rachid Guerraoui, Anne-Marie Kermarrec (2003), The

Many Faces of Publish/Subscribe, http://www.eurecom.fr/~felber/publications/CS-03.pdf, ACM Computing Surveys, Vol. 35., No. 2, June 2003

[24] TIBCO (1999), TIBCO Rendezvous,

http://www.tibco.com/solutions/products/active_enterprise/rv/default.jsp [25] David Linthicum (1999), Enterprise Application Integration

[26] Jeff Sutherland, Willem-Jan van den Heuvel (2002), Enterprise Application Integration

and complex adaptive systems, Communications of the ACM October 2002/Vol. 45, No. 10

[27] Jinyoul Lee, Keng Siau, Soongoo Hong (2003), Enterprise Integration with ERP and

EAI, Communications of the ACM February 2003/Vol. 46, No. 2 [28] Soon Huat Lim, Neal Juster, Alan de Pennington (1997), The Seven Major Aspects of

Enterprise Modelling and Integration: A position paper, SIGGROUP Bulletin, Vol. 18, No. 1 (April 1997)

Page 87

http://docs.sun.com/source/816-5922-10/index.html

http://www-3.ibm.com/software/ts/mqseries/messaging/about/

http://technet.oracle.com/products/aq/pdf/Gawlick.pdf

http://www-3.ibm.com/software/integration/wmq/

http://www.microsoft.com/msmq

http://www-3.ibm.com/software/integration/support/supportpacs/individual/supportpacs/ma0dintr.pdf



http://www.eurecom.fr/~felber/publications/CS-03.pdf

http://www.eurecom.fr/~felber/publications/CS-03.pdf

http://www.amazon.com/exec/obidos/search-handle-url/index=books&field-author=Linthicum%2C David S./102-9476154-5205758

References

[29] Josef Kraus (2002), Enterprise Application Integration (EAI) – Ein Anwenderleitfaden für die IT-Abteilung einer deutschen Großbank - Diplomarbeit

[30] Ovum (2003), Ovum Evaluates: Enterprise Application Integration,

http://www.ovum.com/go/content/EAI.htm

[31] Wilhelm Hasselbring (2000), Information system integration, Communications of the ACM June 2000/Vol. 43, No. 6

[32] Todd Sundsted (1999), XML and Java Technology Tackle Enterprise Application

Integration, http://developer.java.sun.com/developer/technicalArticles/Networking/XMLAndJava/

[33] W3C (1996), Extensible Markup Language, http://www.w3c.org/XML/

[34] Thomas Winkeler, Ernst Raupach, Lothar Westphal (2000), EAI – Enterprise

Application Integration - Die Pflicht vor der E-Business-Kür [35] W3C (1999), XSL Transformations, http://www.w3.org/TR/xslt [36] Mike Gatford (1998), Transaction Control,

http://www.soi.city.ac.uk/~tony/dbms/transaction_control.html

[37] Francisco Curbera, Rania Khalaf, Nirmal Mukhi, Stefan Tai, and Sanjiva Weerawarana (2003), Service-oriented computing: The next step in Web services, Communications of the ACM October 2003/Vol. 46, No. 10

[38] Heather Kreger (2003), Fulfilling the Web services promise, Communications of the

ACM June 2003/Vol. 46, No. 6 29 [39] Paul Fremantle, Sanjiva Weerawarana, Rania Khalaf (2002), Enterprise Services,

Communications of the ACM October 2002/Vol. 45, No. 10 [40] M.P. Papazoglou and D. Georgakopoulos, et al. (2003), Service Oriented computing

Communications of the ACM October 2003/Vol. 46, No. 10 [41] W3C (2002), Web services, http://www.w3c.org/2002/ws/ [42] Softonomy (2002), Web Services,

http://www.softonomy.com/pdf/wp_web_services.pdf [43] W3C (2001), Web services Description Language,

http://www.w3.org/TR/2001/NOTE-wsdl-20010315 [44] W3C (2000), Simple Object Access Protocol, http://www.w3.org/TR/SOAP/

Page 88

http://www.ovum.com/go/content/EAI.htm

http://developer.java.sun.com/developer/technicalArticles/Networking/XMLAndJava/

http://www.w3c.org/XML/

http://www.w3.org/TR/xslt

http://www.soi.city.ac.uk/~tony/dbms/transaction_control.html

http://www.w3c.org/2002/ws/

http://www.softonomy.com/pdf/wp_web_services.pdf

http://www.w3.org/TR/2001/NOTE-wsdl-20010315

http://www.w3.org/TR/SOAP/

References

[45] UDDI (2000), UDDI technical White Paper, http://www.uddi.org/pubs/Iru_UDDI_Technical_White_Paper.pdf

[46] Andrews, Curbera, Dholakia, Goland, Klein, Leymann, Liu, Roller, Smith, Thatte, Trickovic, Weerawarana (2003) Business Process Execution Language for Web Services ftp://www6.software.ibm.com/software/developer/library/ws-bpel.pdf

[47] Bunting, Chapman, Hurley, Little, Mischkinsky, Newcomer, Webber, Swenson (2003), Web Services Composite Application Framework http://www.oasis-open.org/committees/download.php/4343/WS-CAF%20Primer.pdf

[48] Bunting, Chapman, Hurley, Little, Mischkinsky, Newcomer, Webber, Swenson (2003), Web Services Context http://www.arjuna.com/library/specs/ws_caf_1-0/WS-CTX.pdf

[49] D. Langworthy (2003), Web Services Coordination,

ftp://www6.software.ibm.com/software/developer/library/ws-coordination.pdf

[50] Cabrera, Copeland, Cox, Freund, Klein, Storey, Thatte (2002), Web Services Transactions, ftp://www6.software.ibm.com/software/developer/library/ws-transpec.pdf

[51] Atkinson, Della-Libera, Hada, Hondo, Hallam-Baker, Kaler, Klein, LaMacchia, Leach, Manferdelli, Maruyama, Nadalin, Nagaratnam, Prafullchandra, Shewchuk, Simon (2002), Web Services Security, ftp://www6.software.ibm.com/software/developer/library/ws-secure.pdf

[52] Graham, Niblett, Chapell, Lewis, Nagaratnam, Parikh, Patil, Samdarshi, Sedukhin,

Snelling, Tuecke, Vambenepe, Weihl (2004), Publish-Subscribe notification for Web services http://www-106.ibm.com/developerworks/library/ws-pubsub/WS-PubSub.pdf

[53] Czajkowski, Ferguson, Foster, Frey, Graham, Sedukhin, Snelling, Tuecke, Vambenepe (2004), Web Services Resource Framework http://www.ibm.com/developerworks/library/ws-resource/ws-wsrf.pdf

[54] BEA Systems (2003), Introducing BEA Web Logic Plattform, http://edocs.bea.com/platform/docs81/pdf/intro.pdf

[55] W3C (2003), XML Query Language, http://www.w3.org/TR/xquery/ [56] SUN Microsystems, J2EE Connector Architecture (JCA),

http://java.sun.com/j2ee/connector/ [57] BEA (2003), SAP R/3 Adapter Installation Guide,

http://edocs.bea.com/wladapters/sap/docs81/pdf/install.pdf

Page 89

http://www.uddi.org/pubs/Iru_UDDI_Technical_White_Paper.pdf

ftp://www6.software.ibm.com/software/developer/library/ws-bpel.pdf

http://www.oasis-open.org/committees/download.php/4343/WS-CAF Primer.pdf

http://www.arjuna.com/library/specs/ws_caf_1-0/WS-CTX.pdf

ftp://www6.software.ibm.com/software/developer/library/ws-coordination.pdf

ftp://www6.software.ibm.com/software/developer/library/ws-transpec.pdf

ftp://www6.software.ibm.com/software/developer/library/ws-transpec.pdf

ftp://www6.software.ibm.com/software/developer/library/ws-secure.pdf

http://www-106.ibm.com/developerworks/library/ws-pubsub/WS-PubSub.pdf

http://www.ibm.com/developerworks/library/ws-resource/ws-wsrf.pdf

http://edocs.bea.com/platform/docs81/pdf/intro.pdf

http://www.w3.org/TR/xquery/

http://java.sun.com/j2ee/connector/

http://edocs.bea.com/wladapters/sap/docs81/pdf/install.pdf

References

[58] BEA (2003), RDBMS Adapter Installation Guide, http://edocs.bea.com/wladapters/rdbms/docs811/pdf/install.pdf

[59] Microsoft, Microsoft BizTalk Server, http://www.microsoft.com/biztalk/

[60] Satish Thatte (2001) XLANG - Web Services for Business Process Design

http://www.gotdotnet.com/team/xml_wsspecs/xlang-c/default.htm

[61] J. Kohl, C. Neuman (1993), The Kerberos Network Authentication Service, http://www.ietf.org/rfc/rfc1510.txt

[62] webMethods (2003), webMethods Integration platform,

http://www.webmethods.com/content/1,1107,Overview,FF.html [63] webMethods (2003) The Open Management Interface (OMI),

http://www.webmethods.com/OMI_Spec_index/ [64] Brahim Medjahed, Boualem Benatallah, Athman Bouguettaya, Anne H. H. Ngu,

Ahmed K. Elmagarmid (2003), Business-to-business interactions: issues and enabling technologies, Springer-Verlag

Page 90

http://edocs.bea.com/wladapters/rdbms/docs811/pdf/install.pdf

http://www.microsoft.com/biztalk/

http://www.gotdotnet.com/team/xml_wsspecs/xlang-c/default.htm

http://www.ietf.org/rfc/rfc1510.txt

http://www.webmethods.com/content/1,1107,Overview,FF.html

http://www.webmethods.com/OMI_Spec_index/