Service oriented online architecture with Mule ESB

Service-Oriented Online Architecture with Mule

A different approach to building a SOA

Mario-Leander Reimer

Software Architect @ QAware GmbH

Mule Summit Munich, 02. May 2012

Agenda

QAware 22. May 2012

1. Business context and problems faced

2. The idea of a service-oriented online architecture

3. How and why we selected Mule

4. Overview and examples of Mule use cases

5. Best practices and learnings

Business logic and backend access in the portal needs to available to other portals

Server applications only run on specialized hardware and application platform

Everyone is talking to everyone, point-to-point communication is difficult to manage

Business context and problems faced

Existing online infrastructure was complex, expensive

to maintain and could not be used by other portals

2. May 2012 3QAware

Corporate Network

(Head Quarter)

Portal

DMZDistribution

Partner LAN

(*) as Reverse Proxy

Web-

Server (*)

Web-

Server (*)

Browser Oracle

DB2

BackendBackend

BackendBackend

Client-

ApplicationClient-

Application

DataPlatform

ComponentApplication

Platform

Online-

ServerServer-

ApplicationServer-

Application

- Spare Parts

- Technical Info

- …

An ESB-centric service-oriented online architecture is

easier to manage and more extensible

QAware 42. May 2012


Online

Client

Corporate Network

(Head Quarter)DMZ

Primary

ESBBackend

Portal

Online-

Server


Local Area Network

Web-

Server (*)

Web-

Server (*)

Browser

■ The migration of the server applications to standardized

hardware proved to be rather difficult due to close coupling to

platform

■ To get rid of the application platform components we had to

identify the all required services for the online scenario

■ Then a light-weight surrogate based on Mule was implemented

to provide these services to the applications instead

■ The portal independent services had to be identified and

extracted from the portal

■ Access to all backend systems is now provided by one central

Mule ESB instance, implementing logging and security

■ The maintenance costs for the simplified architecture have

already decreased significantly

■ New server application instances can be deploy transparently

within hours instead of days now


The goal was to simplify the architecture, unify the way

backend systems are accessed and cut operation costs

2. May 2012 5QAware

Agenda

QAware 62. May 2012






The improved service-oriented online architecture can be

composed using different building blocks

QAware 72. May 2012

The idea of a service-oriented online architecture

Online

Client

Corporate Network

(Head Quarter)DMZ

Primary

ESBBackend

Portal

Online-

Server


Local Area Network

Web-

Server (*)

Web-

Server (*)

Web-

Server (*)

Local

ESB

Browser

A secondary ESB building block enables the deployment

of dedicated online servers in national networks

QAware 82. May 2012


Local

ESB

Corporate Network

(Head Quarter)

Primary

ESBBackend

Portal

Online-

Server

National Corporate

Network (USA)

Secondary

ESB

DMZ

Backend‘

Local Area

Network


Web-

Server (*)

Web-

Server (*)

Web-

Server (*)

Browser

Online-

Client

Multiple portal and online server blocks can be combined

to support different user groups and network locations

QAware 92. May 2012


Corporate Network

Primary

ESBBackend

Portal

Online-

Server

National Corporate

Network (USA)

Secondary

ESB

DMZ

Backend‘

Local Area

Network

Web-

Server (*)

Web-

Server (*)


Local

ESB

Browser

Online-

Client

Web-

Server (*)

Portal

Online-

Server

Agenda

QAware 102. May 2012






■ We did not Google „open source ESB“ to select Mule …

■ Instead we did a qualitative and quantitative comparison of major open source ESB

products using different criteria:

■ Primary: professional maintenance, commercial support with SLAs, licensing, performance,

operations by IT department possible

■ Secondary: documentation, code quality, activity and size of community, Spring support, sync

and async communication, supported standards, app server integration, development tools

■ Mule quickly emerged as the favored ESB product, followed by Fuse ESB and WSO2

■ Static analysis of the Mule sources (Sonar,

Structure101) showed acceptable quality

■ Modularization and project structure looks well-

thought-out and enables light-weight deployment

■ Good code quality, in spite of found violations and

partially low documentation

■ Test coverage is reasonably high to ensure correct

function in case of changes

How and why we selected Mule

Based on the proposed architecture scenarios we could

identify the requirements on the ESB product

2. May 2012 11QAware

■ Chosen products: Mule ESB, WSO2

and Fuse ESB

■ Are all 3 uses cases supported?

■ Development model: used frameworks,

supported IDEs, build tools?

■ Learning curve: How good is the

documentation? Clear API?

■ Development effort: How long does it

take to implement the uses cases?

■ Mule ESB scored best in comparison,

closely followed by WSO2


Implementation of a small PoC prototype to get a first

impression of the chosen products


1 2 3WS-Call POJO Provider External WS-Call (Asnyc) WS-Proxy with Transform

■ Different load scenarios with constant and

increasing parallel requests (Apache JMeter)

■ Measurement of performance relevant

metrics using Software-EKG

■ Live profiling of system behavior (JProfiler)

■ All findings have been reported to MuleSoft

■ Together with MuleSoft we were able to solve

all the found issues:

■ MuleSoft supplied a working patch for the

Registry synchronization issue within 2 days

■ Other issues could simply be addressed using

the optimized configuration parameters (thread

pool settings, …) supplied to us

■ This was decisive for the confidence and the

final decision for Mule ESB


Intensive performance tests uncovered several findings

(with Mule 3.1.1) …


Agenda







■ Requirement: Mule ESB had to be deployed

as a Java web application to be operated by

the IT department

■ Embedding Mule into a web app is pretty

straight forward using the a context listener

■ Custom listener implementation required to

customize working directory and various other

system properties

■ Each Glassfish v2.1 app server instance runs

with 512mb heap space on SuSe linux

■ The reverse proxies are Apache 2.2 instances

using mod_proxy with stickyness

■ The final WAR is simply assembled from the

Maven artifacts of the individual Mule apps

■ Hot-deployment of individual apps not required

Overview and examples of Mule use cases

Clustered deployment of Mule ESB as a web application

for scalability and high availability



Unified web service interface to access details user

from heterogeneous data sources


■ Access to the endpoint is controlled using a Spring security filter

■ Each data source has specific POJO implementation or private flow

■ Choice is based on payload using a Groovy evaluator

■ Only minor Java code required

■ Web service interface and types

■ Custom transformers

■ Choice uses CXF operation header

■ XSLT to transform XML/RPC to

JAXB XML structure


Web service to XML/RPC service adapter to access the

BZST service for simple and qualified VAT checks


■ Web service interface and types defined as

POJI and POJOs with JAX-WS annotations

■ The service component only performs

validation and preprocessing of request


Web service to email service adapter to send support

requests to a ticketing backend system


■ The actual sending using an SMTP

connector is performed asynchronous

■ Custom transformer uses Velocity to

convert request object to email body

Agenda







■ Mule provides several built-in components

to test Mule XML and flow definitions

■ The MuleFunctionalTest allowed us to test

our flows within the IDE

■ No deployment to a standalone instance

required, thus reducing turn-around times

■ The MuleClient is not really intuitive to use

■ Smart combination of SoapUI test cases

together with mock services allowed 100%

local and off-site development

■ Learning: develop as much as possible as

POJOs and use „traditional“ unit testing

■ Learning: take the time to write a good

mock for the service you are integrating

Best practices and learnings

Test driven development using MuleFunctionalTests,

SoapUI tests and mock services


■ „The Leanest, Meanest ESB: Mule ESB is the world's most efficient Enterprise

Service Bus” (http://www.mulesoft.com/mule-esb-small-footprint)

■ We went well below the mentioned figures by building a custom Mule distribution

tuned and optimized for our specific use cases

■ Based on the default distribution assembly XML found in the Mule community

sources, we

1. got rid of everything not required in production, mainly docs and examples, but also not

required Tanuki EXE wrapper binaries, etc.pp.;

2. selected only the required Mule modules and transports our uses cases really required, this

reduced the amount of 3rd party libs significantly;

3. used Maven dependency management to have full control of all used 3rd party libraries,

used more recent versions where possible (e.g. Spring, CXF, Saxon)

4. added our Mule apps and their dependencies, then repackaged

■ Thorough load tests lead to optimized JVM parameters and high performance:-Xmx=128m -Xms=128m -XX:MaxPermSize=64m -XX:NewRatio=2 -XX:SurvivorRatio=12 -XX:+UseParallelGC

-XX:+UseParallelOldGC


Building a custom Mule distribution for 100% control of

all dependencies and optimal performance


96 MB

30 MB

37 MB

http://www.mulesoft.com/mule-esb-small-footprint

■ A migration of the whole infrastructure in one go would have been impossible; the system needs to be available around the clock

■ Instead a staged migration of the infrastructure components and applications has been used:

■ Phase 1: Migration of all online servers, application by application, introduction of the primary ESB with first required services

■ Phase 2: Integration of a new online portal, operated in parallel to the old portal infrastructure

■ Phase 3: Migration of all „legacy“ portals to access the new online infrastructure components

■ After each phase the behavior of the new components was monitored closely to detect any problems in production

■ The services and backend systems integrated by the primary ESB instance constantly grew (I might still be growing in next phases)


No big bang: start small and migrate in several phases


Questions & Answers

QAware GmbH

Quality and agility in software engineering.

Service oriented online architecture with Mule ESB

Software

asreverse proxyweb server

online scenariothen

simplified architecture

examples of mule use

applicationserver application

application platformeveryone

business logic

central mule esb instance