Guerrilla SOA How to fight back when a vendor takes control of your enterprise Dr. Jim Webber Dr. Jim Webber Service-Oriented Systems Practice Lead Service-Oriented Systems Practice Lead ThoughtWorks ThoughtWorks http://jim.webber.name http://jim.webber.name
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Guerrilla SOAHow to fight back when a vendor takes control of your enterprise
Dr. Jim WebberDr. Jim WebberService-Oriented Systems Practice LeadService-Oriented Systems Practice LeadThoughtWorksThoughtWorkshttp://jim.webber.namehttp://jim.webber.name
Fundamental PremiseFundamental Premise
There are two things money cannot buy:There are two things money cannot buy:
Business domain tierBusiness domain tierIntegration at the object level, as typified by CORBA, Integration at the object level, as typified by CORBA, DCOM etcDCOM etc
User interface User interface Screen scraping, revamping, etc.Screen scraping, revamping, etc.
Last resort, when an application offers no other hooksLast resort, when an application offers no other hooks
To ESB or not to ESB, that is the To ESB or not to ESB, that is the questionquestion
Product vendors are keen to provide product Product vendors are keen to provide product solution for everythingsolution for everything
Or to supply “consultantware” solutionsOr to supply “consultantware” solutions
The Enterprise Service Bus is the latest The Enterprise Service Bus is the latest incarnation of EAI technology that supports a incarnation of EAI technology that supports a number of useful functions:number of useful functions:
Time and technology pressuresTime and technology pressures
Path of least resistance for individual Path of least resistance for individual applicationsapplications
This is the thin end of the wedge, technical debt This is the thin end of the wedge, technical debt can only increase from herecan only increase from here
Help!Help!
Vendor Solutions AppearVendor Solutions Appear
Business needs to Business needs to competecompete
IT needs to be responsiveIT needs to be responsive
SOA gives IT a business SOA gives IT a business process focusprocess focus
Web Services are the Web Services are the most sensible way to most sensible way to implement SOAimplement SOA
More proprietary More proprietary middleware is the middleware is the answer!answer!
Integration Two Years LaterIntegration Two Years Later
Enterprise Service Bus
Skeletons in the Closet...Skeletons in the Closet...
Enterprise Service Bus
The Appealing Rationale for ESB...The Appealing Rationale for ESB...
Perceived single framework for all integration Perceived single framework for all integration needsneeds
Perceived simple connectivity between systemsPerceived simple connectivity between systems
Some features for security, reliable delivery, etc.Some features for security, reliable delivery, etc.
All you have to do is agree to lock yourself into a All you have to do is agree to lock yourself into a ESB and all this can be yours...ESB and all this can be yours...
...And the Reality...And the Reality
The mess is swept under the carpetThe mess is swept under the carpetThe spaghetti is still there, but it’s hidden inside a vendor boxThe spaghetti is still there, but it’s hidden inside a vendor box
But the spaghetti is worse with an ESBBut the spaghetti is worse with an ESBMixing business rules, transformations, QoS etc with connectorsMixing business rules, transformations, QoS etc with connectors
What if I wanted to remove or replace my current ESB What if I wanted to remove or replace my current ESB platform?platform?
Vendor lock-in of the whole network!Vendor lock-in of the whole network!
ESBs are proprietary, so no guarantees that the messages ESBs are proprietary, so no guarantees that the messages transmitted across the bus are actually based on any open transmitted across the bus are actually based on any open protocolprotocol
Held to ransom by the ESB vendor!Held to ransom by the ESB vendor!Cannot easily replace one ESB with anotherCannot easily replace one ESB with another
Can only easily integrate systems for which the ESB vendor Can only easily integrate systems for which the ESB vendor provides specific adaptorsprovides specific adaptors
Or invest your money into extending their productOr invest your money into extending their product
Isn't this precisely what we're trying to get away Isn't this precisely what we're trying to get away from?from?
Integration should happen on the wire by default, Integration should happen on the wire by default, not inside some servernot inside some server
The ESB approach eschews the dumb network, The ESB approach eschews the dumb network, smart endpoint notion that underpins scalable, smart endpoint notion that underpins scalable, robust systemsrobust systems
ESB vendors are the new telcos – telling us that ESB vendors are the new telcos – telling us that smarts in the network is for our own goodsmarts in the network is for our own good
But let’s see how ESBs play out over the longer But let’s see how ESBs play out over the longer termterm
Integration five years from nowIntegration five years from nowAccounting Marketing
SupportProduct Development
ResearchIT
Enterprise Service Bus
Accounting Marketing
SupportProduct Development
ResearchIT
Integration ten years from nowIntegration ten years from now
ESB
How did this happen?How did this happen?
Same old story:Same old story:Tactical decisionsTactical decisions
Time and technology pressuresTime and technology pressures
Path of least resistance for individual applicationsPath of least resistance for individual applications
Centralised ownership of the ESB sometimes is Centralised ownership of the ESB sometimes is an inhibitoran inhibitor
Too much effort to get on the bus, technically, Too much effort to get on the bus, technically, politicallypolitically
Individuals always mean to redress hacked Individuals always mean to redress hacked integrationsintegrations
But seldom do – it’s too hard when systems are liveBut seldom do – it’s too hard when systems are live
Spaghetti is a fact of lifeSpaghetti is a fact of life
Businesses changeBusinesses change
Processes changeProcesses change
Applications changeApplications change
Integration changesIntegration changes
Need an enterprise computing strategy that:Need an enterprise computing strategy that:Reflects the changing structure of the business;Reflects the changing structure of the business;
Is spaghetti-friendly;Is spaghetti-friendly;
Commoditised;Commoditised;
Robust, secure, dependable, etc.Robust, secure, dependable, etc.
Business-Led IntegrationBusiness-Led Integration
ESBs integrate with whatever existing systems exposeESBs integrate with whatever existing systems exposeGreen screen, web pages, CORBA objects, XML, etcGreen screen, web pages, CORBA objects, XML, etc
Integration happens at a low levelIntegration happens at a low levelMapping of bits and bytes of one variety onto bits and bytes of Mapping of bits and bytes of one variety onto bits and bytes of another formatanother format
This makes it hard to engage business in such projectsThis makes it hard to engage business in such projectsWithout business benefit no software has valueWithout business benefit no software has value
Integration is currently opaque to the businessIntegration is currently opaque to the business
Business must be involved in integration projects – not Business must be involved in integration projects – not just initiate themjust initiate them
The integration domain must use the same vocabulary as the The integration domain must use the same vocabulary as the business domainbusiness domain
Focus on business-meaningful process orchestrationFocus on business-meaningful process orchestration
SOA and Web Services ApproachSOA and Web Services Approach
Applications (or subsets of applications) are Applications (or subsets of applications) are identified as being service-amenableidentified as being service-amenable
Or (sub) processes are identified for which there is no Or (sub) processes are identified for which there is no existing application/serviceexisting application/service
Web Services infrastructure is layered on top of Web Services infrastructure is layered on top of the application, exposing a SOAP interface to the the application, exposing a SOAP interface to the rest of the networkrest of the network
Business meaningful message exchangesBusiness meaningful message exchanges
Other services consume the functionality via Other services consume the functionality via SOAP message exchangesSOAP message exchanges
Traditional integration infrastructure is kept within Traditional integration infrastructure is kept within the Web Service implementation, if used at allthe Web Service implementation, if used at all
Building the Service-Oriented Building the Service-Oriented EnterpriseEnterprise
SOAP becomes the ubiquitous transfer mechanism SOAP becomes the ubiquitous transfer mechanism across the enterprise (or Internet!)across the enterprise (or Internet!)
In effect, SOAP messages are the “EAI backbone” In effect, SOAP messages are the “EAI backbone” The underlying transport protocols are arbitraryThe underlying transport protocols are arbitrary
Applications understand SOAP messages nativelyApplications understand SOAP messages nativelyTrue end to end integration, but maintains loose couplingTrue end to end integration, but maintains loose coupling
In this context, existing ESB/EAI software becomes a In this context, existing ESB/EAI software becomes a toolkit for implementing individual Web Servicestoolkit for implementing individual Web Services
But integration happens at the SOAP levelBut integration happens at the SOAP levelCan commoditise what’s underneathCan commoditise what’s underneath
The QoS functionality that a The QoS functionality that a Web Service requires is Web Service requires is implemented on a per-service implemented on a per-service basisbasis
Not “one size fits all”Not “one size fits all”
Implement only those QoS Implement only those QoS protocols that the service protocols that the service currently needscurrently needs
Push the integration Push the integration functionality to the edgesfunctionality to the edges
SOAP + WS-Addressing SOAP + WS-Addressing becomes the “bus”becomes the “bus”
Incremental and autonomousIncremental and autonomousDeliver high business-value Deliver high business-value services first!services first!
EAI/ESB frameworks are fine for application integrationEAI/ESB frameworks are fine for application integrationA framework for development of (distributed) applicationsA framework for development of (distributed) applications
Think of the EAI toolkit as a container for your applicationThink of the EAI toolkit as a container for your applicationApplication versus enterprise frameworkApplication versus enterprise framework
Bus
Choreography/Rules/Routing/Transformations
Adapter Adapter Adapter
Application Domain
...and Composite Business ...and Composite Business ProcessesProcesses
Processes across the enterprise consume and Processes across the enterprise consume and coordinate lower-level applicationscoordinate lower-level applications
Exposed via standards-based servicesExposed via standards-based services
Bus
Choreography/Rules/Routing/Transformations
Adapter Adapter Adapter
Application Domain
Gateway
Enterprise Process Domain
SOAP Messaging,
WS-*
Web Services Standards: The State Web Services Standards: The State of Playof Play
Not all of the WS-* QoS protocols are ready yetNot all of the WS-* QoS protocols are ready yetGenerally in late stages of standardisation and early Generally in late stages of standardisation and early adoptionadoption
WS-Security is ready; WS-ReliableExchange WS-Security is ready; WS-ReliableExchange (WS-RX) in standardisation; WS-AT/BA ready on (WS-RX) in standardisation; WS-AT/BA ready on Java platform; WS-AT on .NetJava platform; WS-AT on .Net
Web Services toolkits for the rest will be Web Services toolkits for the rest will be available in the next year or soavailable in the next year or so
WS-SecurityWS-SecurityMature, widely implemented, and interoperableMature, widely implemented, and interoperable
WS-I Basic Security Profile tooWS-I Basic Security Profile too
Gives end-to-end securityGives end-to-end securityPrivacy, integrityPrivacy, integrityCan be used for authentication, non-repudiationCan be used for authentication, non-repudiation
http tcp jms
logical point-to-point SOAP message transfer
resources
message processing
Web Service logic
transport
resources
message processing
Web Service logic
transport
intermediate intermediate
Reliable SOAP MessagingReliable SOAP Messaging
Non-interoperable implementations from two Non-interoperable implementations from two competing standards todaycompeting standards today
Single, unified standard (WS-ReliableMessaging) Single, unified standard (WS-ReliableMessaging) being developed (by WS-RC committee)being developed (by WS-RC committee)
Not yet widely implementedNot yet widely implemented
Cannot achieve interoperable RM at the SOAP Cannot achieve interoperable RM at the SOAP level todaylevel todayInstead, use existing, mature, transportsInstead, use existing, mature, transports
MQ, JMS, MSMQ, etcMQ, JMS, MSMQ, etc
Reliable Transport AlternativesReliable Transport Alternatives
SOAP messages are transport agnosticSOAP messages are transport agnosticChange the transport, change the bindingChange the transport, change the binding
Route messages over a reliable channel Route messages over a reliable channel E.g. MQ, MSMQ, JMS etcE.g. MQ, MSMQ, JMS etc
ServiceService
TCPQueued Transport
Coordinating ServicesCoordinating ServicesProcesses need end-to-end guaranteesProcesses need end-to-end guarantees
SOAP messaging is the communica tion channel for. ( ) applica tions The ESB if it exis ts is pushed to the
.endpoints
Same Old ArchitectsSame Old Architects
Business people and application architects design Business people and application architects design business-centric workflows which consume servicesbusiness-centric workflows which consume services
Re-using the functionality already deployed into the service Re-using the functionality already deployed into the service ecosystemecosystem
Service architects and developers build servicesService architects and developers build servicesUsing WS toolkits like WCF and AxisUsing WS toolkits like WCF and Axis
Enterprise architects influence QoS at the SOAP level...Enterprise architects influence QoS at the SOAP level...Using the WS-* specsUsing the WS-* specs
...and at the transport level...and at the transport levelExisting investments can form the underlay for SOAExisting investments can form the underlay for SOA
And undertake necessary governance rolesAnd undertake necessary governance roles
ESB or SOA?ESB or SOA?
Investing in proprietary integration systems now Investing in proprietary integration systems now is investing in future legacyis investing in future legacy
ESB is not the solutionESB is not the solutionIt’s oh-so 1990’s integration glueIt’s oh-so 1990’s integration glue
SOA is the solutionSOA is the solutionBecause it focuses on supporting business processesBecause it focuses on supporting business processes
Web Services are a robust and commoditised Web Services are a robust and commoditised platform for SOA deliveryplatform for SOA delivery
ConclusionsConclusions
SOA is the right integration architecture going forwardSOA is the right integration architecture going forwardSOA can be implemented incrementallySOA can be implemented incrementally
Drive SOA from a business perspectiveDrive SOA from a business perspectiveMost valuable processes/applications/services firstMost valuable processes/applications/services first
Commoditisation across the boardCommoditisation across the boardServers, developers, networking, re-use existing software, etcServers, developers, networking, re-use existing software, etc
Migrating towards a successful SOA is not always easyMigrating towards a successful SOA is not always easyLearning to build dependable SOAs can be difficultLearning to build dependable SOAs can be difficult
ESBs and Wizards cannot help – you need service-savvy geeks ESBs and Wizards cannot help – you need service-savvy geeks and process-aware business peopleand process-aware business people