The SOA Journey - Deploying and Managing SOA, a HP IT Case Study Tutorial Anjali Anagol-Subbarao Chief Architect, IDM, Marketing and Direct IT, HP .

The SOA Journey - Deploying and The SOA Journey - Deploying and Managing SOA, a HP IT Case Managing SOA, a HP IT Case Study TutorialStudy Tutorial

Anjali Anagol-SubbaraoAnjali Anagol-Subbarao

Chief Architect, IDM, Marketing and Direct Chief Architect, IDM, Marketing and Direct IT, HPIT, HP

www.oasis-open.org

Polling Question #1

What is your familiarity with SOA

and Web Services

A. Investigation phase

B. Process of implementing a pilot

C. Developed a Web service

D. Developed a cross enterprise solution

Overview of SOA SOA Web services

SOA Case Studies Consumer Business Identity Management

Best Practices

Pressures on the business…

Continuous businesstransformation

Evolv

ing B

usi

ness

Obje

ctiv

es

Changin

g M

ark

ets

New Demands

Growth, profit, and value

Leadership

Customersatisfaction

Innovation

Technology

Regulation/Deregulation

Mergers &acquisitions

Economy

CompetitionSatisfying Unpredictable Needs

CustomerSupplier Partner

Business agility

… result in challenges for the CIO

Support rapid change Security Performance

Improve availability

Consumption-based costing Capacity

Deliverservices

P&L contribution

Increase business relevanceMobility

Distributed systems

Outsourcing

Emerging applications

Heterogeneity

Reduce complexity

Drivecostsdown

Improvequality

ofservice

Goals of SOA

Business and IT Alignment Software design derived from an

intrinsic

understanding of business design IT systems that enable business

agility

DefinitionIn April 2006 The Object Management Group's (OMG ) SOA Special Interest Groupadopted the following definition for SOA:

Service Oriented Architecture is an architectural style for a community of providers andconsumers of services to achieve mutual value, that:

● Allows participants in the communities to work together with minimal co-dependence or technology dependence

● Specifies the contracts to which organizations, people and technologies must adhere in order to

participate in the community● Provides for business value and business processes to be realized by the community● Allows for a variety of technologies to be used to facilitate interactions within the

community

In March 2006 the OASIS group SOA Reference Model released its firstpublic review draft. This defines the basic principles of SOA that apply at all levels ofa service architecture, from business vision through to technical and infrastructureimplementation.

Service-Oriented Architecture: A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce desired effects consistent with measurable preconditions and expectations.

Principles of SOA

● services share a formal contract

● services are loosely coupled

● services abstract underlying logic

● services are composable

● services are reusable

● services are autonomous

● services are stateless

● services are discoverable

Source: Thomas Erl; SearchWebService.com

SOA shifts the way we think

Traditional Applications Service Oriented Architecture

Designed to last Designed to change

Tightly Coupled Loosely Coupled, Agile and Adaptive

Integrate Silos Compose Services

Detailed Abstracted

Long development cycle Interactive and iterative development

Cost, supply centered Business, demand centered

Middleware makes it work Architecture makes it work

Favors Homogeneous Technology Favors Heterogeneous Technology

Implementing Enterprise SOA:A Multi-faceted Approach

SOA Maturity Model

Why an Enterprise SOA Strategy is Important

Create structure around federated SOA efforts – avoid IT mavericks Provide guidance and recommendations to Business and IT teams wanting to

implement SOA solutions Manage and govern the architectural landscape – planning, preparing, and applying

principles, techniques, and technologies to make the business adapt to change. Manage semantic interoperability through Services Reduces integration expenses

Web based SOA reduces integration expense through standardization Increases Asset Reuse

Helps eliminate duplicate functionality Reduces time to market Promotes consistency

Reduces risk More control over business processes by business people

Improves Business Agility Allows the business direct control of business processes to manage rapid

change

Consequences of not having an Enterprise SOA Strategy

Within 2-3 years, we’ll have… Mishmashed implementations of non-cohesive SOAs Islands of architectures – fragmented business functionality &

Business Processes Vendor-defined SOA landscapes (every vendor wants to be the

‘center of the universe’) IT will spend a lot of time in the future unwinding shortsighted

solutions Semantic mess – multiple applications exposing seemingly similar

functionality Lots of non-reusable, un-structured services that don’t enable

business processes Businesses struggle to react to change – reduced competitiveness

SOA Technology and Web Services

One of the key reasons for the today’s focus upon SOA is the emergence of supporting technologies.

SOA is an architectural approach, centered around the concept of services

SOA ≠ Web Services SOA can exist without Web Services Web Services can be utilized without an SOA Using web services can significantly enhance our

ability to implement SOA

A common source of confusion

Web Services Standards World Wide Web

Consortium (W3C)http://www.w3c.org

Organization for the Advancement of Structured Information Standards (OASIS)http://www.oasis-open.org

WS-Interoperability (WS-I) http://www.ws-i.org

Web Services make implementing SOA easier, but they aren’t the same

ERPERPLegacy

AppLegacy

AppCustom

AppCustom

App

Web Services Web Services Web Services

Security SecuritySecurity

Messaging MessagingMessaging

TransactionsTransactions

ERPERPLegacy

AppLegacy

AppCustom

AppCustom

App

Web Services Web ServicesWeb Services

SOA Fabric (Abstraction Layer)SOA Fabric (Abstraction Layer)

Transactions Messaging Security

Business Services Data Services Discovery

Man

agem

ent

Mon

itorin

g

Standard architecture with Web Services

SOA leveraging Web Services

Web Services, the preferred technology for SOA A web service exposes a SOAP XML (industry standard) interface and can be invoked by

any client regardless of platform (e.g. J2EE, .Net etc.) Ideally suited for heterogeneous IT environments (such as HP’s) to enable systems to

interact in a standards-compliant, interoperable manner Web services offer the technology and SOA offers the blueprint

SOAP;WSDL

J2EE Standards/

.Net

WS Security

WS-MgmtQuality of Service

Business ComponentArchitecture

Reuse within the Enterprise; process-to-process b2b

SO Maturity

Event Driven

BPA-Aligned

CompositeServices

LooselyCoupled

Structured Programming

Business Process

Execution Language

MetadataRepository

Reuse within Organizations; Browser-based b2b

Coarse Grained

Client/Server & Traditional Languages

Dynamic business partnerships possible

Fine Grained

Strategic Benefit

Reuse across companies; Scaled process-to-process b2b

WSRP

TechnologyMaturity

Click to edit Master title style

SOA Case Studies

HP-IT Reference SOA

Business Resources

Enterprise Information Stores

Component Services (Application / SOI / Data / Utility Services)

Business Services

BusinessService

BusinessService

Business Process Managment(process automation, service orchestration, rules engine)

Service Consumer/ Presentation

BusinessService

Custom&

Legacy

Packaged AppsCRMERP

ApplicationService

ApplicationService

IntegrationService

Identity Management and

Web Services Management

Policy, Meta Data, and QoS

GovernanceGov of Services &

Gov of Usage(Policy, Classification,

Compliance throughout lifecycle)

ManagementService

MonitoringService

SecurityService

Prin

cipl

es

InfrastructureInfrastructure

ServiceVirtualized infrastructure and provisioning

UtilityService

IntegrationService

Transactional

Analytical Operational

Enterprise Semantics

Portal VoiceWeb Rich Client

Desktop Rich Client

Mobile Device Email

I

nteg

ratio

n

DataService

Content Referential

Service Registry & Repository

HP-IT Reference SOA – Standards View

Business Resources

Enterprise Information Stores

In

tegr

atio

n

Component Services

Business Services

Business Process Management

Service Consumer/ Presentation

ESB

Identity Management and

Web Services Management

Policy, Meta Data, and QoS

GovernanceGov of Services &

Gov of Usage(Policy, Classification,

Compliance throughout lifecycle)

Prin

cipl

es

EAI

InfrastructureVirtualized infrastructure

and provisioning

Enterprise SemanticsManagement

WSDM,WS-ManageabilityWS-Provisioning

SecurityWS-Security

WS-SecurityPolicyWS-

SecureConversationWS-Trust

WS-Federation

Portal and Presentation

WSRPJSR168

Transactions & Business ProcessBPEL4WS, WS-Choreography;

ASAP;WS-Transactions, WS-Coordination, WS-CAF;

SOAP

MessagingWS-Eventing,

WS-Notification, WS-Addressing,

WS-ReliableMessaging, WS-Reliability,

SOAP, MTOM

Service Registry & Repository

MetadataUDDI 2.0, UDDI 3.0,

WSILWS-Policy, WS-PolicyAssertions,

WSDL

E-Business IT – Significant Progress with SOA Evolving to an SOA has been the core of

Architecture Strategy

Progress to date Decouple systems and eliminate the re-integration

problem Enforce greater consistency in processes and re-use Lower cost to serve

Benefits Greater IT agility leading to better business agility Greater Leverage of investment dollars

E-Business IT’s SOA Evolution

Enterprise Repositori

es

Web Site A(e.g., SMB

Store)

Function A1

Function C1

Function B1

Function D1

Function E1

Function G1

Function F1

Function H1

Web Site B(e.g.,

Enterprise)

Web Site C(e.g., Public

Sector)

Web Site D(e.g.,

Consumer)

From “monolithic” solutions…

CRM Master Data

Content FinancialERP

Function A2

Function C2

Function B2

Function D2

Function E2

Function G2

Function F2

Function H2

Function A3

Function C3

Function B3

Function D3

Function E3

Function G3

Function F3

Function H3

Function A4

Function C4

Function B4

Function D4

Function E4

Function G4

Function F4

Function H4

Service A

Service E

Service DService CService B

Service GService F

Web Services exposing standard processe

sService H

Web Site A(e.g., SMB Store)

Web Site B (e.g., Consumer

eSupport)Site C

(e.g., Retail Kiosk)Site D

(e.g., Enterprise Procurement System)

Sites

… to “thin” service consumers that leverage web services for std processes

Enterprise Repositorie

sCRM Master

DataContent FinancialERP

E-Business IT’s SOA Evolution (2)

Click to edit Master title style

Consumer Business Case Study

IT couldn’t keep up with business demands

Externalinterface

Coresystem

ERP (SAP)

Configurator, Catalog DB, Vendor data entry tools

• Not real-time

• Custom developed “pipe” for each business partner was expensive to maintain

• Long lead times to connect new retailers

• Could not support major e-tailers

3rd party systems

RetailerSystems

hp website

RetailOutlet

Why SOA? Service–oriented to offer a menu of services for retailers

to pick and choose from

Leverage the expertise of HP and retail partners

Interoperability with disparate systems of retailers

Standard platform to expose functions from disparate HP systems

Abstracting the interface from the implementation

Reuse of services

SOA Implementation Using Web Services

Web services

Web services clientWeb services client

Retailer systems

Datarepositories

Coresystem

ERP (SAP)

Configurator, Product catalog database

Request/Response technology (Application server)

Request/Response technology (Application server)

Web serviceslayer

HP systems

Distribute

product catalog

Request

price

Request

basket transfe

r

Query

order Statu

s

Place

order

Queryproduc

tinfo

Validateconf

ig

Overview of SOA Solution 4 Web services in production

12 external partners

1st implementation – March 2002

HP’s systems – SAP, Microsoft, J2EE, Oracle

Retailer systems – .Net, VB, J2EE – WebLogic, Web Methods

1. Not all partners ready with XML; EDI has to be part of solution

2. Achieving desired performance is a challenge

3. Development time delayed due to evolving standards and technologies

4. Security and interoperability can be achieved

Lessons Learned

Relative Unit Volume

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan

Account 1 Account 2

Results Achieved – Business Agility1. Increased sales (see chart)

2. Faster order to delivery time (24 hours)

3. 50% decrease in man-months to implement new accounts

4. Savings from closing down systems and moving to an SOA platform

5. New revenue streams generated by offering services like ValidateConfig

Note: circles indicate months accounts transitioned to new infrastructure / program

Case Study: Identity Management

Overview of Customer IDM

subsequent site layers

HP.com

publishing systems

back-end systems

site infrastructure

awareness

buy

use & learn

support

consu

mer

ente

rprise

/corp

ora

te

public se

ctor

small/m

ediu

m b

us.

common services

Cus

tom

er E

xper

ienc

e St

rate

gy, IA

, D

esig

n

Customer IDM provides a mission critical horizontal process and shared service for hp.com web sites

Industry Leading Implementation

One of the largest IDM systems in the industry 35 MM users, growth rate of 700,000/month

One of the highest Available systems in HP SLA of 3 9’s , protects sites which do business of

the order of 4 billion dollars/year

• Many ways to do registration which increased cost of implementation

• Non-standard protocols for authentication

• Tight coupling between client and server

• Only web access management

• Access through different web sites which caused security issues

Challenges for Customer IDM system

Custom pipes to provide IDM functionality

EXTERNAL FIREWALL

DMZ

INTERNAL FIREWALL

HP Passport Components

DATABASE

Registration

Web site

REGISTRATION SERVER

App Server Cluster

Web Services

Web services

Site

APISite

Plugin-auth Plugin-auth Plugin-auth

Policy Server

End-User Web Browser

How did we resolve the challenges To address the HP identity and access management

challenges HP-IT is implementing identity services through an SOA model.

Implementing registration, authentication and federation services

The identity services were hosted centrally and all external facing web sites could consume these common services

Loosely coupled Interoperable across many OS/app/web servers Uses standard protocols Open to services, devices

SOA-based ArchitectureEnd- User (Web Browser )

EXTERNAL FIREWALL

DMZ

INTERNAL FIREWALL

HP Passport Components

DATABASE

Registration

Web Services -1

REGISTRATION

SERVER

App Server Cluster

Web Services

Authentication/

Services -2

Device Rich Client Web ServiceEnterprise Customers

FederationServices

Policy

Server

Identity Services Defined – Burton slide

Underlying Identity Components

Consumers of Identity Operations

Authentication&

Authorization

Query &

Update

Personalization&

VisualizationSecurityFederation

Federated domains

Federated domains

Identity and policy administration

Identity and policy administration

ApplicationsApplicationsApplicationsApplicationsApplicationsApplications

Services

Identity Services Defined – HP’s Identity Services

Underlying Identity Components

Consumers of Identity Operations

Authentication&

Authorization

Query &

Update

Personalization&

VisualizationSecurityFederation

Federated domains

Federated domains

Identity and policy administration

Identity and policy administration

ApplicationsApplicationsApplicationsApplicationsApplicationsApplications

Services

LoginValidate

EditProfileUpdateCredentials

getUserFederationWeb services

Password Management

Benefits Enabling new business opportunities

Cross selling, up selling between SMB and enterprise storefronts Enabling extended enterprise

Identity services help bring these partners/outsourcers to have a more seamless access to HP

Extended functionality beyond web access management Achieved a Cost Reduction of 50%

Leverage Idm to reduce business costs through identity services Used standard protocols and loose coupling Support, integration costs reduced

Risk Mitigation Security Breaches avoided as one registration, authentication service

used throughout company Federation helped in maintaining regulatory compliance

Click to edit Master title style

Best Practices/Lessons Learned

Best Practices Established for SOA

1. Designing for interoperability

2. Publishing enduring Web services contracts

3. Effectively using business tier systems

4. Planning a robust production environment

5. Building with Frameworks

Challenges – Web Services Interoperability The great promise of web services

Service producers and consumers can use any OS / prog. language Web services standards would guarantee seamless interoperability

Reality – Creating interoperable web services is still hard Evolving specs and ambiguity Vendors implementing standards selectively Teams encounter interoperability issues (often discovered during

later stages of testing) In some cases, caused senior management to form a negative

opinion of web services, and the value of SOA in general Compiled best practices with respect to interoperability

Compliance vs interoperability (exceptions to WS-I standards) Issues with specific vendors tools

First design the interface Use WSDL editors

(XMLSpy) to create WSDL (for the validateConfig service)

Three abstract definitions - types, messages and port type

Two concrete definitions - binding and service

Design considerations for Versioning

Leverage XML Schemas

Patterns to facilitate Versioning

Naming Convention

Deployment Strategy

Details of versioning Using date stamp as part of the target

namespace of your XML Schema. <SOAP-ENV:Body>

<m:inValidateConfigv1_2 xmlns:m="http://production.psg.hp.com/types/2004/02/04">

…..

</SOAP-ENV:Body>

Use different end points in WSDL

Use different operations

Versioning Lifecycle1. Build transition plan2. Make Changes to Service. 3. Test new Service version4. Implement new Service version. 5. Add/publish new Service version to WSDL descriptions, UDDI

registries, etc. 6. Notify known Consumers of new Service version and transition

plan7. Run Service versions in Parallel8. Set Date for Retirement of older Service version9. Notify known Consumers of retirement10. Remove old Service version from descriptions, registries etc. to

stop new consumers discovering and using.11. Remove functional behavior of old Service. Only return

appropriate error message12. Retire old Service. Physically remove old Service version.

Key Security Elements

Secured the Web services using Transport Level Security – 2 way SSL Creates performance issues

Now Web services can be secured using message level security - WS-Security

Performance and Web services

Performance numbers without SSL

• Performance numbers with SSL -- degradation of approx 30%

Transaction Name

Minimum Average Maximum Std 90 Perce

nt

Pass

AB_request 0.578 2.168 34.75 2.9 3.928 1,449

placeOrder_request

3.688 6.367 29.344 2.931 9.53 193

VC_request 0.719 2.172 24.078 2.252 3.804 10,080

Enhancing the performance Identifying performance bottlenecks using

HP’s OVTA

Enhancing the performance Making XML more efficient

Use sTAX parser XML Beans for XML to Java Binding (now part of Apache open source) XML accelerators from HP

Making SOAP more efficient SOAP parsers

BEA SOAP engine measurements showed 72% faster than Apache Axis SOAP with attachments

Frameworks support SOA

Dealing with complexity Standards do not specify how to deal with the complexities of

designing and implementing modular, reliable, scalable and high performance services

Frameworks “Productize” best practices and provide a foundation to

developers for creating services

Repeatability and consistency

E-Biz SSA framework for designing and implementing services

E-Biz WPA framework for UIs that consume services

What next for SOA and Web Services?

Infrastructure to support SOA ecosystem

for sustaining

Business Agility

Business Process Management

Lif

ecycle

Man

ag

em

en

t

Secu

rity

Man

ag

em

en

t

Dyn

am

ic R

ero

uti

ng

an

d

tran

sfo

rmati

on

s

Web Services

Business Logic

Enterprise Systems

Summary Introduction to SOA and web services Successful implementation of SOA architecture

Configure to Order Case Study Identity Management Case Study

Lifecycle of development of Web services Challenges of implementing Web services –

security and performance Best Practices

Call to action

Check out http://dev2dev.bea.com/index.jsp for BEA WebLogic references

Look at http://openview.hp.com/bea for the OpenView Products

Access DRC portal at http://devresource.hp.com for Web services, SOA, life cycle development tips

Look at http://www.oasis-open.org/home/index.php

Rest of it is in the book

J2EE Web Services on BEA WebLogic by Anjali Anagol-Subbarao

Questions

www.oasis-open.org