Gaining Control of Your SOA Willie Kirkpatrick VP EMEA AmberPoint.

Gaining Control of Your SOA

Willie KirkpatrickVP EMEAAmberPoint

Long Time Microsoft SOA Partner

Joint development, licensing, and distributionWide support of Microsoft’s SOA stack

Visual Studio.NET variationsBizTalkVisual StudioSystem Center Ops MgrNative C# implementation of AmberPoint

Version of AmberPoint bundled with Visual Studio

Unive

Business ProblemThe Netherlands changed to an “open enrollment” modelCompetitive market required direct customer access to information and enrollmentTargeted Health Insurance System first

Technical ApproachExposing internal processes & applications for internet use by customersMicrosoft “Stack”

Visual Studio.NETSharePointSystems Center Ops Manager

Transformed Mainframe ApplicationsRedundant sites for High Availability

Insurance Company – Netherlands

HTML ServicesHTML Services

Business ServicesBusiness Services

Legacy ServicesLegacy Services

HTML ServicesHTML Services

Business ServicesBusiness Services

Legacy ServicesLegacy Services

177Endpoints

Site 1 Site 2

74Services

Unive

Technical ProblemComplex, distributed transactions (“SOA”) made it hard to manage running applicationsDifficult to:

Diagnose issuesEnsure high availabilityMeet QOS requirements

Solution: AmberPointBusiness Results

With on-line, immediate cross checks and data validation, over 60% of new applications processed directly into mainframe apps

Results“If we hadn’t started using AmberPoint, we would have stopped using SOA.” – Bob Alberts, Project Director

Insurance Company – Netherlands

Vital Forsikring

Business ProblemIncreasingly competitive market driving need to reduce costs and increase agility.

Technical ApproachMigration from Mainframe to SOA overtimeMicrosoft “Stack”

.NET 2.0BizTalk Server 2006Systems Center Ops ManagerWindows 2003 64bit

BenefitsLower mean time to repairAbility to provide reliable, secure self-service application, including a pensions portal for smaller organizations

Largest Life & Pensions company in Norway

Business ServicesBusiness Services Business ServicesBusiness Services

60+Endpoints

30+Services

BizTalkBizTalk

FilesPortalPortal

BizTalkBizTalk

Keys to Successful Runtime Governance of SOA Applications

Visibility – Knowing What’s Out There and What’s Going On…

Control – Putting Policies into Action…

Ensuring Integrity – Ensuring Changes Don’t Impact the Whole Application Environment…

Handle the entire infrastructure

Do it all automaticallyReduces risks and costsAutomation is the single most important thing that makes SOA scaleable

SOA Governance

Design Time Gov.

Dev & QA ToolsLifecycle ManagementApproval Processes

Service RegistryPolicy Requirements

Design Time Gov.

Dev & QA ToolsLifecycle ManagementApproval Processes

Service RegistryPolicy Requirements

Runtime Gov.

Service Level MgmtTransaction Monitoring

Auditing / LoggingSecurity

Policy Enforcement

Runtime Gov.

Service Level MgmtTransaction Monitoring

Auditing / LoggingSecurity

Policy Enforcement

SOA Infrastructure

App ServersEnterprise Service Bus

AppliancesLegacy Systems

Process ManagementDatabases

SOA Infrastructure

App ServersEnterprise Service Bus

AppliancesLegacy Systems

Process ManagementDatabases

Closed Loop SOA Governance

Design Time Gov.

Dev & QA ToolsLifecycle ManagementApproval Processes

Service RegistryPolicy Requirements

Design Time Gov.

Dev & QA ToolsLifecycle ManagementApproval Processes

Service RegistryPolicy Requirements

Runtime Gov.

Service Level MgmtTransaction Monitoring

Auditing / LoggingSecurity

Policy Enforcement

Runtime Gov.

Service Level MgmtTransaction Monitoring

Auditing / LoggingSecurity

Policy Enforcement

SOA Infrastructure

App ServersEnterprise Service Bus

AppliancesLegacy Systems

Process ManagementDatabases

SOA Infrastructure

App ServersEnterprise Service Bus

AppliancesLegacy Systems

Process ManagementDatabases

RunningReality

??

??

IntendedDesign

Closed Loop SOA Governance

Design Time Gov.

Visual StudioTeam System

Repository

Design Time Gov.

Visual StudioTeam System

Repository

Runtime Gov.Runtime Gov.

SOA Infrastructure

.NET / Windows Communication FoundationBizTalk

SharePointSQL Server

Etc.

SOA Infrastructure

.NET / Windows Communication FoundationBizTalk

SharePointSQL Server

Etc.

RunningReality

??

??

IntendedDesign

Closed Loop SOA Governance

Design Time Gov.Design Time Gov. Runtime Gov.Runtime Gov.

SOA InfrastructureSOA Infrastructure

RunningReality

??

??

IntendedDesign

Messaging

Automatic End-to-End Discovery

Dynamic Discovery of your SOA environment…Dependencies

Services & Consumers

Transaction Flow

Runtime Policies & Metadata

…across Heterogeneous InfrastructureContainers

ESBs

Appliances

Registries / Repositories

No application, message or header modifications

Automatically feeds Design Time Governance

Ensures Complete Accounting of Your SOA Environment

Design InformationRunning Environment

Repositories

ServiceRegistries

Home-grownDatabases

AmberPoint SOA ExplorerReal-time view of overall environment and status

Quick filters to rapidly isolate areas of interest

Transaction flow

Recent additions

Problem areas

Specific application groups

“Rogue” services

Take actionPlace under management

Apply missing policies

Drill down into detail

Sortable, printable, exportable information

Filters

Drill Down

Graphical View

Table View

End-to-End Transaction Monitoring & DiagnosisFrom Clients, through Infrastructure, Applications, and Endpoints

BusinessTransactions

Process Flow- Exception context- Response times

Drill into TransactionContent & Context

eCommerce System Order

WarehouseCreditCheck

Shipping PartnerOrder Management

Follows transactions through SOA and non-SOA components:

ESB‘sJava and .NET ApplicationsDatabases

Detects problems with business flows:

Performance issuesMissing or stalled stepsFailures

Performance

and

Real-time Monitoring of Business TransactionsProbe applications for inconsistencies

Expected DeliveryDisconnect Alert

Check transaction progress and correctnessDoes not need to be synchronous or see every step

Service Level ManagementSLA enforcement for transactions, groups, users, and services

Transaction-levelSLA’s

ServiceLevelViolations

User-levelSLA’s

HistoricalReporting

Enforces agreements based on business criteria“Gold” users, Accounting systems at the end of quarter, etc.

Flexible calendars, scheduled downtimes, fixed and sliding time windowsPreventative and corrective actions

MultipleObjectivesperAgreement

Take Preventative / Mitigating ActionA Throttling Example

Service Level Objective (SLO)

For Platinum customers: • Ave. Response time per hour < 6 sec• Warning threshold <= 4 sec - Action: Throttle non-Platinum users

Service Level Objective (SLO)

For Platinum customers: • Ave. Response time per hour < 6 sec• Warning threshold <= 4 sec - Action: Throttle non-Platinum users

11

Usage segmented – e.g. by Platinum, Gold, Silver

22

33Service Level Agreement

Platin

um

Gold

Bronze

Performance against objectives

55 Stabilized response times

Take Preventative / Mitigating ActionA Throttling Example

Service Level Objective (SLO)

For Platinum customers: • Ave. Response time per hour < 6 sec• Warning threshold <= 4 sec - Action: Throttle non-Platinum users

Service Level Objective (SLO)

For Platinum customers: • Ave. Response time per hour < 6 sec• Warning threshold <= 4 sec - Action: Throttle non-Platinum users

11

Usage segmented – e.g. by Platinum, Gold, Silver

22

33Service Level Agreement

Platin

um

Gold

Bronze

Performance against objectives

Automatically triggers throttling before

compliance failure44

Throttling Policy

Closed Loop SOA Governance

Design Time Gov.Design Time Gov. Runtime Gov.Runtime Gov.RunningReality

??

??

IntendedDesign

Approved ServicesIntended reuseQOS requirementsPolicy requirements

Discovered Services & DependenciesTransaction MonitoringPerformance metricsPolicy enforcement

SecurityThrottlingVersion transparency

UsersAuthorizedUnauthorized attempts

Policy additionsDiscovered policiesOperational additions

Policy-based Approach to Runtime GovernanceEnforced across the infrastructure

Pre-built library of most commonly used runtime policies

User-extensible

Instrumentation Version management Service level agreements Exception handling Content-based Policies Authentication – certificates,

credentials, SAML, etc Authorization Censorship Credential Mapping Crypto – Signatures &

Encryption

ThrottlingQuality of Service

Performance Availability Throughput

FailoverLoad balancingValidation

Automated Policy Provisioning

Select policy

Configure

Set policy criteria

Policy enforcement across the infrastructure

ServiceMetadata

Microsoft BizTalk

MEP

MEP

MEP

VSP

Orchestration Protocol Mediation Message Routing Content Transformation

Resource Management• Discovery / Registration• End-to-end Visibility• Measurement • Message Distribution/Utilization• Availability• Access Control

Combing AmberPoint + BizTalk creates a SOA Grid with a clean separation of responsibilities

Controls Service-to-Service Interactions

Controls Service-to-Endpoint Interactions

MEP

MEP

MEP

VSP

MEP

MEP

MEP

VSP

ManagedEndpoints

VirtualServiceProvider

MEP

MEP

MEP

VSP

MEP

MEP

MEP

VSP

MEP

MEPMEP

VSP

Secure Service Provider

Balanced Service Provider Resilient Service Provider

Evolving Service Provider

Qualities: • Security• Integrity• Confidentiality

Benefits:• Tamper Resistant• Private• Controlled Access

Qualities: • Reliability• Availability

Benefits:• Highly Available• Fault Tolerant

Qualities: • Version Transparency• Flexibility

Benefits:• Agile• Controlled Access• Deprecation

Qualities: • Scalability• Performance• Capacity

Benefits:• Balanced• Distributed• Manageable

Runtime Governance of Virtual Service Providers

MEP

MEP

MEP

VSP

LB

MEP

MEP

MEP

VSP

LB FO

MEP MEPMEP

VSP

VT

v1 v2 vn

AmberPoint SOA Runtime Governance

Mark MunroSenior Sales EngineeringNorthern Europe - AmberPoint

In an ESB ScenarioService-Oriented Infrastructure

Service Registry

Service Management

SecurityLegend:Legend:CIM Consumer Integration ModuleCIM Consumer Integration ModuleSIM Service Integration ModuleSIM Service Integration Module

Enterprise Service Bus

ESB Core Engine

Transformation

Routing

Exception Management

Orchestration

Ad

ap

tati

on

B2B Gateway Provisioning Framework

Ad

ap

tati

onSupported ServiceSupported Service

ConsumerConsumer

Native

Supported ServiceSupported ServiceProviderProvider

Native

Standard ServiceStandard ServiceConsumerConsumer

SOAPCIM

Standard ServiceStandard ServiceProviderProvider

SOAP SIM

MedicAlert

Windows 2003

Consumer Client(GUPTA on USB Key)

ChillKat(String Encryption)

AmberPoint Agent(.NET)

LDAP

IIS

Send orCustom Actions

Windows 2003

AmberPoint Agent(.NET)

IIS

Web Service(.NET)

Windows 2003

AmberPoint Agent(.NET)

IIS

Web Service(.NET)

BizTalk

MS NLB (VIP)XWallFirewall

Public Internet

Desktop Applications(future)

MS NLB (VIP) MS NLB (VIP)

IP*Works!(SOAP)

Health Care Services

Personal health records for 4,000,000 world-wide members.

Key Requirements:• Ensure high QoS requirements for access to

MedicAlert services – critical health implications• Ensure ‘last-mile’ security for sensitive & private

patient information

Chosen Solution:AmberPoint

Visibility into service usage and performance bottlenecksMonitor impact of security & management policies on their SOAVersion transparency – different eHealthKey versions

Microsoft .NET and BizTalkOrchestration & Mediation

eHealthKey

Results:• “Can usually have a running prototype assembled

faster than the marketing guys can write up their requirements.” – Jorge Mercado, CTO

Closed Loop SOA Governance

Design Time Gov.Design Time Gov. Runtime Gov.Runtime Gov. RunningReality

??

??

IntendedDesign

Approved ServicesIntended reuseQOS requirementsPolicy requirements

Discovered Services & DependenciesTransaction MonitoringPerformance metricsPolicy enforcement

SecurityThrottlingVersion transparency

UsersAuthorizedUnauthorized attempts

Policy additionsDiscovered policiesOperational additions