Bringing Visibility and Control to Net Centric Systems Approaches for Runtime Governance of Net Centric Systems © 2007 AmberPoint, Inc. John Emerson Vice.

Bringing Visibility and Control

to Net Centric Systems

Approaches for Runtime Governance of

Net Centric Systems

© 2007 AmberPoint, Inc.

John EmersonVice President, US Federal

AmberPoint, Inc.

2© 2007 AmberPoint, Inc.

Topics

Net Centric Environments

ESM across the Lifecycle Visibility Synchronizing with Other Governance Systems Policy Management Service Level Monitoring and Management Exception Management Security Validation

Case Studies

Q&A


Wide Variety of Net Centric Infrastructure

EJB Applications

Mission SpecificServices

MQ-basedWarehouseDBMS

Enterprise Service Bus

Materiel Order Service

AgencyPortal

External Agency Service 1



AgencyPortal

EJB Applications MQ-basedWarehouseDBMS




Wide Variety of Vendors

Mission SpecificServices

Materiel Order Service


New Types of Problems

Full Picture? Map and Monitor?

Everything Seems to be Up & Running (Green Lights), but the users are calling the Help Desk claiming that they aren’t getting service

Non-Responses and Corrupted Responses

Debugging Distributed Systems

How do you lock down Security at the Endpoint?

How do you Validate new versions of Services that are Shared across dozens of applications?

Application Issues, not Operations Issues


The Role of Enterprise Service Management

Traditional Management Tools and Techniques

will not solve these new issues. Traditional

Management is an Operations Problem.

ESM (SOA Management) is an Application

Problem

As a Result, the Enterprise Service Management

system will be Monitored by your Operations Staff but

Used by your Development and Tier II/Tier III Staffs

To solve problems that they would otherwise have to write

complex code and dig through logs to fix.


…Throughout the Lifecycle

ESM should automate real-time visibility and control at every stage of the SOA lifecycle

Development Staging Production

IDE’sProcess Tools

Business Logic

Discovery

Runtime Policies• Performance• Availability

Performance

• Automatically enforce governance

• Security• Logging

Diagnostics

Validation

CapacityPlanning

ServiceLevels

Discovery• Automatically

discover rogue services

More Policies• Performance• Availability• SLAs

• Security• Logging• Audit




Enterprise Service ManagementReal-time visibility, control and validation

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <po-number> A234235 </po-number> </soapenv:Body></soapenv:Envelope>

env:Fault>Unknown Servic"urn:ups-shippingService Downserver:8192/e

/soapenv:

Visibility, control and validation regardless of SOA infrastructure decisions



Keys to Successful ESM

Visibility – Knowing What’s Out There and What’s Going On… What services, application flow, and infrastructure are installed and in use Who’s using it Is it meeting expectations / obligations

Control – Taking Action to Prevent or Correct Issues… Define and enforce runtime policies – make sure proper policies are active Diagnose failures / prevent them

Ensuring Integrity – Ensuring Changes Don’t Impact the Whole Application Environment… Automatically check for the correctness of the running system Detect and validate changes before they impact users and partners

Do This across the Entire Infrastructure

And Do It All as Automatically as Possible Reduces risks and costs Automation is the single most important thing that makes SOA scaleable


Automatic End-to-End Visibility

Dynamic discovery of service and supporting components “in the wild” Web services Consumers Dependencies Application Flow &

Transactions Runtime Policies Runtime Metadata

Automatically track transactions Non-invasive; no message

modifications Feed impact analysis, error

detection, etc.

In most environments, no single source of information is always right

Ensures a complete view of the SOA application environmentIntended Design Running Reality

PlatformRepositories

ServiceRegistries

Home-grownDatabases




Automatic Synchronizationwith Other Governance Systems

RuntimeGovernance

ServiceNetwork

• Services• Metadata• Policies

policies

data

servicecontract

• Services (discovered,changes)• Scorecard Information• Policies (new, changes)

Read

Write

Publish Changes to endpoints and policies Scorecard metrics Dependencies

Discover discrepancies between intentions (design/dev) and reality (runtime)

RealityDesign

vs.

Support Federated Information Exchange

??

?

RuntimeRepository

Service Levels

Discovery

Exceptions

Policy Mgmt

PlatformRepositories

Home-grownDatabases

ServiceRegistries


Policies that follow the WS-Policy Standard

Provisioned with a “where clause”

Automatically applies policies based on dynamic attributes and message content. All production services All services in Accounting application All services deployed in WebLogic containers

User-defined attributes for services, containers & policies

Assignments are reevaluated as attributes change

Automatic Policy Provisioning via the WS-Policy Standard

where“Blue Force”

SecurityEncryption

allservices

One-at-a-Time Approach

where deployedon WebLogic app servers

Logging

Profile Based Approach

p1 p1 p1 p50

100 svcs x 50 policies

5,000policy points

Load-BalWeighted

Manage system on “autopilot” where policies are automatically assigned as appropriate.

Eliminate production mistakes by reducing manual steps.

S1

S2

S3

Sn

S5

S3S1

S6S4

S2


Service Level ManagementService- and Mission-level Visibility

ServiceView

Alerts

HistoricalReporting

Enforce agreements based on Mission-Specific criteria “Gold” users, Warfighters in Battle, First Responders, etc.

Granular visibility – groups, users, services, operations

Preventative and corrective actions

User Summary and Objectives


Real-time Monitoring of Mission Centric ProcessesProbe Applications for Inconsistencies

Expected DeliveryDisconnect Alert

Check transaction progress and correctness

Does not need to be synchronous or see every step


Exception ManagementStop Digging through Logs!

TechnicalFaultsMission-Specific

Exceptions

Process Flow- Exception context- Response times

Mission-specific visibility using exception content and context Ex: External Agency Services failure Alert when “no order confirmation within 3 minutes after completion”

Visibility in operational issues – services, transactions, operations, messages SOAP faults, database errors, etc.

System Mgmt

MachinesApp ServersNetwork

AugmentsSystem-level

View

Drill into ExceptionContent & Context

Agency Portal

MaterielOrders

WarehouseRed ForceTracking

Shipping PartnerExternal Agency


Firewall

SecurityFirst- and Last-Mile Enforcement

First-Mile Security- Client-side agent- Automatic enforcement of

out-bound security

Last-Mile Security- Plug-ins provide endpoint protection- No ability to circumvent

Extensive Integration- Identity Management Systems- Security Appliances- App Server / ESB / OS Security

<?xml version='1.0'?> <PaymentInfo xmlns='http://example.org/paymentv2'> <Name>John Smith</Name> <EncryptedData

Type='http://www.w3.org/2001/04/xmlenc#Element' xmlns='http://www.w3.org/2001/04/xmlenc#'> <CipherData>

<CipherValue>A23B45C56</CipherValue> </CipherData>

</EncryptedData> </PaymentInfo>

env:Fault>Unknown Servic"urn:ups-shippingService Downserver:8192/e

/soapenv:

<Name><Encrypted

Type='http <CipherDa

<Cipher</Ciphe

Security Policies- Authentication- Authorization- Credential Mapping- Censorship- Crypto

Shipping Partner

Red ForceTracking Service

Logistics andMateriel

Agency Portal

Blue ForceTracking Service

IdentityManagemen

tSystems

ClientApplication

17© 2007 AmberPoint, Inc. 17

Check SOA Services & Application Integrity for…

Code Is the new version backwards compatible?

PolicyIf we institute this new governance policy

(security, routing, etc.), will it break dependent applications?

UsageWhat will happen if our usage doubles?

Triples? Quadruples?

InfrastructureWill an OS security patch break my system?

How will introducing a load-balancer change my capacity?

Type of Changes

FunctionalityDoes the service produce the same responses

as last time?

Quality of Service Acceptable performance and throughput?

Impact of Change


The SOA Validation ProblemSystem Integrity Always at Risk

Service reuse creates dependencies

Impact of any changes ripple throughout the system Real impact of planned changes is hard to predict Impact of unplanned or unannounced changes can be devastating

Yet, it quickly becomes impossible to setup and replicate all dependent systems for testing elsewhere

18

Agency Portal

WarehouseBlue ForceTracking

Red ForceTracking

Shipping Partner

A Change Here May Impact Everyone

Need way to continuously check for integrity – bothin staging and in production

MaterielOrders

19© 2007 AmberPoint, Inc. 19

SOA ValidationSafe environment to validate changes before deployment

Agency Portal

WarehouseBlue ForceTracking

Red ForceTracking

USTRANSCOM

Changes to:- Code- Policy- Usage- Infrastructure

Captures requests/responses from all dependent consumers Message content Context – headers, distribution patterns

Replays looking for differences in response

Capacity planning using real-world message distributions

MaterielOrders

Case Studies


Real-World Examples


Rapidly Becoming the SOA Runtime Governance Standard in the US Federal Government

ESM Product of Choice, DISA Net-Centric Enterprise Services (NCES) Working with DISA for 3+ years to define ESM Services under NCES

Program

NGA GeoScout Program

Intelligence Community – Multiple Installed Projects

US Transportation Command – GTN Modernization Effort

Engaged with Multiple Net-Centric Labs, COIs and NCES Pilot Programs JHU APL, MIT Lincoln Labs ASAP ACTD – Integrated Broadcast Service program C2 SSA COI - C2 Community

Engaged with all major NCES and Related Industry Organizations AFEI and NCOIF SOA Foundation, ISR, Security Working Groups

NIAP Common Criteria Security Certification Only SOA Management Vendor in the Program


Design-timeDesign-timeRegistryRegistry

Unapproved SvcsUnapproved Svcs

ProductionProductionRegistryRegistry

Approved SvcsApproved Svcs

ApprovalProcess

3,000 Developers Different departments / divisions Different priorities and “fire drills” Different locations

NYLondonDesign

Dev

QA

Stage

Prod

Design

Dev

QA

Stage

Prod

Lehman BrothersMaking enterprise-wide SOA governance pain-free

After a year, only 8 registered services Too busy to update the registry No value to the developers


Uses AmberPoint’s automatic discovery of running services and dependencies at each stage of their SOA lifecycle Synchronizes with home-grown Asset DB and Design-time Repository

NYLondonDesign

Dev

QA

Stage

Prod

Design

Dev

QA

Stage

Prod

RuntimeRuntimeRepositoryRepository

Discovered SvcsDiscovered SvcsPerformancePerformance

Errors/ExceptionsErrors/Exceptions

AssetAssetDatabaseDatabase

MachinesMachinesOwnersOwners

LocationsLocations

Design-timeDesign-timeRegistryRegistry

Unapproved SvcsUnapproved Svcs

ProductionProductionRegistryRegistry

Approved SvcsApproved Svcs

ApprovalApprovalProcessProcess

Using Automatic Runtime Governance toAchieve Design-time Governance


Service Detail Screen

When service was discovered

How long service has been up

Type of service

Link to WSDL

Metadata from Asset DB (42 fields)

All data can be used in policy definitions

Discovered Info

Asset DB Info


NYLondonDesign

Dev

QA

Stage

Prod

Design

Dev

QA

Stage

Prod

Opt-in for expanded control Security, load balancing, failover, etc.

Dependencies

Performance

Diagnostics

ChangeAnalysis

“What’s in it for me?” A lot.Comprehensive insight without lifting a finger


Results: Visibility and Cooperation

From only 8 registered services after previous approach to 100 registered services in first couple months

ROI reporting visible throughout the company

Runtime results automatically feed other consoles JMX-based home grown system Internal SOA coordination site HP OpenView

Transformed the environment to one where groups were vying to be the ones that could “cooperate the most”


Some Other AmberPoint Customers

Manufacturing/Retail

Health CareFin. Services

InternationalGovernment

Power/Utility

Telecom

Other

Q&A


John EmersonVice President, US Federal

AmberPoint, [email protected]

Bringing Visibility and Control to Net Centric Systems Approaches for Runtime Governance of Net Centric Systems © 2007 AmberPoint, Inc. John Emerson Vice.

Documents

service nonresponses

rogue services

production services

esm soa management

soa scaleable slide

new versions of services

application issues

us federal amberpoint