I4E MDM Architecture Sadagopan Krishnamachari IBM

© 2008 IBM Corporation

MDM Architecture and Implementation Best Practices

Sadagopan Krishnamachari

2

Agenda

Fragmented Data Problems …

Master Data & Master Data Management

IBM Infosphere Master Data Management Server

MDM Server Implementation Pattern

MDM Architecture

MDM Data Stewardship Application

MDM Search / Matching Algorithm

3

Islands of InformationReduce customer satisfaction, decrease revenue, hinder relationships

DocumentManagement

Michael Johnson

Mortgage.tif ERP

JP Morgan, USA

Cust ID : JP003

CRMCall Center

Mike Johnson

JP Morgan Chase

Last Interaction: 4/11/03 (product not

received)

Retail Pointof Sale

JP Morgan & Chase

Contact : Michael A Johnson, CIO

270 West St, NY

Portal

Michael Johnson

User ID: Mjohnso

! Personalized access

! Gold Customer

! Sub: Newsletter 1

CRMMarketing

Michael Johnson

! Opt-Out flag

! No Promotion flag

Application Forms

PaymentsService and

Support

Online Registration

In-StoreInteractions

DirectMail Response

DataWarehouse

Michael P Johnson1400 54rd Avenue

NY NY212 995-3345

3rd PartyInformation

Internet Commerce

Michael Johnson

User id :mjohnson

JP Morgan

Contract:: JP987

Online Purchases

7

4

Coffee Beans GTIN 20012294219421

CANCode : 21204

USACode : 21192

BR, CR, MEXCode : 21186

CH, AUT, DE, UK, FR, BEL, NL, IT : Code : 21184

DE, FIN, SWE, NOR, ESP, POR, Code : 21190

BUL, YUG, CR, RO, SLOVCode : 19616

ISRCode : 21204

MidEaCode : 21204

World TradeCode : 19619, 19616

AUSCode : 21190

HK, TAI, SIN, MAL, S.KORCode : 21188

JAP, THAI, INDO, PHICode : 21189

CZ, LIT, EST, SLOV, RUCode : 2002494

Inconsistent Master Information is a Major HurdleImpacts Revenue, Cost, Agility and Compliance

ARGCode : 21184

Gaining control over product information results: Errors in data – 30% of data in retailers systems is wrong Lost productiv ity – 25 minutes manual cleansing per SKU, per year Slow time to market – 4 weeks to introduce new products Inv oice deductions – 43% of invoices result in deductions Failed scans – up to 70,000 per week (1 large US Retailer) Lost sales – up to 3.5% per year

Source: A.T. Kearney, GMA, AMR

Industry Driv ers: RFID, Waste Electrical and Electronic Equipment Recycling, Product Information Exchange Standards, Return of Hazardous Substances, Global Data Synchronization, Sarbanes Oxley, etc. (Yankee Group, 2005)

6

5

Master Data

Facts describing the core business entities: customers, suppliers, partners, products, materials, bill of materials, chart of accounts, location and employees•The high value information an organization uses repeatedly across many business processes•Generally used across multiple LOB

Master Data Management

Master Data Management (MDM) is the set of disciplines, technologies and solutions to create and maintain consistent, complete, contextual and accurate business data for all stakeholders across and beyond the enterprise

6

MDM Repository / Hub is analogous to a Version Control System, albeit for your Master Data

7

MDM Server Implementation Environment

MDM Domains

Business Services

Pre-buil t Customi zable

Data Stewardship

Party Account Product Custom

Integratio n

Content Data Analy tics

InfoSphere MDM ServerInfoSphere MDM Server

MDM Domains

Business Services

Pre-buil t Customi zable

Data Stewardship

Party Account Product Custom

Integratio n

Content Data Analy tics

InfoSphere MDM ServerInfoSphere MDM Server

RDBMS

Security,Mail, etc

Real-time/Near-Real-Time Connectivity Services (ESB, EAI, Web Services, MQ, etc.)

Enterprise Data Integration (InfoServer)

Billing/Provisioning

Call CenterW eb Phone Sales Vendor & Other Business Partners

Management

Service

Master Data Batch Load

External Data Providers (e.g. D&B, ACXIOM,

Experian)

EnterpriseData Warehouse/ Data Mart

Corporate & Others

Content Management

Process Server

Understand Cleanse Transform Deliv er

Order

Customer Product Account Others

Customer Product Account Others

Customer Product Account Others

Customer Product Account Others

New Systems (e.g. SOA –based)

(e.g. Siebel) (e.g COGNOS)Real-time

IBM InfoSphere MDM Server

8

Understand Cleanse Transform DeliverUnified Deployment

ETL Tooling

Unified Metadata Management

The Complete Picture – Multi Form MDM Server

Multiform MDM manages data

domains critical to business processes

Multiform MDM leverages merged,

cleansed and standardized

data via the ETL

IBM Master Data Management

Banking Insurance Government Healthcare Retail TelcoFocused on critical

information intensive business problems

Indu

stry

Mod

els

& A

sset

s

Multiform Master Data Management

Collaborate Operationalize Analyze

Party (Customer, citizen, prospect, organization, supplier, distributor, etc.)

Product (good, service, product bundle, catalogue, product component, etc.)

Account (Agreement, financial account, reward program, etc.)

9

A dminWeb A pp

DataStewardship

Web A pp

WebServicesA dapter

BatchP rocessor

EventManager

ESB / MQ / EA I Broker

Dashboard/Portal U Is

C lientA pplications

MDM Consumers

Service Controller P arser

C onstructor

Request Framework

Bus iness Transaction Manager

Bus iness Proxies

MDM Core

UtilityComponents

Bus iness Logic C omponents

ExtensionC ontroller

Java Classes

RuleSets

RulesEngine

Pre/PostTxn

Pre/PostAction

Extension Framework

Common Components

Standardization

Audit Data

Metadata

ErrorMessages

Data-LevelEntitlements

MetaData

Event Manager

P erformanceT racker

ErrorMessaging

Logging

T ransaction AuditInformation Log

ExternalV alidation

ExternalBus iness Rules

Rules ofV isibility

C onfigurationManager

RulesEngine

Logs

ARMAgent

ConfigurationSettings

ValidationRules

XMLC ompositeT ransaction

Handler

Java Classes

RuleSets

MessagingA dapter

MDM Product Component Common Component 3rd Party Product Client ComponentMDM 8.0Jan, 2008 Significant Pluggable Component

NotificationRequest Handler

C ontroller Components

BusinessServices

AdminServices

PartyServices

ContractServices

HistoryServices

FastTrack

Security

Events

Validators

EvergreenP rocessor

Suspect ProcessingComponents

Notifications

Adapters

Web Services Asynchronous Synchronous Client-Defined Interface

Data

W C C C o r e

OperationalTables

History Data

HistoryTables

Rule Data

CodeTablesData

ExtensionTables

Behavior Extensions

Data Extensions

JMS Topic

ExtensionToolkit

10

MDM Workbench – A Model based Data Model

11

Data Stewardship Application – Duplicate Suspect Processing

12

Step 1: Optimizes data for statistical comparisons– Normalizes & compacts data, creates derived data layer

source data remains intact– Phonetic equivalences, tokenization, nicknames, etc.

Step 2: Finds all the potential matches– Casts a wide net – all matches on current or historical attributes,

prevents misses– Partial matches, reversals, anonymous values, etc.

Step 3: Scores accurately via probabilistic statistics– Compares attributes one-by-one and produces a weighted score

(likelihood ratio) for each pair of records– Frequency weights specific to your business– Edit distance, proximity of match

Step 4: Custom threshold settings– Single or dual threshold models– Link, don’t link, don’t know – “learns”

from manual input

Manual reviewManual reviewLowestpossiblescore

Lowestpossiblescore

Highestpossiblescore

Highestpossiblescore

Don’tlinkDon’tlink LinkLink

Lowestthreshold

Upperthreshold

Should be linked

Should not be linked

Name122222223334

Data Derived Hash Buckets

Robert RBT 121213444

Potter PTR 34839020

ZIP345666665435

SSN342133333555

7 9

8.63

Matching and Searching Algorithm

13

14

Extras

15

Useful Links IBM Information On Demand website

(http://www-306.ibm.com/software/data/information-on-demand)

IBM Information Management website (http://www.ibm.com/software/data)

Extreme Leverage website

(http://w3-103.ibm.com/software/xl/portal/viewcontent?type=doc&srcID=XT&docID=B329727F31168I32) Customer Success Stories

(http://www-306.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=db2software) Information Management Demos

(http://demos.dfw.ibm.com/solutions/infomgmt/) Information as a Service Demo

(http://media.dvdpowertools.com/ibm/infomanagement/interface.php) Videos

(http://www-306.ibm.com/software/info/television/index.jsp)