IKnow Process Visibility Process Management iRule Nordic QualityStage rule sets Petter Enqvist Senior systems consultant Intelligent Communication AS.

iKnowProcess Visibility

Process Management

iRuleNordic QualityStage rule sets

Petter EnqvistSenior systems consultantIntelligent Communication AS

• iKnow30 minutes

–Introduction–Case–Demo / Q&A

• iRule15 minutes

–Introduction

Agenda

• iKnow provides tools for operating DataStage:

– Monitoring job progress and server health

– Evaluating job statistics to improve performance and optimize process flow

– Process flow design and execution

– Recovery and notifications

• iKnow adds visibility into data integration processes on the IBM InfoSphere Information Server platform

• iKnow is easy to use!

Introduction to iKnow

• Mitigate the risk of the “shrinking batch window” and ensure faster access to business critical information

• Optimize hardware resource usage• Provide process visibility to operators and

managers• Reduce skill requirements for operating

DataStage

Introduction - Benefits

• Real-time monitoring provides early indication of problems, reducing the effects of errors

• Reduced process complexity keeps the costs of managing the organisation’s data integration solutions down

• Add value to existing scheduling environments by extending monitoring and statistics capabilities

Introduction – Benefits cont.

iKnow is designed for DataStage customers with complex data integration requirements:

– Mission critical data integration processes

– Large number of jobs

– Complex interdependencies between jobs

– Enterprise wide scheduling needs: non-DataStage jobs

– Need to monitor and optimize their data integration environment

Introduction - Target audience

?

Process Visibility

1. What’s going on on the servers right now?

iKnow Monitoring

?

Process Visibility

2. How have my DataStage jobs performed over time?

iKnow Statistics

?

Process Visibility

3. How do DataStage job performance and hardware resource usage correlate?

iKnow Analytics

Process Management

Ensure optimal batch performance

iKnow Process Manager

Operator Tools

Manage scheduled processes

iKnow Director

Operator Tools

Web access to DataStage

iKnow Director for DataStage

• Platform and requirements– Works with DataStage 8.0.1, 8.1 & 8.5

– Windows 2003 & 2008, Linux, AIX and Solaris supported

– Java 1.5 or higher required

– Apache Tomcat v6.0 or WebSphere v6.1 or higher required

– DB2 9.1 – 9.7, Oracle 10g and 11g supported

• Development resources– http://

publib.boulder.ibm.com/infocenter/iisinfsv/v8r5/topic/com.ibm.swg.im.iis.ds.cliapi.ref.doc/topics/c_dsvjbref_WebSphere_DataStage_Development_Kit_Job_Control_Inter.html

Technology

DataStage Engine

iKnow Server

Server

Database server

iKnow Web App

iKnow Client

iKnow Agent

iKnow Agent

iKnow Service

Provider

iKnow Service

ProviderDataStage

Engine

Architecture

iKnow Agent

iKnow Agent

• iKnow30 minutes

–Introduction–Case–Demo / Q&A

• iRule15 minutes

–Introduction

Agenda

• Main fetaures–Standardisation–Matching

• Data de-duplication• Reference matching

–Survive

• Supporting features– Investigate

QualityStage in a nutshell

• Step 1: Standardisation– Input:

–Output:

QualityStage in a nutshell

Name Address1 Address2 CountryLindkvist, Karl Anders Mannerheimvägen 15 02700 Grankulla FinlandKarl Anders Lindquist Mannerheimv. 15 B 02700 Grankulla FinlanfAnna Lindkvist Mannerheimvägen 15 01300 Helsingfors Finland

FName MName LName StreetName Type StreetComplete HNo HDet Zip City Country

Karl Lindkvist Mannerheim v Mannerheimvägen 15 02700 Grankulla Finland

Karl Anders Lindquist Mannerheim v Mannerheimvägen 15 B 02700 Grankulla Finland

Anna Lindkvist Mannerheim v Mannerheimvägen 15 01300 Helsingfors Finland

FName MName LName StreetName Type StreetComplete Country Set Weight Match

Karl Lindkvist Mannerheim v Mannerheimvägen Finland 1 25,96 MP

Karl Anders Lindquist Mannerheim v Mannerheimvägen Finland 1 16,19 DA

Anna Lindkvist Mannerheim v Mannerheimvägen Finland RA

• Step 2: Matching– Input:

–Output:

QualityStage in a nutshell

FName MName LName StreetName Type StreetComplete HNo HDet Zip City Country

Karl Lindkvist Mannerheim v Mannerheimvägen 15 02700 Grankulla Finland

Karl Anders Lindquist Mannerheim v Mannerheimvägen 15 B 02700 Grankulla Finland

Anna Lindkvist Mannerheim v Mannerheimvägen 15 01300 Helsingfors Finland

QualityStage in a nutshell

IBM InfoSphere QualityStage v8.5

Intelcom iRule Nordic+

• Standard QualityStage structure– NORDICPREP– NORDICNAME– NORDICADDR– NORDICAREA

• Countries covered– Denmark– Finland– Norway– Sweden

iRule - Introduction

iRule - Introduction

Steps:1. Preprocessor

2. Investigate

3. Domain standardisation

4. Investigate

Matching

Customisation

iRule - Introduction

• Overrides • Pattern programming

• Rule set development package– Standardization rules template

• Includes templates for domains: address, area, and name.

– Standardization rules development kit• Consists of a series of jobs that produce reports

to assist in custom rules development.

– Standardization quality assessment kit• Consists of a series of jobs that produce reports

to assist in evaluating standardization results.

QualityStage - What’s new in 8.5

QualityStage - What’s new in 8.5

24

• Summary reporting to understand how well standardisation is performing

• Sections

• Summary results for records assessed

• Percentage of records that populate each individual output category to determining anomalies

• Composition of standardized output patterns

Standardisation quality assessment

QualityStage - What’s new in 8.5

• Introduced a match specification report to capture the details of the match specification

• Includes a glossary and all the pertinent details for the specification and passes

• Enables the Business to review Match Rules more effectively

Match specification report

Intelligent Communication ASAkersgata 49

0180 Oslo

Norge

+ 47 22 89 20 90

[email protected]

[email protected]

http://www.intelcom.no

http://www.iknowsw.com/

Thank you!