Using Oracle Data Integrator

8/11/2019 Using Oracle Data Integrator

1/54

Using Oracle Data Integrator with

Essbase, Planning and the Rest of

the Oracle EPM Products

Edward Roske

[email protected]

BLOG: LookSmarter.blogspot.com

WEBSITE: www.interrel.com

TWITTER: ERoske
mailto:[email protected]://www.interrel.com/http://www.interrel.com/mailto:[email protected]


2/54

2


3/54


4/54

About interRel

2008 & 2009 Oracle Titan Award winner - EPM Solution of the

year

2008 Oracle EPM Excellence Award

2009 Oracle EPM/BI Innovation Award

One of the fastest growing companies in the world

(Inc. Magazine, 08 & 09)

Two of the three Hyperion Oracle ACE Directors in the world

Founding Hyperion Platinum Partner; now Oracle Certified Partner

Focused exclusively on Oracle Hyperion EPM software

Consulting

Training

Infrastructure and Installation

Support

Software sales

4


5/54

5 Hyperion Books Available:

Essbase (7): Complete Guide

Essbase System 9: Complete Guide

Essbase System 9: End User Guide

Smart View 11: End User Guide

Essbase 11: Adm in Guide

eBooks avai lab le on Amazon Kind le

Just out!

Hyperion Planning for End Users

Coming Soon Hyperion Planning for Admins (days now)

Hyperion Financial Management (Q1 2010)

To order, check out www.lulu.com

Copyright 2007, Hyperion. All rights reserved.5
http://www.lulu.com/http://www.lulu.com/


6/54

Disclaimer

These slides represent the work and opinions of the

presenter and do not constitute official positions of Oracle

or any other organization.

This material has not been peer reviewed and is presented

here with the permission of the presenter. This material should not be reproduced without the written

permission of interRel Consulting.


7/54

Agenda

What is ODI?

A Real World Example ODI and Planning

ODI Demo

A glimpse into the concepts

Tips

Getting Started

Questions and Answers


8/54

What is ODI?

Oracle Data Integrator


9/54

Oracle Data Integrator (ODI) Overview

Integration tool that replaces HAL and DIM

Used to move and transform information across systems

Processes are based on metadata of existing systems

Only metadata is stored in ODI, not data

Sometimes referred to as Sunopsis


10/54

How is ODI used with EPM applications?

Interface metadata and data between source data systems

Build dimensions

Load data

Transfer data across files and relational databases and the

EPM applications All supported with Hyperion Planning, Financial

Management and Essbase

Not supported with EPMA


11/54

Oracle ODI to EPM Functionality

Planning Essbase HFM

Metadatadiscovery &

model creation

P P P

Load dataP P P

Load MetadataP P P

Extract dataO P P

Extractmetadata O P POther Refresh to

Essbase

Calc, post and

pre MaxL

scripts

Consolidate


12/54

Marketing Message

Three main points:

Extract-Load & Transform (E-LT) architecture

Removes ETL server from infrastructure

USP = Native target database performs

transformations Declarative Design

Developer define the rules, ODI:

Writes the data flows

Manages the data flow complexity

Writes the behind-the-scenes code

Knowledge Modules

Similar to HAL adapters

Talks (and almost always writes) to almost everything


13/54

MaxL Run Alloc at ions,

Agg regate Data, etc

A Real World Integration with ODI

DW

Oracle

GL

SAP

Planning

HFM

Essbase

Fcst Data

Consol idated

Results

Sales by

Customer

DB

Accoun ts /

Enti t ies

1

Accoun ts /

Enti t ies

2

Accoun ts /

Enti t ies

3

Custom er Dimension

Sales by Cus tomer Data

4

5

6

7

8


14/54

ODI Demo


15/54

A Glimpse

ODI Components and Concepts


16/54

ODI Components

Master

Repository

(Infrastructure)

Work

Repository

(Interfaces)

Topology Manager

Configure Architecture /

Infrastructure of source

& target technologies

Designer

Design & execute metadata

models and interfaces to move

data across systems

Operator

Monitor Sessions

Review errors

Agent

Coordinates

communication &

scheduling

Security Manager

Maintain ODI Security

Metadata Navigator

Web access to

mettadata repository

Source Systems Target Systems

ODI(ELT) ExtractLoad-Transform

ODI Components


17/54

Components

GUI Purpose Users

Repository Configuration Info Metadata

OLTP RDBMS

Sunopsis Infrastructure

AdministratorsDevelopers

Operators

Designer Reverse Engineer

Develop Projects Release Scenarios

Developers

Metadata Administrators

Operator Operate Production

Monitor Sessions

Production Operators

Topology

Manager

Physical and Logical

Architecture of theinfrastructure (servers,

schemas and agents)

Infrastructure Administrators

Security

Manager

User privileges Security Administrators


18/54

\

\

Deployment

Work

Repository

(s)(Interfaces)

TopologySecurity

VersioningModels

Projects

Execution

Typically have 1 Master Repository that talks to multipleWork Repository (Dev, Test & Prod).

Can set the production work repository to execution only

Master

Repository

(Infrastructure)


19/54

Topology Manager

Representation of the Information System

Connect to resources for integration purposes

Contains:

Physical Architecture

Data Server Physical Schemas and Physical Agents

Logical Architecture (logical schemas & logical agents)

Contexts Links logical and physical architectures

Repositories Master and attached work repositories Languages - different types of languages available


20/54

A Sample Topology

DW Essbase Source

FilesPlanning

Windows Machine X

(Development Environment)

PlanSamp.

Consol

FinOp.Plan

1

Windows Machine Y

(Production Environment)

Windows Machine Z

(Production Environment)

DW PlanningEssbase Source

FilesRep.App1

Rep.App2

Rep.App1

Essbase

Rep.App2

Essbase

Rep.App1

Rep.App2

PlanSamp.

Consol

FinOp.Plan1

C:\ODI\FileC:\ODI\File

dbo.databa

sedbo.databa

se

PlanSamp.

Consol

Planning

FinOp.Plan

1

Planning

C:\ODI\File

Filedbo.database

MSSQL

Physical Agents

Logical Agents

Logical Architecture

Physical Architecture

Logical Schema

Physical SchemaData Server


21/54

Languages

Languages used to generate code for the specific

technologies


22/54

Designer

Designer includes:

Models Metadata for your source and target systems

Projects - Integrations that move data from your source

and target systems

Interfaces

Packages

Procedures

Variables

Knowledge Modules

The Designer module stores this information in a work

repository, references the topology and the security

information defined in the master repository.


23/54

Designing an Interface

Before you start building an interface you:

Import Knowledge Modules into the project

Create your Metadata Models

Then in the interface you:

Define your source and target

Define the mapping and translations (business rules)

Define the flow requirements

Execute

To automate execution, move on to:

Packages

Procedures

Variables

Sequencing


24/54

Knowledge Modules

Knowledge Modules (KMs) are components with built in

knowledge (functions and logic) that connect technologies

to ODI.

Similar to HAL Adapters, but many more connection

options.

KMs connect to the technology, extract data from it,

transform the data, check it, integrate it, etc.

You can find a complete list of KMs at

http://www.oracle.com/technology/products/oracle-data-

integrator/10.1.3/htdocs/documentation/oracledi_km_reference.pdf


25/54

Knowledge Modules

Interfaces

LKM (Loading Knowledge Modules) are used to extractdata from the source database tables and other systems

(files, middleware, mainframe, etc.).

IKM (Integration Knowledge Modules) are used to

integrate (load) data to the target system.

CKM (Check Knowledge Modules) are used to check thatconstraints on the sources and targets are not violated.

.


26/54

Knowledge Modules

Models

RKM (Reverse Knowledge Modules) are used to perform acustomized reverse-engineering of data models for a

specific technology.

JKM (Journalizing Knowledge Modules) are used to create

a journal of data modifications (insert, update and delete)

of the source databases to keep track of the changes.


27/54

Knowledge Modules

Others

SKM (Service Knowledge Modules) are used to generatethe code required for creating dataservices.


28/54

An Example: Hyperion Planning Model

Dimensions

Planning

Metadata


29/54

An Example: Hyperion Essbase Model

Dimensions

Essbase

Metadata


30/54

Interfaces

An ODI object which loads one target data store with data

from one or more sources, based on business rules

implemented as mappings


31/54

Inside the Interface: Mappings


32/54

More on Mappings

Expression

Editor

ExecuteOn

UpdateTarget

Column


33/54

Multiple Sources, Joins and Filters

Joins (similar to Joins in RDMS)

Links multiple data sources together

Uses SQL Clauses

Multiple sources in a single interface must be connected

with a join. Create joins by dragging and dropping columns together

Alternatively, you can use the expression editor to add

them manually

Advanced joins

Connect models from different technologies or model /

schemas. (Heterogeneous Join)

Connect 2 or more datastores


34/54

Join Definition

Specify

Join type

Use ISO-92 Syntax,Join Order, Auto-

calculate Order

SQL

Syntax


35/54

Filter

Reduce amount of data

processed by the interface

by applying a filter.

Funnel icon represents

filter applied on the source

Use the Expression Editor

or manually type in the

SQL.

Test filter, right-click and

select Data.


36/54

Expression Editor


37/54

Flow Tab


38/54

SUNOPSIS_MEMORY_ENGINE

SUNOPSIS_MEMORY_ENGINE sits between source and

target

Temporary staging area that is created during interfaces to

facilitate mapping and translation

Not recommended for large data sets for performancereasons

Can be difficult to debug

We recommend configuring a separate staging area


39/54

Operator: Checking Status


40/54

Automation

Packages - Group of steps to complete a workflow.

Can include:

Interfaces

OS Commands

Procedures

Workflow can include error trapping and email status

notifications

Scenarios can schedule as a scenario or call a scenario

from a batch file

Variables - can have variables defined to pass parameters

(i.e. file name, current month, etc.)


41/54

Create Error Log Files in the Interface

Enable logging inthe interface in the

files to obtain more

details or send

information to a

text file for use in

debugging.


42/54

Errors

Errors are not always ODI errors, they are typically being

sent from other technologies (Oracle EPM in our case)

You will begin to see commonality of errors but they are

not always descriptive or useful.

Plan extra time for debugging interfaces


43/54

Real Life Tips


44/54

TipsGeneral Information

Good sources of information:

interRel 3 day ODI and EPM training class (available

this fall)

OTN Forum:

http://forums.oracle.com/forums/forum.jspa?forumID=3

74

http://john-goodwin.blogspot.com/

Get the 10.1.3.5 release

No patching required for Essbase, Planning, and HFM

Available at:

http://www.oracle.com/technology/software/products/odi/index.html
http://forums.oracle.com/forums/forum.jspa?forumID=374http://forums.oracle.com/forums/forum.jspa?forumID=374http://john-goodwin.blogspot.com/http://john-goodwin.blogspot.com/http://john-goodwin.blogspot.com/http://john-goodwin.blogspot.com/http://forums.oracle.com/forums/forum.jspa?forumID=374http://forums.oracle.com/forums/forum.jspa?forumID=374


45/54

Design and Development Tips

Whiteboard the Topology before configuring in ODI

Standardize naming conventions for topology components

Incorporate as many error logs and error trapping

processes as possible.

Consider using a staging area versus Sunopsis Memory

Engine. This will allow performance optimization and error

resolution. If the source is a relational database, perform as many

translations in a view as possible.


46/54

Design and Development Tips

Recommend against using comma as a file delimiter for

source files. Use | or !. Versus loading Essbase Data with ODI, write a MaxL to

load data to Essbase and add it as a step in the ODI

package.

Plan extra development time, learning curve is steep and it

is more complex than HAL.


47/54

Project Tasks: Getting Started

Plan the scope, identify source and target

Establish success criteria

Identify the resources

Define the Topology

Define your Models Create Interfaces

Unit Testing

Automate with Packages

Schedule with Scenarios

Integration Testing

Deploy


48/54

How long will take?

No short answer, depends on the complexity of the

interface. If this is the first time, allow ample time to debug

issues.

Minimize risk by:

Working with infrastructure team for connectivity issues

Keeping data transformations simple or nonexistent in the

initial development (prototype)

Then consider the opportunities to automate translations or

business rules Plan out your topology on a whiteboard first! It is critical to

understand what links to what to avoid silly mistakes.

Use Relational Sources where possible.


49/54

OTHER


50/54

Security Manager

Separate from Shared Services

Manage security for all ODI modules

Create Users & Profiles

Assign access ODI Objects (data servers, projects, etc)

Methods (Edit, Create, Delete)

Servers


51/54

Metadata Navigator

J2EE application that provides web access to the

repositories

View Flow maps, trace the source of all data and drill down

to field level

Possible to trigger and monitor processing jobs from a web

browser using this tool

By default, installed on Jakarta Tomcat Application Server


52/54

Oracle Data Profiling and Quality

Integrator extends data quality features of Oracle Data

Integrator to provide more advanced data governance

capabilities.

Data Cleansing

Duplicates

Parsing

Data Enrichment

User driven rules

Integrated with ODI


53/54

ODI and EPM Training

Coming Soon from interRel Training!

Introduct ion to ODI for EPM Adm inistrators

This class will be useful for administrators or developers thatwill design, develop and / or deploy data integrations that

update Essbase, Planning or HFM with ODI.

Includes several hours of hands on workshops !!!


54/54

Thank You.

Edward [email protected]

BLOG: LookSmarter.blogspot.com

WEBSITE: www.interrel.com
mailto:[email protected]://www.interrel.com/http://www.interrel.com/mailto:[email protected]

Using Oracle Data Integrator

Documents