BP3[1]

8/8/2019 BP3[1]

1/42

Helping organizations leverage their investment in Information Builders technologyHelping organizations leverage their investment in Information Builders technology

www.resilientbiz.comwww.resilientbiz.com

Todays AgendaTodays Agenda

Theory of Fives

A Bit About Methodology

Star & Snowflake Schemas

Populating a Star Schema Slowly Changing Dimensions (the real

reason to warehouse data)

ResiliEnt Overview Presenter(s):Roger Panfil, President,

ResiliEnt Business Solutions

8/8/2019 BP3[1]

2/42


Presentation Information

Author: Roger Panfil

Company: ResiliEnt Business Solutions

Presentation Title: Data Warehousing Best Practices

Presentation Abstract: Do you know the difference between a

star schema and snowflake design? What are type 1, 2, and

3 dimensions and when should they be important in providing

the business intelligence you need? Do you know the telltale

signs for pre-aggregating/summarizing data in the data mart

for efficient reporting? These questions and more will beanswered with tips forhandling each in iWay's DataMigrator

product. A customer success story will accompany the

presentation.

8/8/2019 BP3[1]

3/42


Before We Get Started . . .

1) How many of you are building your first DW?

2) Are you going to use an ETL tool?

3) Are you going it alone, or are you partnering with

aconsulting firm?

4) What is the expected time frame to deliver the project?

5) What is the expected time frame to deliver the first

report?

8/8/2019 BP3[1]

4/42


Better Service at a Better PriceBetter Service at a Better Price

The Theory of FivesThe Theory of Fives

8/8/2019 BP3[1]

5/42


The Theory of Fives

1) Pick a business line VP with an information problem . . .

2) Gatherhis/herFive top key performance indicators

3) DeliverFive answers:

Requires Five things: Web interface, Security, Supporting tables

and ETL

process, Content (Reports, Graph

s, Etc). Five reports

Five drills

Five more drills

4) Solicit VP #2

Repeat Step #3

5) Repeat steps 4-5 until you have satisfied Five VPs.

You have just delivered and enterprise solution across Five

business lines (subject areas) withBIG value.

8/8/2019 BP3[1]

6/42


The Theory of Fives

1) It adheres to a time-box methodology

2) Its a repeatable model for success

3) It delivers real value to the organization

4) Its easily expanded5) It works!

8/8/2019 BP3[1]

7/42

Helping organizations leverage their investment in Information Builders technologyHelping organizations leverage their investment in Information Builders technologyBetter Service at a Better PriceBetter Service at a Better Price

A Bit About MethodologyA Bit About Methodology

8/8/2019 BP3[1]

8/42


Methodology

Ours Looks Like This

Project Requirements.

JAD Session..

Project Management

Measurable Milestones...

Client Satisfaction ...

Findings Document

Initial Proposal

Project Charter

Functional Specifications

Gap AnalysisStatement Of Work

Technical Requirements

Investment Analysis

Project Kickoff Meeting

Detailed Project Plan

Technical SpecificationsWeekly Status Reporting

Frequent, Tangible Results

An End Result You Are Happy With!

Unparalleled Client Care

We Work With You:We Work With You: We Deliver:We Deliver:

8/8/2019 BP3[1]

9/42


Apply Proven App Dev Concepts to

Data Warehousing

Define the problem

Define the project and team

Define the solution

Define t

h

e acceptance criteria Define the time-line

Define the deliverables

Deliver

Get Feedback

Revise

Achieve signoff

Promote Success

JAD: JointJAD: Joint

Application DesignApplication Design

TimeTime--BoxingBoxing

RAD: Rapid ApplicationRAD: Rapid Application

DevelopmentDevelopment

Client SatisfactionClient Satisfaction

8/8/2019 BP3[1]

10/42


Resilient Business Intelligence

A Matterof Topology, Technology and

Methodology

Where the circlesWhere the circles

converge marks theconverge marks the

sweet spot for resilientsweet spot for resilient

business intelligence . . .business intelligence . . .

TopologyTopology

Tech

nologyTech

nology

MethodologyMethodology

Cover all of yourCover all of your

platform,platform,

connectivity andconnectivity and

information assetinformation asset

needs . . . Past,needs . . . Past,

present & future.present & future.

Ensure you haveEnsure you have

all of the rightall of the right

tools across astools across as

few vendors asfew vendors as

possible.possible.

Spend the money onSpend the money on

services, training, andservices, training, and

mentoring to ensure yourmentoring to ensure your

technology expendituretechnology expenditure

is safe!is safe!

8/8/2019 BP3[1]

11/42


The StarSchemaThe StarSchema

8/8/2019 BP3[1]

12/42


Some Questions Well Answer . . .

1) What are some of biggest reasons to report from

staged data rather than live data?

2) What is a Star Schema?

3) Whats the difference between a Star Schema and a

snowflake schema?

4) What are Slowly Changing Dimensions?

8/8/2019 BP3[1]

13/42


The StarSchema

The star schema is a favorite

tool of warehouse designers forproviding high performance

query structures in a relational

environment.

8/8/2019 BP3[1]

14/42


The StarSchema

Pre-defined paths based on the way users

want to see th

e data th

at is important to th

em. Pre-defined aggregates and calculations turn

raw data into information.

Data Keys replaced withhigh performance

Surrogate Keys

The Star Schema is query fastbecause . . .

8/8/2019 BP3[1]

15/42


The StarSchema

DimensionTables:

Product

Customer

Time

Facts Table:

Sales

Fact Measures:

Units Sold

Unit Price

Unit Cost

Example taken from Oracle 8i manual Chapter 7

8/8/2019 BP3[1]

16/42


The StarSchema

Dimension Tables are short, but wide (low # rows,

high # fields)

Fact Table is long, but narrow (high # rows, low #

fields)

Theory of Fives: Five or less dimensions per Star

keeps it manageable.

ETL tool is absolutely required to assist with thebuild and maintenance of these structures.

8/8/2019 BP3[1]

17/42


The StarSchema

Fact table only

contains

generated

surrogate keys

Dates and flag for

tracking of slowly

changing dimensions

Time largely controls

granularity . . . And

granularity largely

controls the size of the

Fact data store

Dimension tablescontain both

original data and

surrogate keys

Why?Why?

8/8/2019 BP3[1]

18/42


The Snowflake Schema =

StarSchema + Normalization

. . .

Shorten the width of the

Dimensions thru

normalization

8/8/2019 BP3[1]

19/42


The Snowflake Schema

While this design can save some storage space . . .

More complex to build and maintain

Negative effect on response time

8/8/2019 BP3[1]

20/42


Loading a StarSchemaLoading a StarSchema

with DataMigratorwith DataMigrator

8/8/2019 BP3[1]

21/42

8/8/2019 BP3[1]

22/42


The Star SchemaData Flow at Work Create Dimension Tables

Simple source

transform to create

surrogate key

8/8/2019 BP3[1]

23/42


The Star SchemaData Flow at Work Create Dimension Tables

Simple targettransforms to populate

fields fortracking

slowly changing

dimensions (SCD)

8/8/2019 BP3[1]

24/42


The Star SchemaData Flow at Work Create Staging Table

Source tables are joined toextract the fact data, then the

measures are aggregated by

dimension, the Time dimension

typically controlling granularity

8/8/2019 BP3[1]

25/42


The Star SchemaData Flow at Work Create Staging Table

Another transform,

this time in the column

object, to derive LineCost of Goods

(LINECOGS)

Dimension tables and the staging

table are created in the same RDBMS

foroptimal join performance . . .

8/8/2019 BP3[1]

26/42


The Star SchemaData Flow at Work Create Fact Table

Join all dimension

tables to the staging

table by data-key

fields, but selecting

the surrogate key

fields, to create the

fact table

8/8/2019 BP3[1]

27/42


The Star Schema

Process Flow at Work

Concurrent execution of

jobs to load dimension

tables and staging table

CRE8ECNTFACTS waits

until all jobs in group

are finished

eMail DW

Admin that ETL

process iscomplete

8/8/2019 BP3[1]

28/42


The Star Schema

70k rows, 5 tables, 57 seconds!

8/8/2019 BP3[1]

29/42


Slowly ChangingSlowly Changing

DimensionsDimensions

8/8/2019 BP3[1]

30/42


Slowly Changing Dimensions

Source: www.1keydata.com/datawarehousing

8/8/2019 BP3[1]

31/42



Type 1

Type 1Type 1

slowly changingslowly changing

dimension should bedimension should be

applied to changesapplied to changes

in source data fieldsin source data fields

when it is notwhen it is notnecessary for thenecessary for the

data warehouse todata warehouse to

keep track ofkeep track of

historical changes.historical changes.

No need forNo need foractive recordactive record

flag and dateflag and date--

added stamp.added stamp.


8/8/2019 BP3[1]

32/42



Type 2

Type 2Type 2


dimension shoulddimension should

be used for whenbe used for when

it is necessary forit is necessary for

the datathe datawarehouse towarehouse to

track historicaltrack historical

changes in sourcechanges in source

data fields.data fields.

Active recordActive record

flag and dateflag and date--

added stampadded stamp

are critical.are critical.


8/8/2019 BP3[1]

33/42



Type 3

Type 3Type 3


dimension can bedimension can be

used for when it isused for when it is

necessary for thenecessary for the

data warehousedata warehouse

to track a limitedto track a limitedand finite numberand finite number

ofhistoricalofhistorical

changes in sourcechanges in source

data fields.data fields.

Requires prior andRequires prior and

current fields andcurrent fields and

effective date ofeffective date of

change fields.change fields.


8/8/2019 BP3[1]

34/42



Quiz

A top salesman for Information Builders gets

married and changes his name . . .

Type 1 orType 2 . . . And Why?

Dave Kemler decides to make Pittsburgh part of

the Mid-West region, and does this on June 1 . . .


Jerry Duci changes the product code forWebFOCUS on the IBI Fee Schedule . . .


8/8/2019 BP3[1]

35/42


Wrap UpWrap Up

8/8/2019 BP3[1]

36/42


Who We Are . . .

Founded September 2004.

Co-Founders: Roger Panfil (formerly with IBI/iWay Software) & LailaSamawi-Utley (formerly with SunTrust Bank), Penny Lester (formerlywith IBI).

12 full time staff, with the key positions being:

SVP Delivery

DW/ETL Specialist

BI Specialist

Technical Recruiter

Over 350 screened consultants in our DB promotes on demandstaffing.

State of the art back office system (WebPAS) for managing a consultingpractice.

Administaff runs our payroll & benefits keeping our internal headcountlight.

8/8/2019 BP3[1]

37/42


ResiliEnts Six Drivers of

CustomerSatisfaction . . .

8/8/2019 BP3[1]

38/42


What We Do . . .

Practice Expertise:

Business Intelligence

Data Warehousing

Healthcare and Financial Services verticals

We can help with staffing or full project outsourcing for manypopular technologies:

DW: IBI DataMigrator, DataStage, Informatica, MS DTS

BI: IBI WebFOCUS, BO/Crystal, Cognos, DB: IBI FOCUS, Oracle, DB2, MS SQL Server

Technologies: IBI Maintain, Java, .Net

8/8/2019 BP3[1]

39/42


Full Service Consulting . . .

Information System DeliveryInformation System Delivery Development of New Business Intelligence

Applications

Operational/Production Report Writing

Legacy Report Conversion to New

Technologies

Development of Data Warehouses, Data

Marts and Operational Data Stores Enterprise Application Integration

System/Application/Product Configuration

Review and Tuning

IT Project AssistanceIT Project Assistance

Full Project Outsourcing Project Management

Staff Supplementation

Staff Mentoring

Training and Education

Business PlanningBusiness Planning Vendor and Asset Management

Organizational/Infrastructure Design

Tactical and Strategic Solutions Design

Global IT Infrastructure Operations

Assessment

Business Impact Analysis

Cost Reduction Assessment ROI & TCO (Total Cost of Ownership)

Analysis, Quantification and Optimization

We excel at evaluating business needs, then leveraging existing

resources and combining them with new technologies to createefficient, effective and resilient solutions. We ensure success by

providing the resources and knowledge that will help your team

effectively execute your company's strategic plan.

Laila Utley, CEO, ResiliEnt Business Solutions

8/8/2019 BP3[1]

40/42


Creative Solutions . . .

We excel at expense free consulting . . . We run ourcompany as a virtual organization, thats how good we areat it!

Student labor program with 17 universities that make up the

North Carolina University System Great for conversionwork and contained projects.

Deep list of consultants that will take on part-timeassignments at reduced rates.

Offshore capacity established for large contained projectswhere the rate must be brought down (e.g. conversionwork).

8/8/2019 BP3[1]

41/42


Testimonials

ResiliEnt Business Solutions can be counted on to deliver. They have quickly become

a trusted partner to my IT staff. I consider them an essential part of my team.

Suresh Ramakrishnan, Vice President of Information Technology, XLHealth

ResiliEnt came into a project that was deemed un-doable by the experts . . . un-

doable because of time and technology. ResiliEnt parachuted 3 developers into the

account and on time and on budget what others said couldn't be done. Actually, the

project team was done 3 days ahead of schedule!

Craig Kozlow, Project Manager, Lead Developerand Independent Consultant

ResiliEnt Business Solutions has proven to be worthy of our confidence and highregard.

Ken Heller, President, Thrift Credit Union

8/8/2019 BP3[1]

42/42


www.resilientbiz.comwww.resilientbiz.com

770770--625625--50905090

BP3[1]

Documents