8/15/2019 93039928 TDWI DW Concepts and Principles Preview
1/48
Previews of TDWI course books are provided asan opportunity to see the quality of our materialand help you to select the courses that best fityour needs. The previews can not be printed.
TDWI strives to provide course books that arecontent-rich and that serve as useful referencedocuments after a class has ended.
This preview shows selected pages that arerepresentative of the entire course book. Thepages shown are not consecutive. The pagenumbers as they appear in the actual coursematerial are shown at the bottom of each page.
All table-of-contents pages are included toillustrate all of the topics covered by a course.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
2/48
TDWI Data Warehousing Concepts and Principles
© The Data Warehousing Institute i
TDWI Data Warehousing Concepts & Principlesan Introduction to the Field of Data Warehousing
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
3/48
TDWI Data Warehousing Concepts and Principles
ii © The Data Warehousing Institute
The Data Warehousing Institute takes pride in the educational soundness and technicalaccuracy of all of our courses. Please give us your comments – we’d like to hear fromyou. Address your feedback to:
email: [email protected]
Publication Date: May 2004
© Copyright 2002-2004 by The Data Warehousing Institute. All rights reserved. No partof this document may be reproduced in any form, or by any means, without writtenpermission from The Data Warehousing Institute.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
4/48
TDWI Data Warehousing Concepts and Principles
© The Data Warehousing Institute iii
Module 1 Data Warehousing Concepts …………................ 1-1
Module 2 Data Warehousing Architecture ………...…….... 2-1
Module 3 Data Warehouse Implementation ….....……….... 3-1
Module 4 Data Warehouse Operation …………………...…. 4-1
Module 5 Summary and Conclusions …….....…..……….... 5-1
Appendix A Bibl iography and References …………………… A-1
T A
B L E O
F C O N T E N T
S
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
5/48
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
6/48
Data Warehousing Concepts TDWI Data Warehousing Concepts and Principles
1-2 © The Data Warehousing Institute
Data Warehousing BasicsUnderstanding Data, Information, and Knowledge
Outcomeachievement, discovery
Actioninsight, resolve, decision, innovation
Knowledgerecall, experience, instinct, beliefs
Datadescriptive, quantitative, qualitative
Informationfacts, metrics
impact
done by software& databases
done by people
realizes business value
Outcomeachievement, discovery
Actioninsight, resolve, decision, innovation
Knowledgerecall, experience, instinct, beliefs
Datadescriptive, quantitative, qualitative
Informationfacts, metrics
impact
done by software& databases
done by software& databases
done by peopledone by people
realizes business valuerealizes business value
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
7/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Concepts
© The Data Warehousing Institute 1-3
Data Warehousing BasicsUnderstanding Data, Information, and Knowledge
DATA Data is composed of individual and discrete facts that collect descriptive,quantitative, and qualitative values of business interest. Data warehousinginvolves two types of data – operational data which describe the day-to-day events and transactions of the business, and informational data thatare reconciled, integrated, and cleansed to constitute the raw materialfrom which information is constructed.
INFORMATION Information is an organized collection of data presented in a specific andmeaningful context. The purpose of business information is to inform
people and processes – to provide facts and metrics vital to the processesand useful to the people who carry out those processes. Information adds
to the collection of knowledge that is available to business people and business processes.
KNOWLEDGE Knowledge is a personal and individual thing. Here we leave the realm ofwhat computers and software do, and enter the domain of what people do.Knowledge encompasses the familiarity, awareness, understanding, and
perceptions of a person about a given subject. Knowledge is gainedthrough many channels including study, recall, experience, instinct, and
beliefs. These factors are different for each person, thus the knowledge ofevery individual is unique
ACTIONS ANDOUTCOMES
Action is a process of doing something. Effective action is the process ofdoing the right thing. It is described as a process because we need to look
beyond the event of doing and consider the activities and behaviors thatlead to that event. Any combination of insight, resolve, decision, andinnovation may drive a person to act – the “doing” part of action.Outcomes are the results of actions. Favorable business outcomes aregenerally those that reduce cost, save time, optimize resources, increaserevenue, satisfy customers, or otherwise help to fulfill the businessmission and goals.
IMPACT ANDVALUE
Value is realized at the bottom line of the business – when outcomes
reduce cost or increase revenue either directly or indirectly. The value ofan action is determined by the outcomes produced. The value ofinformation is derived through contribution to valued action – providingsupport for insight, resolve, decision, and innovation. The value of thedata warehouse depends entirely on the value of the information servicesthat it delivers.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
8/48
Data Warehousing Concepts TDWI Data Warehousing Concepts and Principles
1-16 © The Data Warehousing Institute
Warehousing Data StoresData Store Responsibilities
Data Marts(cubes, views, web reports, spreadsheets, etc.)
SourceData
source to warehouseETLs
queries & analysis
Business Intelligence Tools
DataStaging
DataWarehouse
Data Marts(star-schema, cubes, views, web reports, spreadsheets, etc.)
intake
integration
distribution
delivery
access Data Marts(cubes, views, web reports, spreadsheets, etc.)
SourceData
source to warehouseETLs
queries & analysis
Business Intelligence Tools
DataStaging
DataWarehouse
Data Marts(star-schema, cubes, views, web reports, spreadsheets, etc.)
Data Marts(cubes, views, web reports, spreadsheets, etc.)
SourceData
source to warehouseETLs
queries & analysis
Business Intelligence Tools
DataStaging
DataWarehouse
SourceData
source to warehouseETLs
queries & analysis
Business Intelligence Tools
DataStaging
DataWarehouse
DataStaging
DataStaging
DataWarehouse
DataWarehouse
Data Marts(star-schema, cubes, views, web reports, spreadsheets, etc.)
intake
integration
distribution
delivery
access
intake
integration
distribution
delivery
access
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
9/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Concepts
© The Data Warehousing Institute 1-17
Warehousing Data StoresData Store Responsibilities
THE ROLES Every data warehousing environment, regardless of architecture and flowof data, must provide for five roles to be complete. Different architecturesassign these roles to data stores in various ways.
INTAKE Data stores with intake responsibility receive data into warehousingenvironment. Data is acquired from multiple source systems, of varyingtechnologies, at different frequencies, and into numerous warehousingfiles and/or tables. Further, the data typically requires many and diversetransformations. Most data is extracted from operational systems whosedata is most certainly not all clean, error-free and complete. Datacleansing is commonly performed as part of the intake process to ensure
completeness and correctness of data.
INTEGRATION Integration describes how the data fits together. The challenge for thewarehousing architect is to design and implement consistent andinterconnected data that provides readily accessible, meaningful businessinformation. Integration occurs at many levels – “the key level, theattribute level, the definition level, the structural level, and so forth …”( Data Warehouse Types , www.billinmon.com) Additional data cleansing
processes, beyond those performed at intake, may be required to achievedesired levels of data integration.
DISTRIBUTIONData stores with distribution responsibility serve as long-term informationassets with broad scope. Distribution is the progression of consistent datafrom such a data store to those data stores designed to address specific
business needs for decision support and analysis.
DELIVERY Data stores with delivery responsibility combine data as “in businesscontext” information structures to present to business units who need it.Delivery is facilitated by a host of technologies and related tools - datamarts, data views, multidimensional cubes, web reports, spreadsheets,queries, etc.
ACCESS Data stores with access responsibility are those that provide businessretrieval of integrated data – typically the targets of a distribution process.Access-optimized data stores are biased toward easy of understanding andnavigation by business users.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
10/48
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
11/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Concepts
© The Data Warehousing Institute 1-19
Warehousing Data StoresThe Data Warehouse
CENTRAL DATAWAREHOUSE (HUB)
As previously discussed, Inmon defines a data warehouse “a subject-oriented, integrated, non-volatile, time-variant, collection of dataorganized to support management needs.” (W. H. Inmon, Database
Newsletter, July/August 1992) The intent of this definition is that the datawarehouse serves as a single-source hub of integrated data upon which alldownstream data stores are dependent. The Inmon data warehouse hasroles of intake, integration, and distribution.
KIMBALL’SDEFINITION (BUS)
Kimball defines the warehouse as “nothing more than the union of all theconstituent data marts.” (Ralph Kimball, et. al, The Data Warehouse LifeCycle Toolkit, Wiley Computer Publishing, 1998) This definition
contradicts the concept of the data warehouse as a single-source hub. TheKimball data warehouse assumes all data store roles -- intake, integration,distribution, access, and delivery
DIFFERENCES INPRACTICE
Given these two predominant definitions of the data warehouse - Inmon’s(hub-and-spoke architecture) and Kimball’s (bus architecture), what arethe implications with regard to the five roles of a data store – intake,integration, distribution, access and delivery?
Inmon Warehouse Kimball Warehouse
intake fills the intake role, but may be
downstream from staging area
Fills the intake role –
downstream from “backroom”transient staging
integration Primary integrated data storewith data at the atomic level
Integration through standardsand conformity of data marts
distribution Designed and optimized fordistribution to data marts
Distribution is insignificantbecause data marts are a subsetof the data warehouse
access May provide limited data accessto some “power” users
Specifically designed forbusiness access and analysis
delivery Not designed or intended fordelivery
Supports delivery of informationto the business
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
12/48
Data Warehousing Concepts TDWI Data Warehousing Concepts and Principles
1-34 © The Data Warehousing Institute
Data Warehousing DeliverablesResults of Architecture, Implementation & Operation Activities
data warehousing program charter data warehousing readiness assessmentdefined business architecturedefined data architecturedefined technology architecturedefined project architecturedefined organizational architecture
Architecture
project planstarget data modelsdata warehousing process modelsdeployed technologywarehousing databasesdata acquisition processesdata transformation processes
data transport & load processespopulated warehousing databasesbusiness analysis applicationsdelivered data warehousing capabili ties
Implementation
business servicesdata refreshmanaged platforms
managed environmentcustomer servicemanaged qualitymanaged infrastructure
Operation
data warehousing program charter data warehousing readiness assessmentdefined business architecturedefined data architecturedefined technology architecturedefined project architecturedefined organizational architecture
Architecture
data warehousing program charter data warehousing readiness assessmentdefined business architecturedefined data architecturedefined technology architecturedefined project architecturedefined organizational architecture
Architecture
project planstarget data modelsdata warehousing process modelsdeployed technologywarehousing databasesdata acquisition processesdata transformation processes
data transport & load processespopulated warehousing databasesbusiness analysis applicationsdelivered data warehousing capabili ties
Implementation
project planstarget data modelsdata warehousing process modelsdeployed technologywarehousing databasesdata acquisition processesdata transformation processes
data transport & load processespopulated warehousing databasesbusiness analysis applicationsdelivered data warehousing capabili ties
Implementation
business servicesdata refreshmanaged platforms
managed environmentcustomer servicemanaged qualitymanaged infrastructure
Operation
business servicesdata refreshmanaged platforms
managed environmentcustomer servicemanaged qualitymanaged infrastructure
Operation
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
13/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Concepts
© The Data Warehousing Institute 1-35
Data Warehousing DeliverablesResults of Architecture, Implementation & Operation Activities
ARCHITECTURERESULTS
Architectural activities establish the standards, conventions, andguidelines that ensure consistency and integration among results ofmultiple implementation projects. Architectural work begins by defininga warehousing program and assessing organizational readiness.Architecture is broad in scope and focused on analysis and design in thefollowing areas:
• Business Architecture – Understanding of business goals, drivers, andinformation needs.
• Data Architecture – Understanding of source data. Requirements andstandards for warehousing data and warehouse metadata.
• Technology Architecture – Identification of standards for hardware,software, and communications technology. Specification of the datawarehousing toolset.
• Project Architecture – Incremental development plan for the datawarehouse. Defined scope of each increment. Sequence anddependencies among increments.
• Organizational Architecture – Identification of training, support, andcommunications responsibilities.
IMPLEMENTATIONRESULTS
Where architecture is broad in scope, implementation narrows the scopeto that of a single increment. Each increment is defined as a project that
focuses on design, construction, and deployment of warehousing productsincluding:
• Warehousing Databases – Data models and implemented databasesfor staging data, data warehouse, and data marts.
• Warehousing Processes – Source –to-target mapping, specification ofdata transformation rules, and development of processes to move datathrough the warehousing environment.
• Business Analysis Applications – Standard queries, decision supportsystems (DSS), warehouse published reports, and other standardmeans of receiving information from the data warehouse.
OPERATIONRESULTS
Operation is the phase where data warehousing delivers value. That valueis realized through business services that provide data and informationand enable confident decisions and positive actions. Training, support,and administration are also key elements of data warehouse operation.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
14/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Architecture
© The Data Warehousing Institute 2-1
Module 2Data Warehousing Architecture
Topic Page
Business Architecture 2-2
Data Architecture 2-10
Technology Architecture 2-46
Project Architecture 2-48
Organizational Architecture 2-58
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
15/48
This page intentionally left blank.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
16/48
Data Warehousing Architecture TDWI Data Warehousing Concepts and Principles
2-6 © The Data Warehousing Institute
Business ArchitectureBusiness Processes
activities
i n p u
t s pr o d u c t customerssources w
o r k f o r c e
events / transactions
b u s i n
e s s
p r o c e s
s
• which processes are in scope of the warehousing program?• who (customer, source, workforce) needs information?• which business process components are information subjects?
• how can inputs be optimized?• how can activities be streamlined?• who can the workforce contribute?• how can suppliers contribute?• how can events be managed?• how can product value be enhanced?
activities
i n p u
t s pr o d u c t customerssources w
o r k f o r c e
events / transactions
b u s i n
e s s
p r o c e s
s
activities
i n p u
t s pr o d u c t customerssources i n
p u t s
i n p u
t s pr o d u c t customerscustomerssourcessources w
o r k f o r c e
events / transactions
b u s i n
e s s
p r o c e s
s
• which processes are in scope of the warehousing program?• who (customer, source, workforce) needs information?• which business process components are information subjects?
• how can inputs be optimized?• how can activities be streamlined?• who can the workforce contribute?• how can suppliers contribute?• how can events be managed?• how can product value be enhanced?
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
17/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Architecture
© The Data Warehousing Institute 2-7
Business ArchitectureBusiness Processes
UNDERSTANDINGBUSINESSPROCESSES
Business processes are the things that a business does to produce its products, deliver its services, manage its infrastructure, etc. Every business process can be understood in terms of the components of that process:
• the product that the process produces,• the customer who uses the product,• the inputs that are needed to produce the product,• the sources/suppliers that provide the inputs,• the activities that comprise the process,• the actors who perform the activities,• the events that drive the activities.
Recognizing which processes will be information-enabled through datawarehousing, and which process components will become subjects ofwarehousing data, offers valuable input to all phases of data warehouse
planning, development, and operation.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
18/48
Data Warehousing Architecture TDWI Data Warehousing Concepts and Principles
2-20 © The Data Warehousing Institute
Data ArchitectureData Modeling Concepts
• Warehousing Subjects• Business Questions• Facts & Qualifiers• Target Configuration
• Staging, Warehouse, & MartER Models
• Data Mart DDMs
• Staging Area Structure• Warehouse Structure• Relational Mart Structures• Dimensional Mart Structures
• Staging Physical Design• Warehouse Physical Design• Data Mart Physical Designs
(relational & dimensional)
• Implemented WarehousingDatabases
• Source Composition• Source Subjects
• Integrated Source DataModel (ERM)
• Source Data Structure Model
• Source Data Files
• Source Data FileDescriptions
• Business Goals & Drivers• Information Needs
Triage
ContextualModels
ConceptualModels
LogicalModels
StructuralModels
PhysicalModels
ImplementedData
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
19/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Architecture
© The Data Warehousing Institute 2-21
Data ArchitectureData Modeling Concepts
FAMILIAR DATAMODELINGPRINCIPLES
Like application data modeling, warehouse modeling works well when practiced at multiple levels of abstraction. Modeling either application orwarehouse data may develop any or all of:
• Contextual Models describing the scope of requirements, establishinga context for analysis.
• Conceptual Models describing requirements without consideration forcomputer implementation.
• Logical Models describing data from a computer system perspective,yet free of any implementation platform specifics.
• Structural Models specifying data structures that account for variablesof access, navigation, security, distribution, and time-variance.
• Physical Models providing detailed design and specification of datastructures to be implemented using a particular technology.
WAREHOUSEMODELINGDIFFERENCES
Even the most experienced application data modelers are challenged byearly warehouse modeling experiences. New issues, terminology, andtechniques combine to make warehouse data modeling more complexthan application data modeling. The primary differences include:
This Facet of WarehouseModeling …
Differs from Application Modelingin This Way …
Multiple Data Types Both source data and warehousing data need to be modeled. Each is modeled separately, andthey are associated through a technique called “triage.”
Multiple Ways to UseWarehouse Data
Warehouse data uses range from publishing and managed query to complex OLAP applicationsand data mining. The ideal data structure depends on planned uses of the data.
Multiple Ways to Organize theData
Warehouse databases may be organized relationally, dimensionally, or with a combination of thetwo techniques. The ideal organization depends on both the planned uses of the data and thecharacteristics of the data.
Multiple Modeling Techniques The complexities of warehouse data modeling require that many modeling techniques be used.Matrix models, E/R models, subject models, dimensional models, star-schema, and snowflake-schema are all used to meet various data modeling needs.
Planned and ManagedRedundancy
Redundancy, typically avoided in application databases, is an asset to warehouse databases.Planning and managing redundancy is a key skill for warehouse data modelers.
Large Data Volumes Redundancy and time-variance combine to make a very large database (VLDB) a commonwarehouse consideration. Optimizing for data volumes and database size is a commonrequirement of warehouse modeling.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
20/48
Data Warehousing Architecture TDWI Data Warehousing Concepts and Principles
2-34 © The Data Warehousing Institute
Data ArchitectureIntegration and Data Flow Standards
DataSources
IntegrationHub
DataMart Data
Mart
DataMart
Hub and Spoke Integration
DataSources
IntegrationHub
DataMart Data
Mart
DataMart
Hub and Spoke Integration
DataSources
Bus Integration
Integration Bus
DataMart Data
Mart
DataMart
DataSources
DataSources
Bus Integration
Integration BusIntegration Bus
DataMart Data
Mart
DataMart
DataMartDataMart Data
MartDataMart
DataMartDataMart
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
21/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Architecture
© The Data Warehousing Institute 2-35
Data ArchitectureIntegration and Data Flow Standards
HUB-AND-SPOKEINTEGRATION
The hub-and-spoke architecture provides a single integrated andconsistent source of data from which data marts are populated. Thewarehouse structure is defined through enterprise modeling (top downmethodology). The ETL processes acquire the data from the sources,transform the data in accordance with established enterprise-wide
business rules, and load the hub data store (central data warehouse or persistent staging area). The strength of this architecture is enforcedintegration of data.
BUS INTEGRATION The Bus Architecture relies on the development of conformed data marts populated directly from the operational sources or through a transient
staging area. Data consistency from source-to-mart and mart-to-mart areachieved through applying conventions and standards (conformed factsand dimensions) as the data marts are populated. The strength of thisarchitecture is consistency without the overhead of the central datawarehouse.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
22/48
Data Warehousing Architecture TDWI Data Warehousing Concepts and Principles
2-50 © The Data Warehousing Institute
Project ArchitectureMethodology
Enterprise Modeling & Architecture
Incremental Development Planning
Data Warehouse Design & Development
Data Mart Design & Development
Operation & Support
Incremental Deployment
Top-Down Development
Data Mart Deployment
Data Mart Design & Development
Operation & Support
Identify Business Area Scope
Bottom-Up Development
IncrementalEnterprise Modeling
Integration StructureDesign & Development
IncrementalDeployment
Data Warehouse / MartDesign & Development
Operation & Support
Incremental DevelopmentPlanning
Identify Business Area Scope
Hybrid Methods
Enterprise Modeling & Architecture
Incremental Development Planning
Data Warehouse Design & Development
Data Mart Design & Development
Operation & Support
Incremental Deployment
Enterprise Modeling & Architecture
Incremental Development Planning
Data Warehouse Design & Development
Data Mart Design & Development
Operation & Support
Incremental Deployment
Top-Down Development
Data Mart Deployment
Data Mart Design & Development
Operation & Support
Identify Business Area Scope
Data Mart Deployment
Data Mart Design & Development
Operation & Support
Identify Business Area Scope
Bottom-Up Development
IncrementalEnterprise Modeling
Integration StructureDesign & Development
IncrementalDeployment
Data Warehouse / MartDesign & Development
Operation & Support
Incremental DevelopmentPlanning
Identify Business Area Scope
IncrementalEnterprise Modeling
Integration StructureDesign & Development
IncrementalDeployment
Data Warehouse / MartDesign & Development
Operation & Support
Incremental DevelopmentPlanning
Identify Business Area Scope
IncrementalEnterprise Modeling
Integration StructureDesign & Development
IncrementalDeployment
Data Warehouse / MartDesign & Development
Operation & Support
Incremental DevelopmentPlanning
Identify Business Area Scope
Hybrid Methods
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
23/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Architecture
© The Data Warehousing Institute 2-51
Project ArchitectureMethodology
TOP-DOWN Top-down approaches are also commonly called enterprise approaches.Top-down data warehouse development begins at the enterprise, andtypically emphasizes the data warehouse as a primary integratedinformation resource. Data warehouse structure is determined throughenterprise modeling. Content is determined by a combination of businessinformation needs and available source data. Top-down approaches aregenerally associated with longer start-up times due to the need forenterprise perspective.
BOTTOM-UP Bottom-up approaches begin with business information needs for a single business unit or limited business domain. Bottom-up methods are most
compatible with bus integration approaches, using conformity instead ofan enterprise repository to achieve integration. Bottom-up developmentgenerally trades strength of an integration hub for the benefits of quickstart-up and rapid deployment.
BALANCINGENTERPRISE &BUSINESS UNITFOCUS
Hybrid approaches combine some elements of bottom-up developmentwith some from top-down methods. The objective of a hybrid approach israpid development within an enterprise context. A typical hybridapproach quickly develops a skeletal enterprise model before beginningiterative development of data marts. The data warehouse is populatedonly as data is needed by data marts, and is sometimes constructed in a
retrofit mode after data marts have been deployed. Metadata consistencyand conformed dimensions are the initial integration tools, with the datawarehouse being a secondary means of integration
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
24/48
Data Warehousing Architecture TDWI Data Warehousing Concepts and Principles
2-64 © The Data Warehousing Institute
Organizational ArchitectureProgram, Project & Operations Roles
sponsorship program management data governance
metadata management architecture specification quality management
b u s i n e s s r u
l e s s p e c
i f i c a t
i o n
b u s i n e s s r e q ui r em en t s d ef i ni t i on
integration design database development ETL development
data mart development BI application development
p r o j e c
t m a n a g e m e n t s
o ur c e d a t a an al y s i s
data integration & cleansingdata access, analysis, & mining
business metrics usagesystem & database administrationprocess execution & monitoring
training & support
BI Program
BI Projects
BI Operations
sponsorship program management data governance
metadata management architecture specification quality management
b u s i n e s s r u
l e s s p e c
i f i c a t
i o n
b u s i n e s s r e q ui r em en t s d ef i ni t i on
integration design database development ETL development
data mart development BI application development
p r o j e c
t m a n a g e m e n t s
o ur c e d a t a an al y s i s
data integration & cleansingdata access, analysis, & mining
business metrics usagesystem & database administrationprocess execution & monitoring
training & support
BI Program
BI Projects
BI Operations
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
25/48
TDWI Data Warehousing Concepts and Principles Data Warehousing Architecture
© The Data Warehousing Institute 2-65
Organizational ArchitectureProgram, Project & Operations Roles
ROLES ANDRESPONSIBILITIES
The program, project, and operation activities of data warehousing aredifferent from those of developing and supporting operational systems.The work is different; therefore the roles and responsibilities are different.Data warehousing has different goals and challenges. It demands differentkinds of organizations and teams. Common data warehousing roles andresponsibilities include:
BI Program Roles & Responsibil ities
Program Management Managing business/IT relationship, multiple dependent projects, issue resolution, etc.
Sponsorship Advocacy, political will, resource acquisition, issue resolution, expectation setting, etc.
Data Governance Data definitions, business rules alignment, data quality management, access authorization, etc.
Business Rules Specification Business basis for data rules about content, relationships, correctness, integrity, etc.
Business Requirements Definition Requirements for data & information, service levels, quality & reliability, etc.
Architecture Specification Frameworks & standards for business alignment, data, technology, projects, etc.
Quality Management Beyond data quality – quality of information, delivery, interface, reporting, services, etc.
Meta Data Management Meta data strategy, meta data implementation, meta data content, etc.
BI Project Roles & Responsibilities
Project Management Work breakdown, scheduling, resource allocation, deliverables, deployment, etc.
Integration Design Data source selection, source/target mapping, transformation rules, populating databases
Database Development Logical and physical database design, database specification and creation
ETL Development Analysis, design, construction, and deployment of data movement processesSource Data Analysis Data profiling, source content analysis, source data modeling
Data Mart Development Analysis, design, construction, and deployment of data marts
BI Application Development Analysis, design, construction, and deployment of information services & analytic applications
BI Operations Roles
Data Integration & Cleansing Maintenance and support of data migration processes; Continuous data quality management
Data Access, Analysis, & Mining Access and application of data to make business decisions
Business Metrics Usage Application of business measures to drive business actions
System & Database Administration Installation, configuration, and management of BI operating platforms
Process Execution & Monitoring Scheduling, execution, verification, and support of data warehousing processesTraining & Support Customer care activities for BI customers
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
26/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Implementation
© Th Data Warehousing Institute 3-1
Module 3Data Warehouse Implementation
Topic Page
Implementation Planning 3-2
Warehouse Data Modeling 3-8
The Warehouse Process Model 3-22Deployed Technology 3-40
Implementation Components 3-44
Delivery Results 3-48
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
27/48
This page intentionally left blank.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
28/48
Data Warehouse Implementation TDWI Data Warehousing Concepts and Principles
3-16 © The Data Warehousing Institute
Warehouse Data ModelingLogical Models of Dimensional Data
• Warehousing Subjects• Business Questions• Facts & Qualifiers• Target Configuration
• Staging, Warehouse, & MartER Models
• Data Mart DDMs
• Staging Area Structure• Warehouse Structure• Relational Mart Structures• Dimensional Mart Structures
• Source Composition• Source Subjects
• Integrated Source Data Model(ERM)
• Source Data Structure Model
• Business Goals & Drivers• Information Needs
Triage
ContextualModels
ConceptualModels
Logical Models
StructuralModels
• Staging Physical Design• Warehouse Physical Design• Data Mart Physical Designs
(relational & dimensional)
• Source Data FileDescriptions
c u s t o m e r - c o u n t
h o u s e h o l d - c o u n t
S I Z E O F
C U S T O M E R B A S E
P r o d u c t L O B
l o b - c o d e
l o b - n a m e
P R O D U C T L I N E
l i n e - c o d e
l i n e - d e s c r i p t i o n
P R O D U C T
p r o d u c t - i d p r o d u c t - d e
s c
p r o d u c t - n a m e
T i m e
M O N T H
m o n t h - n u m b e r
Q U A R T E R
q u a r t e r - n u m b e r
Y E A R
y e a r - n u m b e r
G e o g r a p h i c A r e a
R E G I O N r g n - c o d e
r g n - n a m e
D I S T R I C T
d i s t - n u m b e r
d i s t - n a m e
Z O N E
z o n e - n u m b e r
z o n e - n a m e
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
29/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Implementation
© The Data Warehousing Institute 3-17
Warehouse Data ModelingLogical Models of Dimensional Data
EXAMPLE The diagram on the facing page illustrates an example of a dimensionaldata model at the logical level. This example shows a data mart whose
purpose is to measure the size of the customer base.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
30/48
Data Warehouse Implementation TDWI Data Warehousing Concepts and Principles
3-24 © The Data Warehousing Institute
The Warehouse Process ModelSource/Target Maps
member-number
membership-type
date-joined
date-last-renewed
term-last-renewed
date-of-last-activity
last-name
first-name
business-name
addresscity-and-state
zip-code
M E M B E R S H I P M A S T E R
date-time
terminal-id
transaction-id
line-number
SKU
F i l e s
/ T a
b l e s a n
d F i e l d s
f r o m
S o u r c e
S t r u c
t u r a
l M o d e
l
Tables and Data Elements from Target Structural Model
t r a n s a c
t i o n - d
a t e
t r a n s a c
t i o n - t
i m e
s t o r e - n
u m
b e r
t r a n s a c
t i o n - a
m t
r e g
i s t e r -
i d
t r a n s a c
t i o n - s
t a t u s
p a y m e n
t - m e
t h o
d
p r o
d u c
t - c o
d e
p r o
d u c
t - S K U
p r o
d u c
t - t y p e
p r o
d u c
t - d e s c r i p
.
m e m
b e r n u m
b e r
c u s
t o m e r n a m e
m e m
b e r s
h i p d a
t e
r e n e w a
l d a
t e
c u s
t o m e r a
d d r e s s
SALES TRANSACTIONCUSTOMER PRODUCT
P O I N T
- O F
- S A L E D E T A I L
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
31/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Implementation
© The Data Warehousing Institute 3-25
The Warehouse Process ModelSource/Target Maps
SOURCE ANDTARGET DATA ASSOCIATIONS
Source/target mapping develops detailed understanding of theassociations between source data and target data. Mapping may occur atthree levels:
• Mapping entities to understand the business associations
• Mapping tables and files to understand associations among datastores
• Mapping columns and fields to understand associations at the dataelement level
The focus of this mapping is on what associations exist, withoutexamining which are the most desirable sources or how the data might
be translated.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
32/48
Data Warehouse Implementation TDWI Data Warehousing Concepts and Principles
3-34 © The Data Warehousing Institute
The Warehouse Process ModelData Transformation Rules
member-number
membership-typedate-joined
date-last-renewed
term-last-renewed
date-of-last-activity
last-name
first-name
business-name
address
city-and-state
zip-code
M E M B E R S H I P M A S T E R
date-time
terminal-id
transaction-id
line-number
SKU
F i l e s
/ T a
b l e s a n
d F i e l d s
f r o m
S o u r c e
S t r u c
t u r a
l M o
d e
l
Tables and Data Elements from Target Structural Model
t r a n s a c
t i o n - d
a t e
t r a n s a c
t i o n - t
i m e
s t o r e - n
u m
b e r
t r a n s a c
t i o n - a
m t
r e g
i s t e r -
i d
t r a n s a c
t i o n - s
t a t u s
p a y m e n
t - m e
t h o
d
p r o
d u c
t - c o
d e
p r o
d u c
t - S K U
p r o
d u c
t - t y p e
p r o
d u c
t - d e s c r i p
.
m e m
b e r n u m
b e r
c u s
t o m e r n a m e
m e m
b e r s
h i p d a
t e
r e n e w a
l d a
t e
c u s
t o m e r a
d d r e s s
SALES TRANSACTIONCUSTOMER PRODUCT
P O I N T
- O F
- S A L E D E T A I L
DTR027 (Default Membership Type) If membership-type is null or invalid assume “ family” membership
DTR009 (Translate StatusDTR008 (Derive Name) If membership-type is business use business-name else concatenate last-name and first -name separated by a commaDTR009 (Translate Statu s
c e l l s e x p a n d t o i d e n t i f y t r a n s f o r m a t i o n s b y t y p e & n a m e
l o g i c o f t r a n s f o r m a t i o n s i s s e p a r a t e l y d o c u m e n t e d
Cleansing DTR027 (Default Value)
Derivation DTR008 (Derive Name)
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
33/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Implementation
© The Data Warehousing Institute 3-35
The Warehouse Process ModelData Transformation Rules
DETAILEDSPECIFICATION
Specification of data transformations develops a large set of details abouthow source data is to be processed prior to loading of a warehousingdatabase. Documenting data transformation must address both theidentification of what transformations are needed, and the logic of thetransformation process.
Documenting which transformations occur can readily be achieved byextending the source/target maps. View the set of logic for eachtransformation as a unique rule, and develop a convention for namingthese rules. As each transformation need is identified, assign a name and
place that name in the appropriate cell of a source/target map. Then
document the logic of each transformation rule. For each source/targetassociation consider possible rules for each of the transformation types.
In addition, consider need for data cleansing. Although data clean-up isnot a unique transformation rule type, it is a common reason for filtering,conversion, and derivation.
Transform Need Descriptio n
Specify SelectionRequirements
Identifies and describes the selection processes needed to choose among multiple sources. Theobjective is to select the best data to be used for warehouse population. Selection requirementsmay exist at both data store and date element levels.
Specify FilteringRequirements
Identifies and describes the filtering processes needed to choose records from a source file (orrows from a source table) to be used for data warehouse population.
Specify Conversion andTranslationRequirements
Identifies and describes the conversion and translation processes which change the formats andvalues of data elements. Conversion processing achieves consistency of formats and value setsamong data extracted from multiple sources. Translation processes change data formats andvalues from encoded and cryptic to descriptive and meaningful.
Specify Derivation andSummarizationRequirements
Identifies and describes needed derivation processes used to develop a value for a single dataelement by applying logic to the values of some other data elements. It also identifies anddescribes the processes through which summary data values are created.
Specify Clean-upRequirements
Identifies and describes the clean-up processing needed to ensure quality and integrity of the datathat is placed into the data warehouse. Clean-up needs may exist at both data record and data
element levels. Among the issues of clean-up processing are intra- and inter-record consistencychecking, and decisions regarding elements with null values or invalid values.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
34/48
Data Warehouse Implementation TDWI Data Warehousing Concepts and Principles
3-42 © The Data Warehousing Institute
Deployed TechnologyRange and Roles of Technology
web desktop email wireless voice
B 2 E P o r t a
l ( i n
t r a n e
t )
B 2 B & B 2
C P
o r t a l s
( i n t e r n
e t / e x
t r a n
e t )
Anal yti c Ap pli cati ons
BPM (scorecards & dashboards)
CRM Analytics Supply Chain Analytics Operations Analytics
Anal yti c Ap ps DevelopmentTools, Packages, Templates Collaboration
E-mail, Groupware, Workflow
Data Access & An alysisQuery, Reporting, OLAP, Mining, Forecasting Text Analysis
Text Search & Text Mining
Data Warehouse / Data Marts Content Management
Data IntegrationModeling, Mapping, Cleansing, ETL
Data ResourcesOperational Systems, Documents, Images, External Data, Audio/Visual
I n f r a s
t r u c
t u r e
S t o r a g e ,
S e r v e r s , D
a t a b a s e s , M
e t a d a t a ,
A d m i n i s t r a t
i o n
& M a n a g e m e n
t , N e t w o r
k i n g
web desktop email wireless voice
B 2 E P o r t a
l ( i n
t r a n e
t )
B 2 B & B 2
C P
o r t a l s
( i n t e r n
e t / e x
t r a n
e t )
Anal yti c Ap pli cati ons
BPM (scorecards & dashboards)
CRM Analytics Supply Chain Analytics Operations Analytics
Anal yti c Ap ps DevelopmentTools, Packages, Templates Collaboration
E-mail, Groupware, Workflow
Data Access & An alysisQuery, Reporting, OLAP, Mining, Forecasting Text Analysis
Text Search & Text Mining
Data Warehouse / Data Marts Content Management
Data IntegrationModeling, Mapping, Cleansing, ETL
Data ResourcesOperational Systems, Documents, Images, External Data, Audio/Visual
I n f r a s
t r u c
t u r e
S t o r a g e ,
S e r v e r s , D
a t a b a s e s , M
e t a d a t a ,
A d m i n i s t r a t
i o n
& M a n a g e m e n
t , N e t w o r
k i n g
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
35/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Implementation
© The Data Warehousing Institute 3-43
Deployed TechnologyRange and Roles of Technology
TECHNOLOGYROLES ANDRELATIONSHIPS
The technology framework illustrates classes of tools and technology frominfrastructure through information delivery. This framework includesestablished and mainstream technologies as well as emerging technologies(content management, text analysis, text mining, collaboration, etc.) that aregaining significance in data warehousing. The major technology classes are:
Delivery Delivery media includes web portals, desktop clients, email, wireless, voice print, pager and fax.Delivery technology sets include (1) B2E Portal – intranet business-to-enterprise delivery to theworkforce, (2) B2B Portal– internet business-to-delivery to vendors, customers, partners and anyonewith internet access, (3) B2C Portal – extranet business-to-customer delivery.
Analytic Applications Analytic applications are the technology components of business applications, ranging from staticreporting to dashboards and scorecards. They place information into business function context, i.e.Customer Relationship Management (CRM), Supply Chain Management (SCM), BusinessPerformance Management (BPM), etc.
Analytic ApplicationDevelopment Tools
Tools, templates, and packaged applications to quickly build views, reports, dashboards, scorecards,and other applications to deliver information in context of a business function or business process.
Collaboration Web applications to support employees, partners, customers, vendors and others to collaborate ondocuments, share business metrics, manage content, and work collectively. While reporting is stilldominant today, collaboration capabilities will grow as the technology and market place mature.
Data Access & Analysis Data access and analysis tools are today’s most common delivery technologies. Unlike analyticapplications, these tools focus on data before information, and they provide less business contextthan analytic applications. The most widely-used tools include managed reporting, query, and OLAP.
Text Analysis Text analysis tools use semantics and statistical techniques to identify, tag, and select relevantcontent from text documents. Parsing, pattern recognition, natural language processing and otheradvanced techniques are used to transform unstructured text into data and/or information structures.
Data Warehouse / Marts Data warehouses and data marts integrate and reconcile data from multiple data sources. Theirpurpose is to prepare data to serve as the raw material from which information is created. Regardlessof the multiple definitions of data warehouse and data mart that are used in the industry, allwarehouses and marts exist primarily to serve this purpose.
Content Management Content management technology first emerged as an internet technology – to support management ofcontent-rich web sites. Uses of the technology in BI are emerging as the industry evolves from datawarehousing to business intelligence, and from integration of structured data integrating all types ofbusiness information resources. Basic content management functions include indexing, searching,and retrieval.
Data Resources This class includes all sources from which data can be acquired. When both internal and external dataare considered, and when both structured and unstructured data are included, the range of possiblesource technologies becomes exceptionally broad.
Infrastructure This technology class describes the underlying hardware, software, networking, administration andsupport structures upon which systems and data sources are constructed and operated.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
36/48
Data Warehouse Implementation TDWI Data Warehousing Concepts and Principles
3-50 © The Data Warehousing Institute
Delivery ResultsData and Information Services
The right kinds of services matched to the customer’s roles, responsib ilities, and experience level
ExpertOccasional
Use Beginner
Data Access and
Information DeliveryServices
Analysis & ReportingServices
Training Services
Support Services
Regular Use
Business Manager
Knowledge Worker
Executive
a r r a y o f
B I s e r v i c e
s
ExpertOccasional
Use Beginner
Data Access and
Information DeliveryServices
Analysis & ReportingServices
Training Services
Support Services
Regular Use
Business Manager
Knowledge Worker
Executive
a r r a y o f
B I s e r v i c e
s a r r a y o f
B I s e r v i c e
s
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
37/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Implementation
© The Data Warehousing Institute 3-51
Delivery ResultsData and Information Services
MEETINGCUSTOMER NEEDS
A mature data warehousing environment includes a robust set of servicesthat support the goal of delivering the right services to the right people atthe right time. A three-dimensional view of the services array is useful toclassify services and to assess customer needs and match them withavailable services. The services dimensions are:
• Classification of customers aso Knowledge workers who carry out the day-to-day activities of the
businesso Managers responsible for performance of individual business
processeso Executives responsible for business performance across many
business processes
• Classification of customer experience aso Experts who use the data warehouse regularly and have a high
level of computer and analytic skills combined with an intimateknowledge of data warehouse content
o Regular users of the data warehouse with moderate computer andanalytic skills combined with a working knowledge of datawarehouse content
o Occasional users of the data warehouse who may have necessarycomputer and analytic skills, but have limited knowledge of datawarehouse content
o Beginners with little or no knowledge of data warehouse content,and who may have limited computer or analytic skills
• Classification of services aso Data access and information delivery services that make data and
information available to the business.o Analysis and reporting services that deliver analytic applications
of greater complexity than simple data access and informationdelivery.
o Training services that develop customer skill and ability to usethe data warehouse, with a goal of making each customer self-sufficient.
o Support services that augment the services culture, enhancecommunications with customers, and ensure rapid resolution of
problems.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
38/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Operation
© The Data Warehousing Institute 4-1
Module 4Data Warehouse Operation
Topic Page
Business Services 4-2
Data Warehouse Administration 4-6
Managed Quality 4-14
Managed Infrastructure 4-16
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
39/48
This page intentionally left blank.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
40/48
Data Warehouse Operation TDWI Data Warehousing Concepts and Principles
4-2 © The Data Warehousing Institute
Business ServicesValuable and Sustainable Services
V A L U A B L E A N D
S U S T A I N A B L E
D A T A & I N F O R M
A T I O N S E R V I C E
S
F O R T H E B U S I N
E S Sbusiness servicesdata refreshmanaged platformsmanaged environmentcustomer servicemanaged qualitymanaged infrastructure
Operation
Outcomeachievement, discovery
Actioninsight, resolve, decision, innovation
Knowledgerecall, experience, instinct, beliefs
Datadescriptive, quantitative, qualitative
Informationfacts, metrics
impact
V A L U A B L E A N D
S U S T A I N A B L E
D A T A & I N F O R M
A T I O N S E R V I C E
S
F O R T H E B U S I N
E S Sbusiness servicesdata refreshmanaged platformsmanaged environmentcustomer servicemanaged qualitymanaged infrastructure
Operation
business servicesdata refreshmanaged platformsmanaged environmentcustomer servicemanaged qualitymanaged infrastructure
Operation
Outcomeachievement, discovery
Actioninsight, resolve, decision, innovation
Knowledgerecall, experience, instinct, beliefs
Datadescriptive, quantitative, qualitative
Informationfacts, metrics
impact
Outcomeachievement, discovery
Actioninsight, resolve, decision, innovation
Knowledgerecall, experience, instinct, beliefs
Datadescriptive, quantitative, qualitative
Informationfacts, metrics
impact
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
41/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Operation
© The Data Warehousing Institute 4-3
Business ServicesValuable and Sustainable Services
WAREHOUSINGFOR THELONG TERM
Sustaining the data warehouse demands a commitment to deliveringreliable and valuable business services in an environment of high-frequency change. Value is sustained by ensuring continuous alignmentwith changing business needs and with a changing customer base.
Reliability is sustained by attention to all of the “under the hood”components upon which the services depend including:
• Regular, routine, and dependable data refresh despite changing datasources and systems.
• Effectively managed technology platforms from data acquisition toinformation delivery in a climate of rapid technological change.
• Managed environment including security, growth, capacity planning,and configuration management.
• Customer service including support, help desk, and training services.• Continuous quality management for all aspects of quality – business
quality, data and information quality, and technical quality.• Actively managed infrastructure that ensures continued alignment of
people, processes, and technology for optimum business value.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
42/48
Data Warehouse Operation TDWI Data Warehousing Concepts and Principles
4-14 © The Data Warehousing Institute
Managed QualityDimensions of Quality
w h a
t l e v e
l o
f n e e
d s a n
d e x p e c t a
t i o n s
?
w h a
t c u r r e n
t l e v e
l o
f s e r v
i c e
?
focus on business driversalignment with business strategies
enabling of business tactics
understanding of purpose, content, & servicesaccess to needed business information
satisfaction with information availability and reliability
reach into the business communityrange of data and services
maneuverability as change occurscapability to use, adapt, extend & evolve business intelligence
w h a
t l e v e
l o
f n e e
d s a n
d e x p e c t a
t i o n s
?
w h a
t l e v e
l o
f n e e
d s a n
d e x p e c t a
t i o n s
?
w h a
t c u r r e n
t l e v e
l o
f s e r v
i c e
?
w h a
t c u r r e n
t l e v e
l o
f s e r v
i c e
?
focus on business driversalignment with business strategies
enabling of business tactics
understanding of purpose, content, & servicesaccess to needed business information
satisfaction with information availability and reliability
reach into the business communityrange of data and services
maneuverability as change occurscapability to use, adapt, extend & evolve business intelligence
focus on business driversalignment with business strategies
enabling of business tactics
understanding of purpose, content, & servicesaccess to needed business information
satisfaction with information availability and reliability
reach into the business communityrange of data and services
maneuverability as change occurscapability to use, adapt, extend & evolve business intelligence
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
43/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Operation
© The Data Warehousing Institute 4-15
Managed QualityDimensions of Quality
QUALITYIMPROVEMENT
Quality, as with any other aspect of business, is effectively managed withmeasures and metrics. A metrics foundation for quality managementincludes both measures of product quality and measures of the processthat produces the product. In the case of business intelligence, the
products are BI results – information delivered to the business, analyticsused by the business, actions and outcomes enabled through BI, etc. The
processes are those necessary to execute the entire chain of events fromdata warehousing to business action, and to sustain a BI program overtime. Product measures are used to detect defects in BI products and toimprove those products. Process measures help to identify causes ofdefects and prevent reoccurrence through process improvement. A mature
quality process regularly adjusts quality targets to achieve continuousimprovement.
DIMENSIONS OFQUALITY
Business intelligence quality is much more than simple data quality. Dataquality is, in fact, a relatively small and easy piece of the overall qualitydomain. BI quality is measured and managed in three major categories:
• Business Quality directly affects the business value derived from BI,and the economic success of the BI program.
• Information Quality is related to acceptance and use of BI products
– the extent to which BI customers value those products. Informationquality is a significant factor in political success of BI.
• Technical Quality involves choosing the right technologies,configuring multiple technologies to work well together, and usingthe right tools for the right job. High-quality implementation oftechnology is typically unnoticed by the business. Low-quality,however, is highly visible and directly affects overall acceptance,usage, trust, value realization, and sustainability of a BI program.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
44/48
Data Warehouse Operation TDWI Data Warehousing Concepts and Principles
4-16 © The Data Warehousing Institute
Managed InfrastructureProcesses, Technology, and People
program managementchange managementquality management
data governancedevelopment methodologies
project managementdata warehouse administration
metadata managementdata warehousing tools & technology
BI tools & technologyinfrastructure tools & technology
BI roles & responsibilitiesBI organizations
program managementchange managementquality management
data governancedevelopment methodologies
project managementdata warehouse administration
metadata managementdata warehousing tools & technology
BI tools & technologyinfrastructure tools & technology
BI roles & responsibilitiesBI organizations
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
45/48
TDWI Data Warehousing Concepts and Principles Data Warehouse Operation
© The Data Warehousing Institute 4-17
Managed InfrastructureProcesses, Technology, and People
COMPLEXINFRASTRUCTURE
Infrastructure is the foundation upon which BI operates and grows. Whilethe infrastructure supports development, it’s more critical role is inoperating and sustaining BI solutions. Operation and sustenance are bothmore demanding and of longer duration than development. An effectiveBI infrastructure is one in which processes, technology, and people workseamlessly to support a BI culture and to realize business value from BIsolutions.
PROCESS This course has already discussed the analytics processes of BI. Whensuccessful, BI becomes a key component in decision making processes. Itdepends, however, on many other processes to achieve this level of
success. The process components of BI infrastructure are programmanagement, change management, data governance, developmentmethodology, project management, data warehouse administration, andmetadata management.
TECHNOLOGY While technology can’t create BI, neither can BI be created without use oftechnology. Blending the right technologies with the process and peoplecomponents of BI is a key to success. Technology infrastructure includesdata warehousing tools, BI tools, and enabling/infrastructure hardwareand software.
PEOPLEPeople are integral to effective BI. Neither processes nor technology candeliver value independently of the knowledge, decisions, and actions of
people. Human infrastructure is arguably the single most important of allBI infrastructure categories. Identifying the right set of roles andresponsibilities, assigning them to people with the right skills, andconstructing the right kinds of organizations and relationships are allcritical to BI success.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
46/48
TDWI Data Warehousing Concepts and Principles Summary and Conclusions
© The Data Warehousing Institute 5-1
Module 5Summary and Conclusions
Topic Page
Common Mistakes 5-2
References and Resources 5-6
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
47/48
TDWI Data Warehousing Concepts and Principles Summary and Conclusions
© The Data Warehousing Institute 5-3
Common MistakesFrom TDWI’s 10 Mistakes Series
An eff ect ive pro jec t manager wil l not …1 Accept an unrealistic schedule.2 Take on a failing project.3 Launch a project with a dysfunctional team.4 Choose the wrong sponsor.5 Accept unrealistic expectations.6 Expand the project scope.7 Skip the project plan.8 Fail to put the project agreement in writing.9 Let IT drive the project.
10 Give others authority to select software.11 Market the project alone.
Effective team-builders will avoid …
1 Hiring yourself.2 Squelching disagreement.3 Confusing titles with roles and responsibilities.4 “Talking the walk.”5 Thinking one size fits all.6 Pointing fingers.7 Interviewing only for technical skills.8 Limiting leadership.9 Becoming too task focused.
10 Believing that all decisions are created equal.
An eff ect ive data m odeler wil l avoid …
1 Not gathering business requirements.2 Saving time by not creating a subject area model.3 Delivering normalized tables to drive data mart design.4 Designing the staging process for ease of developers at end-user expense.5 De-normalizing without starting from a fully normalized data model.
6 Allowing users to drive the level of detail.7 Not modeling all levels of a multi-tiered warehousing environment.8 Developing a data model from a list of required data elements.9 Believing you must choose between relational and dimensional models.
10 Jumping straight into data mart design.
8/15/2019 93039928 TDWI DW Concepts and Principles Preview
48/48
Summary and Conclusions TDWI Data Warehousing Concepts and Principles
References and ResourcesPublications
BEST BOOKS:
INTERNET SITES:
Marco – Building & Managing the Metadata Repository2000, John Wiley & Sons
Moss & Atre - Business Intelligence Roadmap2003, Addison -Wesley
Inmon – Building the Data Warehouse (3rd Edition)2002, John Wiley & Sons
Inmon, Imhoff & Sousa – Corporate Information Factory (2nd Edition)2000, Johy Wiley & Sons
Kimball - The Data Warehouse Toolkit1996, John Wiley & Sons
Kimball, Reeves, Ross & Thornthwaite – The Data Warehouse Lifecycle Toolkit1998, John Wiley & Sons
Adelman & Moss – Data Warehouse Project Management2000, Pearson Education
The Data Warehousing Institute (www.dw-institute.com)Business Intelligence and Data Warehousing
The Data Administration Newsletter (www.tdan.com)Information and Data Management
The Data Warehousing Information Center (www.dwinfocenter.org)Data Warehousing Resources
Inmon Associates, Inc.(www.billinmon.com)The Inmon Approach to Data Warehousing
The Ralph Kimball Group (www.rkimball.com)The Kimball Approach to Data Warehousing