MSBI – Microsoft Business Intelligence A Brief Introduction on DATA Warehouse SSIS , SSAS , SSRS
MSBI – Microsoft BusinessIntelligence
A Brief Introduction on DATA Warehouse
SSIS , SSAS , SSRS
• what is Data warehouse ?? with different examples. How data mining works with data warehouse.
• What is MSBI ??
• Why Customers are moving towards MSBI (Microsoft SQL Server 2005/2008) though they already had Other RDBMS for eg. (Oracle) as source.
• Understanding the terminologies of Data warehouse (DWH)
This Presentation will Clears you the
following things.
What is Data Warehouse
Raugh kimball – In simplest terms Data Warehouse can be defined as collection of Data marts. -Data marts : Subjective collection of Data.
Bill Inmon – A data warehouse is a “subject-oriented, integrated, timevariant,and nonvolatile” collection of data in support of management’s decision-making process.”ERP
will Run the Business- like how Tyres Run the Car
BI (Reports,Data mining,Dashboards,kpi’s)will help you to take business decisions based on your historical data. - like Steering, mirrors, breaks, dashboards will help, how smoothly you can run the Car or reach the Destination.
4
Which are our lowest/highest margin customers ?
Which are our lowest/highest margin customers ?
Who are my customers and what products are they buying?
Who are my customers and what products are they buying?
Which customers are most likely to go to the competition ?
Which customers are most likely to go to the competition ?
What impact will new products/services have on revenue and margins?
What impact will new products/services have on revenue and margins?
What product prom--otions have the biggest impact on revenue?
What product prom--otions have the biggest impact on revenue?
What is the most effective distribution channel?
What is the most effective distribution channel?
In What way a Data warehouse helps any BusinessLet’s say A producer wants to know….
5
Data, Data everywhere yet ...• I can’t find the data I need
– data is scattered over the network– many versions, subtle differences
• I can’t get the data I need• need an expert to get the data
• I can’t understand the data I found• available data poorly documented
• I can’t use the data I found• results are unexpected• data needs to be transformed from one
form to other
6
A single, complete and consistent store of data obtained from a variety of different sources made available to end users in a what they can understand and use in a business context.
[Barry Devlin]
What is Data Warehouseing
7
What are the users saying...
• Data should be integrated across the enterprise
• Summary data has a real value to the organization
• Historical data holds the key to understanding data over time
• What-if capabilities are required
8
A process of transforming data into information and making it available to users in a timely enough manner to make a difference
[Forrester Research, April 1996]
Data
Information
What is Data Warehouseing
9
Data Warehousing -- It is a process• Technique for assembling and
managing data from various sources for the purpose of answering business questions. Thus making decisions that were not previous possible
• A decision support database maintained separately from the organization’s operational database
10
Data Mining works with Warehouse Data
Data Warehousing provides the Enterprise with a memory
Data Mining provides the Enterprise with intelligence
We want to know ...• Given a database of 100,000 names, which persons are the least likely to
default on their credit cards?
• Which types of transactions are likely to be fraudulent given the demographics and transactional history of a particular customer?
• If I raise the price of my product by Rs. 2, what is the effect on my ROI?
• If I offer only 2,500 airline miles as an incentive to purchase rather than 5,000, how many lost responses will result?
• If I emphasize ease-of-use of the product as opposed to its technical capabilities, what will be the net effect on my revenues?
• Which of my customers are likely to be the most loyal?
Data Mining helps to extract such information
What is BI : Implementing a data warehouse on any business using some set of tools is called Business Intelligence.
Now what is MSBI : From SQL SERVER 2005 Microsoft introduced a complete tool kit inside the SQL SERVER to implement Data warehoue.
That means a set tools from MICROSOFT SQL SERVER which helps you to implement a data warehouse is called MSBI.
Which includes the following SSIS – SQL Server Integration services (ETL Tool)SSAS – SQL Server Analysis servicesSSRS – SQL Server Reporting services
Now what is MSBI ??
Administrator : The person who can manage the complete administration part of the Data warehouse, similar to DBA.
ETL Developer : The person who works on SSIS or any other ETL Tools like Informatics.
Cube Developer/OLAP Developer : The person who works on SSAS Cube creations and MDX language.
Report Developer : The person who works on SSRS
The Different Roles of Data WarehouseDevelopers
Base Product$ 25K $ 40K $ 25K
Oracle 10g
IBM DB2
Why Only MSBI ?When you had OBIEE
(Oracle)When you had BWH
(SAP)
Base Product
Manageability
(included)
$ 25K $ 40K $ 25K $ 56K $ 35K
Tuning $3K
Diagnostics $3K
Partitioning $10K
Performance
Expert$10K
Base Product
Manageability
(included)
$ 25K $ 35K $ 154.5K $ 56K$ 116K
Business Intelligence
OLAP $20k
Mining$20k
BI Bundle$20k
DB2 OLAP $35KDB2
Warehouse $75K
Cube Views $9.5K
Base Product
Manageability
(included)
$ 25K $ 154.5K $ 164.5K $ 232K$ 116K
Business Intelligence
High Availability
Data Guard $116K Recovery
Expert$10k
Base Product
Manageability
(included)
High Availability
Business Intelligence
Multi-core
$348k - $464k
$ 232K$ 25K $ 164.5K $ 329K
$164.5K$116K - $232K
Additional BenefitNumber of Users
What happened?
Why did it happen?
What will happen?
What happened why and how?
Reporting
Interactive OLAP
Ad hoc-Queries
Data Mining
FRONTEND TOOLS
DWH TERMINOLOGIESOLTP – Online Transaction ProcessingOLAP – Online Analytical ProcessingMOLAP – Multidimensional OLAPROLAP – Relational OLAPHOLAP – Hybrid OALP Dimensions – De-normalized master tables Attributes – Columns of DimensionsHierarchies – sequential order of attributesFacts (Measure group) – Transactions tables in DWHFact (Measures)Cubes – Multidimensional storage of DataKPI’s – Key performance indicatorDashboards – combination of reports,kpis,chartsData Marts – Subjective Collection of DataSCD’s – Slowly changing DimensionsPerspectives – Child Cube
DWH ARCHITECTURE
OperationalData Sources
Data-Migration Middleware (Populations-Tools)
DataStorage
Repository
DataAnalysis
Reporting, OLAP,Data Mining
PRACTICAL SCENARIO
Stage DB Optional
ROLAP
OLTP
MOLAP
O L A P
SSIS
Integration Services Analysis Services
Reporting Services
SSAS
SSRS
SSISData Marts
CUBE
OLTP VS OLAP1. OLTP (on-line transaction processing)
2. Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll, registration, accounting, etc.
1. OLAP (on-line analytical processing)
2. Data analysis and decision making
3. The tables are in the Normalized form. 3. The tables are in the De-Normalized form.
5. For Designing OLTP we used data modeling.
5. For Designing OLTP we used Dimension modeling.OLAP is classified into two i.e.,MOLAP & ROLAP
4. We Called the Storage objects as Tables. i.e., All the masters and the Transactions are stored in the tables.
4. We Called the Storage objects as Dimension and Facts. i.e., All the masters Are dimension and the Transactions are Facts.
OLTP ROLAPM a s t e r s D i m e n s i o n s
Product
Prod_Id
Prod_Name
Base_Rate
Cat_IdCategory
Cat_Id
Cat_Name
Cat_Desc
Group_IdGroup
Group_Id
Group_Name
Group_Desc
Product_Dim
Prod_Id
Prod_Name
Base_Rate
Cat_Name
Cat_Desc
Group_Name
Group_Desc
Topics Later We will Cover
2. Slowly changing Dimensions1. Types of Dimensions
3. Hierarchies
Normalized Tables De-Normalized Tables
OLTP ROLAPTr a n s a c t i o n s F a c t s
SalesOrder_Fact
Cust_Id
Prod_Id
Order_Date
Delivery_Date
Unit_Price
Qty
Total_Amount
Tax
SalesOrderDetails
Cust_Id
SalesPerson
Prod_Id
Order_Date
Booked_Date
Delivery_Date
Unit_Price
Qty
Tax
Created_By Qty*Unit_Price+Tax=Total AmountUsually calculate all the calculations before storing into OLAP
Reference keys of Dimensions
Numeric fields called as Fact or measure
ROLAP MOLAPD i m e n s i o n s & F a c t s C u b e s
Prod_Dim
Prod_Id
………
Cust_Dim
Cust_Id
………
Time_Dim
Date
Year
Month
………
Org_Dim
Org_Id
………SalesOrder_Fact
Cust_Id
Prod_Id
Order_Date
Delivery_Date
Org_Id
Unit_Price
Qty
Total_Amount
Tax
STAR Schema
NOW WHAT IS OLAP ?
Product_Dim
Prod_Id
Prod_Name
Base_Rate
Cat_Name
Cat_Desc
Group_Name
Group_Desc
SalesOrder_Fact
Cust_Id
Prod_Id
Order_Date
Delivery_Date
Unit_Price
Qty
Total_Amount
Tax
D i m e n s i o n s F a c t s C u b e s
ROLAP MOLAP
MOLAP SCHEMAS t a r S n o w F l a k e
1. Dimensions will have only relation with the Fact. (Normalized model)
1. Dimension will have a relation other than Fact. (De-Normalized model)
2. One to many or One to One relation will Occur.
2. Used for many to many relation.
3. Performance is fast but required huge storage space.
3. Performance is Low but required Less storage space.
QUERIES ?
THANK YOU