Informatica Scd Type-2 implementation What is SCD Type-2: The Type 2 method tracks historical data by creating multiple records for a given natural key in the dimensional tables with separate surrogate keys and/or different version numbers. With Type 2, we have unlimited history preservation as a new record is inserted each time a change is made. Type2 can be achieved in different ways. How to implement SCD Type2 through Informatica: There is number of way to implement SCD Type2 in informatica. Example: Create source table CUST with below query. CREATE TABLE CUST ( CUST_ID NUMBER, CUST_NM VARCHAR2(250), ADDRESS VARCHAR2(250), CITY VARCHAR2(50), STATE VARCHAR2(50), INSERT_DT DATE, UPDATE_DT DATE); Insert following data in CUST table. Source Data: CUST_ID CUST_NAME ADDRESS CITY STATE INSER_DT UPDATE_DT 80001 Marion Atkins 100 Main St. Bangalore KA 1/7/2011 1/7/2011 80002 Laura Jones 510 Broadway Ave. Hyderabad AP 1/7/2011 1/7/2011 80003 Jon Freeman 555 6th Ave. Bangalore KA 1/7/2011 1/7/2011 The Type 2 method tracks historical data by creating multiple records for a given natural key (CUST_ID) in the dimensional tables with separate surrogate keys (PM_PRIMARYKEY). For this create table with below query
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Informatica Scd Type-2 implementation
What is SCD Type-2:
The Type 2 method tracks historical data by creating multiple records for a given natural key in the
dimensional tables with separate surrogate keys and/or different version numbers. With Type 2, we
have unlimited history preservation as a new record is inserted each time a change is made. Type2 can
be achieved in different ways.
How to implement SCD Type2 through Informatica:
There is number of way to implement SCD Type2 in informatica.
Example:
Create source table CUST with below query.
CREATE TABLE CUST
(
CUST_ID NUMBER,
CUST_NM VARCHAR2(250),
ADDRESS VARCHAR2(250),
CITY VARCHAR2(50),
STATE VARCHAR2(50),
INSERT_DT DATE,
UPDATE_DT DATE);
Insert following data in CUST table.
Source Data:
CUST_ID CUST_NAME ADDRESS CITY STATE INSER_DT UPDATE_DT
80001 Marion Atkins 100 Main St.
Bangalore KA 1/7/2011 1/7/2011
80002 Laura Jones 510 Broadway Ave.
Hyderabad AP 1/7/2011 1/7/2011
80003 Jon Freeman 555 6th Ave. Bangalore KA 1/7/2011 1/7/2011
The Type 2 method tracks historical data by creating multiple records for a given natural key (CUST_ID)
in the dimensional tables with separate surrogate keys (PM_PRIMARYKEY).
Here active_dt and inactive_dt used to indentify history data.
PM_PRIMARYKEY is surrogate key which is used to identify each and every record uniquely in target.
The below diagram will explain high level over view of SCD type2 through informatica.
SCD Type-2 Over View
Flo
wF
low CUST SQ_CUST
lkp_CUST_D
exp_FLAG rtr_INS_UPD CUST_D_INSupd_INSERT
upd_UPDATE CUST_D_UPD
Before implementing we need to identify the attributes needs to be consider for history maintain.
In this example we will consider if any change in ADDRESS or CITY OR STATE.
So if any change in ADDRESS or CITY or STATE then we need to insert new record and inactivate old
record.
Creation of mapping:
Step1: First import source and target to informatica from data base.
Import the source definition CUST table using the Source Analyzer workspace. Go to Sources > Import
from Database.
This opens the Import Tables window. Assuming that a system DSN is already created for this
connection, specify all the necessary details and click Connect.
Select the CUST table to import and click OK to continue.
The CUST source definition is created and appears in the workspace. Click Save to save the source
definition in the repository.
The source table CUST contains only current data and doesn't have any historical data. This mapping
would be run daily to capture the historical data in the CUST_D target table. The Active and Inactive
Date logic would be used for SCD Type 2 mapping.
Follow the same steps using target Designer to import CUST_D table.
Now we have source CUST and target CUST_D tables are available.
Mapping creation:
Create mapping with name m_SCD_Type_2.
Now drag CUST table from sources to mapping designer workspace.
Create lookup table CUST_D
Create one input port in lkp_CUST_D table with name in_CUST_D with data type double. And add
condition CUST_ID=in_CUST_ID
Connect CUST_ID from source to in_CUST_ID lookup table
Create expression transformation and drag CUST_ID, CUST_NM, ADDRESS, CITY, STATE from Source
qualifier to expression. In the same why drag CUST_NM, ADDRESS, CITY, STATE from lkp_CUST_D table
to expression. Change attributes names in expression to identify source and lookup attributes as shown
in diagram.
In lookup transformation apply filter to retrieve only active records. This you can do it in lookup sql override. SELECT CUST_D.PM_PRIMARYKEY as PM_PRIMARYKEY, CUST_D.CUST_NM as CUST_NM, CUST_D.ADDRESS as ADDRESS, CUST_D.CITY as CITY, CUST_D.STATE as STATE, CUST_D.ACTIVE_DT as ACTIVE_DT, CUST_D.INACTIVE_DT as INACTIVE_DT, CUST_D.INSERT_DT as INSERT_DT, CUST_D.UPDATE_DT as UPDATE_DT, CUST_D.CUST_ID as CUST_ID FROM CUST_D WHERE INACTIVE_DT IS NOT NULL
Now create two output ports in expression transformation.
Out_DUMMY_DATE-- > which is used to populate ACTIVE_DT, INACTIVE_DT, INSERT_DT, UPDATE_DT
attributes in target. Assign SYSDATE for this attribute.
Out_FALG-- > which is used to flag record for insert new/insert update or update/Inactivate record.
To flag a record we need to check three conditions
Source record present in target or not.
For this if PM_PRIMARYKEY which is coming from source is null that means the record with
CUST_ID coming from source not present in target. This type of records needs to be inserted in
target directily.
Are there any changes in attributes from source and Target?
For this if PM_PRIMARYKEY is not null means the CUST_ID coming from source present in target.
Now we need to validate is there any changes in CUST_NM, ADDRESS, CITY, and STATE between
source and target data. If any change in data then need to insert this record in target at the
same time need to inactivate record present in target.
If record coming from source present in target and no changes between source and target
attributes then filter those records.
Write this condition in out_FLAG expression editor.
IIF(ISNULL(lkp_PM_PRIMARYKEY),1,IIF(lkp_CUST_NM!=src_CUST_NM OR
lkp_ADDRESS!=src_ADDRESS OR lkp_CITY!=src_CITY OR lkp_STATE!=src_STATE,2,3))
From the above condition if any record flagged as 1 that means it new record which is coming
from source and this is not available in target.
If any record flagged as 2 that means it’s already exists in target and there is a difference in
attributes from source and target.
If any record flagged as 3 that means this record present in source and target, there is no
difference between attributes.
Now create router transformation and drag the following attributes from expression to router