DEPARTMENT OF POSTS MINISTRY OF …tamilnadupost.nic.in/sdc/dpt/DataProfilingTool_Manual.pdf · MINISTRY OF COMMUNICATIONS & IT GOVERNMENT OF INDIA Data Profiler Tool ... (e.g. a

Confidential and Proprietary to India Post and Infosys Limited 1

DEPARTMENT OF POSTS

MINISTRY OF COMMUNICATIONS & IT GOVERNMENT OF INDIA

Data Profiler Tool Manual

Submitted by

Infosys Limited 44 Electronics City, Hosur Road

Bangalore – 560100 India

Document Name

DATA PROFILER-Operating

Procedure.doc

Version Rev.

1.0

Document Owner

Swetha (Infosys Data

Migration Team)

Date

3-May-2013

Confidential and Proprietary to India Post and Infosys Limited 2

Revision History:

Version Author Changes

1.0 Swetha Base line document

2.0 Selvakumar Changes to the section “Instructions to install and to

execute Data Profiler Tool” and section “Others”

3.0 Selvakumar Added new sections “Link for JRE7 download” and

“Instruction to find 32 or 64 bit operating system”

4.0 Selvakumar /

Meenakshi

Made changes to the section “Instructions to install” ,

added new section “How to execute Data Profiler

Tool” and updated Support Contact for North East

circle

[Meenakshi]:- Added a new section named FAQ

5.0 Selvakumar /

Anindya

Made changes in the following section

Instructions to Install How to execute Data Profiler Tool Detailed report Support Contact

5.1 Selvakumar Made change in the following section

Link for JRE7 download

Confidential and Proprietary to India Post and Infosys Limited 3

Table of Contents Data profiling: ............................................................................................................................................... 4

Prerequisites for using the tool: ................................................................................................................... 4

About Tool: ................................................................................................................................................... 4

Link for JRE7 download ................................................................................................................................. 4

Instruction to find 32 or 64 bit operating system ......................................................................................... 5

Instructions to Install .................................................................................................................................... 5

How to execute Data Profiler Tool ................................................................................................................ 5

Reports .......................................................................................................................................................... 8

Summary report ........................................................................................................................................ 8

Detailed report .......................................................................................................................................... 9

Instruction to upload to CBS Site ................................................................................................................ 10

Others ......................................................................................................................................................... 10

Frequently Asked Questions ....................................................................................................................... 10

Support Contact .......................................................................................................................................... 13

Confidential and Proprietary to India Post and Infosys Limited 4

Data profiling: Data profiling is the process of examining the data available in an existing data source (e.g. a database or a file) and collecting statistics and information about that data.

Prerequisites for using the tool: JRE 1.7 Supports MS SQL Server with version from 2000, 2005 and 2008 Adobe reader to view the report

About Tool: Data Profiler Tool is used for creating files that can be used for analyzing the quality of data based on

data migration rules.

Link for JRE7 download The link for JRE7 installable is available in the link

http://www.oracle.com/technetwork/java/javase/downloads/jre7-downloads-1880261.html

Please refer the link http://docs.oracle.com/javase/7/docs/webnotes/install/windows/jre-installation-

windows.html#choosing-which-jre-installer-to-download to know more about JRE Installation for

Microsoft Windows

Note: Since Infosys is not updating the site

http://www.oracle.com/technetwork/java/javase/downloads/jre7-downloads-1880261.html, the

content of the site may change and hence please take the exe present from the above link as below at

the time of referring.

Product / File Description

Download

Windows x86 Offline jre-7u45-windows-i586.exe (This may get changed time to time by Oracle)

Windows x64 jre-7u45-windows-x64.exe(This may get changed time to time by Oracle)

Confidential and Proprietary to India Post and Infosys Limited 5

Instruction to find 32 or 64 bit operating system

The following instruction is taken from the link http://windows.microsoft.com/en-us/windows7/32-bit-and-64-

bit-Windows-frequently-asked-questions

To find out if your computer is running a 32-bit or 64-bit version of Windows in Windows 7 or

Windows Vista, do the following:

1. Open System by clicking the Start button , right-clicking Computer, and then clicking

Properties.

2. Under System, you can view the system type.

If your computer is running Windows XP, do the following:

1. Click Start.

2. Right-click My Computer, and then click Properties.

If you don't see "x64 Edition" listed, then you're running the 32-bit version of

Windows XP.

If "x64 Edition" is listed under System, you're running the 64-bit version of Windows XP.

Instructions to Install

1. Create a folder called DataProfiler.

2. Download the DataProfilingTool.zip file from the site given by DoP and Save the .zip file to the DataProfiler folder.

3. Extract the zip file and put the contents of the zip file in the folder DataProfiler created in step 1

4. Open the userdpt .reg present in the folder DataProfiler and then replace (local) present in the line "Server"="(local)" with MS SQL server name having Sanchay Post data and save the file. For example, if the server name is SQLSERVER, then that line has to be

"Server"=" SQLSERVER "

How to execute Data Profiler Tool Preconditions:

Before executing DP the following needs to be done.

Confidential and Proprietary to India Post and Infosys Limited 6

a) DBA discrepancies should be selected before generation of DPT since few discrepancies

are being updated automatically.

b) DPT should be run after the day end process and also when the SQL server is in idle state

as it is a time consuming process.

Ensure that day end process is completed and MS SQL Server is in idle state and then follow the steps

1. Double Click on the dpt.bat present in the folder DataProfiler which is created in “Instructions to install” section. On double clicking the following screen will be displayed.

5. Enter the sa password and press enter

a. Where password is the sa password for connecting to MS SQL Server

6. Once the connection is established with given password to MS SQL Server, then the following screen will display and wait for the next details in the screen

Confidential and Proprietary to India Post and Infosys Limited 7

7. Once the execution is completed, the following screen will display and press enter key

Note:

It will take minimum 1 to 5 minutes based on volume of data. So please wait until you see the message “Press any key to continue”

Log file can be referred to know the details of error if any.

Confidential and Proprietary to India Post and Infosys Limited 8

8. The output files will be generated in the folder DataProfiler which is created in step 1 with the new name as current date time format every time the tool is executed (Example 20130503113819 if the tool is run on 3rd May 2013 at 11:38:19).

9. All files except the overall summary report will be in PDF format.

10. Open the folder to view the generated output files. Below picture is the screenshot of the generated files.

Reports

There are two types of Report

1. Summary report

2. Detailed report

Summary report

The summary reports with name as “Overall Summary Report <date time>_dcy.csv” is having

discrepancy count of each rule scheme wise along with its description as shown below

Product Name Rule name Rule Description

Record Count

SB SB_Accounts_Minor_status_as_Y_but_no_DOB_Count

Count of Accounts with Minor status is Y but no DOB 4

Confidential and Proprietary to India Post and Infosys Limited 9

SB SB_Ledger_Entries_without_index_entries_Count

Count of Ledger Entries that has no entries in Index table 1

SB SB_JointAccounts_without_depositor_address2_Count

Count of Joint Account Entries without address of the Depositor 9

SB SB_Ledger_Entries_0_account_number_Count

Count of Ledger table Entries with Account number as zero or no account number 1

SB SB_LotDate_greaterthan_CurrentDate_Count

Count of Last transaction date greater than current date 1

The report with name as Overall Summary Report <date time>_ency.csv is also having discrepancy count

of each rule scheme wise in encrypted format.

Detailed report

For each scheme, there will be a detailed discrepancy report rule wise. Each rule will have account

information along with respective fields if needed.

For Agent, there will be a separate detailed discrepancy report rule wise.

The sample detailed report for the product SB (Savings Bank) is given below

Confidential and Proprietary to India Post and Infosys Limited 10

The detailed report (PDF) will be generated only if there is a discrepancy for a product. If a product is not

having any discrepancy as per DPT, then detailed report (PDF) will not be generated for that product.

Instruction to upload to CBS Site To track the discrepancy level of each PO at central level, NISG team has come up with upload feature in

CBS Monitoring site.

For this, once the report is generated using the given Data Profiling Tool, follow the instruction to

upload

1. Open the CBS Monitoring web site given by NISG

2. Click on the upload button or upload link present in the CBS Monitoring web site given by NISG

3. Browse the folder and select the latest generated report with file name as “Overall Summary

Report <date time>_ency.csv” and then upload the file.

Others If there is any change in password or SQL Server name, then remove the already created Data Profiler

folder and its contents and then follow the steps mentioned in the section Instructions to install and to

execute Data Profiler Tool.

It is recommended to run after EOD operation is completed and there is no or minimal connection to the

SQL Server database.

Frequently Asked Questions 1. Java version Problem

If the below message is displayed on running the DPT tool. Kindly install the JRE version 7. Please

refer the section Link for JRE Download.

Confidential and Proprietary to India Post and Infosys Limited 11

2. Registry Entry Problem

If the below message is displayed on running the DPT tool. Kindly follow the below steps

Ensure that you login as an Admin in the Desktop in which you are running DPT tool.

Ensure SQL server name is edited correctly in the userdpt .reg as instructed in the

Instruction to install.

If still unresolved – follow the below steps and mail the screenshot to the relevant contact person

of your circle as in contacts below

1. Click the Start button, click Run, and type REGEDIT. Click OK.

You will view the screen as below

Confidential and Proprietary to India Post and Infosys Limited 12

2.

Check If the server name here is created as your SQL server name. kindly send the screenshot of the

same

3. Check in path C:\\Windows\\system32 whether you have sqlsrv32.dll

Confidential and Proprietary to India Post and Infosys Limited 13

Support Contact For any support, please contact the respective CPM for your respective circle.

Name of the Circle

Location Infosys Circle PM Name Email ID

Andhra Pradesh Hyderabad Sanjay Kumar Saboo [email protected]

Chattisgarh Raipur Swati Garg [email protected]

Delhi Delhi Saurabh Gupta [email protected]

Haryana Ambala Amit Soni [email protected]

Karnataka Bangalore Girisha Prabhu [email protected]

Kerala Trivandrum Rohin Prasad [email protected]

Punjab Chandigarh Jasbir singh [email protected]

Rajasthan Jaipur Yogesh Chutani [email protected]

Tamil Nadu Chennai Meenakshi Boopathi [email protected]

Maharashtra Mumbai Shreya Kohojkar [email protected]

Orissa Bhubaneshwar Debashish Mishra [email protected]

Uttarakhand Dehradun Ruchi Singh [email protected]

Confidential and Proprietary to India Post and Infosys Limited 14

Himachal Pradesh Shimla Vivek Walia [email protected]

Gujarat Ahmedabad Manisha Kanaiyalal Seth [email protected]

Madhya Pradesh Bhopal Samir Shrivastava [email protected]

Uttar Pradesh Lucknow Divya Diwedi [email protected]

Jammu & Kashmir Srinagar Peerzada Waseem Hussain Shah

[email protected]

Jharkhand Ranchi Susanta Kumar Banerjee [email protected]

West Bengal Bengal Anindya Ghosh [email protected]

Assam Guwahati Rudra Prasad Sharma [email protected]

Bihar Patna Siddarth Kumar [email protected]

North East Shillong Deepak Iyer [email protected]