10/3/2012 1 1 Instituting Good Practices for Data Testing Best Practices for Data Integration / ETL Testing Series David Loshin, Industry Analyst Senthil Kanakarajan, Wells Fargo Ash Parikh, Informatica Next-Generation Data Integration Series 30 Minutes with Industry Experts
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
10/3/2012
1
1
Instituting Good Practices for Data Testing Best Practices for Data Integration / ETL Testing Series
David Loshin, Industry Analyst
Senthil Kanakarajan, Wells Fargo
Ash Parikh, Informatica
Next-Generation Data Integration Series
30 Minutes with Industry Experts
10/3/2012
2
2
10/3/2012
3
3
10/3/2012
4
4
SQL scripts?
Spreadsheets?
Hand coding?
Testing tool?
Not enough?
10/3/2012
5
5
Long time
Expensive
Error-prone
Manual
Not thorough
No reuse
No audit trail
No methodology
10/3/2012
6
6
Managing the Usability of Test Data
Production Data vs. Test Data
Reusable Logic / Repeatable Process
Maintaining Same Test Data Set
Other…
Meeting Project Deadlines
Improving Test Data Over Time
10/3/2012
7
7
definition of business rules introduction of data controls automated testing, validation & reporting
10/3/2012
8
8
A way to identify ERRORS in data sets that have
been MOVED or TRANSFORMED to ensure
they are COMPLETE and ACCURATE and
meet EXPECTATIONS or REQUIREMENTS.
10/3/2012
9
9
AUTOMATION
REPEATABILITY
AUDITABILITY
10/3/2012
10
10
Data Being Transformed
• ETL Reconciliation
• Data Masking
• ETL Testing
• Application Migration
Data is Identical
• ETL version upgrade
• ETL Migration
• Database migration
• Application Retirement
10/3/2012
11
11
Production Reconciliation
Protect the integrity of data
that is loaded into
production systems.
Erroneous data due to failed
loads, faulty logic or operational
issues is caught in a proactive
automated manner and can be
addressed as needed
Development & Test
Provide automation for unit and
regression testing
of integration logic.
Ensure that data produced by DI
code meets requirements and
expectations
Informatica’s Data Validation Solution (DVO)
Ensures the integrity of data as it moves through the IT environment...
10/3/2012
12
12
Production Reconciliation
Protect the integrity of data
that is loaded into
production systems.
Erroneous data due to failed
loads, faulty logic or operational
issues is caught in a proactive
automated manner and can be
addressed as needed
Development & Test
Provide automation for unit and
regression testing
of integration logic.
Ensure that data produced by DI
code meets requirements and
expectations
Informatica’s Data Validation Solution (DVO)
Ensures the integrity of data as it moves through the IT environment...
10/3/2012
13
13
Production Reconciliation
Protect the integrity of data
that is loaded into
production systems.
Erroneous data due to failed
loads, faulty logic or operational
issues is caught in a proactive
automated manner and can be
addressed as needed
Development & Test
Provide automation for unit and
regression testing
of integration logic.
Ensure that data produced by DI
code meets requirements and
expectations
Informatica’s Data Validation Solution (DVO)
Ensures the integrity of data as it moves through the IT environment...
10/3/2012
14
14
Benefits
REQUIREMENTS MANUAL
TESTING INFORMATICA
“We used Informatica to compare 14 tables & appx. 30 million rows in less than 5 hours. The largest of the tables was 94 columns.
When I asked our QA people how long it would take them to run the scripts and test this amount of data, they mentioned months…”
- Customer
10/3/2012
15
15
Informatica.com > Products > Enterprise Data Integration >