This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Business Analytics is the engine and Data is the Oil of the 21st Century Accurate Decision Making Requires Trusted Data Trusted Data needs End-to-End Governance
The Journey to Better Decision Making: Data Governance Maturity Model
TELS
TRA T
EMPL
ATE
4X3
BLU
E BET
A |
TELP
PTV4
TELS
TRA T
EMPL
ATE
4X3
BLU
E BET
A |
TELP
PTV4
TELS
TRA T
EMPL
ATE
4X3
BLU
E BET
A |
TELP
PTV4
A JOURNEY TO TRUSTED DATA
RICK ANDREWS 21 MAY 2012
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
ABOUT YOUR SPEAKER
RICK ANDREWS, SENIOR DATA QUALITY SPECIALIST BUSINESS INTELLIGENCE CENTRE OF EXCELLENCE, TELSTRA.
RICK HAS BEEN WORKING FOR OVER 25 YEARS IN THE TELECOMMUNICATIONS INDUSTRY, DESIGNING AND MANAGING APPLICATIONS TO DELIVER IMPROVED CUSTOMER EXPERIENCE.
THE PAST 10 YEARS HAVE SEEN A PARTICULAR FOCUS ON DATA QUALITY PROGRAMS BRINGING EXPERIENCE TO DELIVER KEY INSIGHT INTO THE GOVERNANCE AND METHODOLOGY REQUIRED TO ACHIEVE SUCCESS.
RICK HOLDS A MASTERS OF BUSINESS ADMINISTRATION SPECIALISING IN INFORMATION SYSTEMS, WITH A BACKGROUND IN ELECTRICAL ENGINEERING.
CONTENT
- About Telstra - The Information Journey – Information Factory - The Challenge – Data Trust - The Solution – A Data Quality Firewall - Data Quality Firewall Framework – Overview - Initial Dimensions of Data Quality - Initial Data Quality Measure Matrix - How It Works - Simple Version - Data Quality Firewall – Project Approach - Data Management – A Holistic Approach
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
TELSTRA IS AUSTRALIA'S LEADING FULL SERVICES TELECOMMUNICATIONS AND INFORMATION SERVICES COMPANY, WITH ONE OF THE BEST KNOWN BRANDS IN THE COUNTRY.
TELSTRA OFFERS A FULL RANGE OF PRODUCTS AND SERVICES TO MEET THE NEEDS OF OUR CUSTOMERS. OUR TELECOMMUNICATIONS SERVICES PROVIDE VAST GEOGRAPHICAL COVERAGE THROUGH BOTH OUR FIXED AND MOBILE NETWORK INFRASTRUCTURE - SERVICING MANY MILLIONS OF AUSTRALIAN CUSTOMERS.
IN 2010-11, SALES REVENUE WAS $25.0 BILLION AND REPORTED EBITDA WAS $10.2 BILLION.
IN 2010-11, TELSTRA GENERATED $5.5 BILLION OF FREE CASH FLOW AND PAID $3.5 BILLION IN DIVIDENDS TO SHAREHOLDERS.
TELSTRA HAS ONE OF THE LARGEST SHAREHOLDER BASES IN AUSTRALIA, WITH 1.4 MILLION SHAREHOLDERS AND MILLIONS MORE INVESTING THROUGH SUPERANNUATION FUNDS.
TELSTRA EMPLOYS ALMOST 40 THOUSAND PEOPLE AROUND THE WORLD.
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
ABOUT TELSTRA
IN 2010-2011 TELSTRA:
HANDLED 3.6 BILLION LOCAL CALLS, 5.4 BILLION NATIONAL LONG DISTANCE MINUTES, 9.9 BILLION TEXT MESSAGES AND MORE THAN 13.5 BILLION MOBILE VOICE TELEPHONE MINUTES.
PROVIDED 8.4 MILLION FIXED LINE AND 12.3 MILLION MOBILE SERVICES, INCLUDING WHOLESALE SERVICES.
COMMENCED WEEKEND TECHNICIAN SERVICES, WITH AROUND 80 THOUSAND WEEKEND APPOINTMENTS USED BY CUSTOMERS.
LAUNCHED 24/7 CONTACT CENTRES, WITH MORE THAN 5 MILLION CALLS MADE TO US AFTER HOURS.
IMPLEMENTED TELSTRA.COM IMPROVEMENTS WHICH HAVE ENABLED MORE THAN 300 THOUSAND CUSTOMERS TO ACTIVATE INTERNATIONAL ROAMING ONLINE.
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
ABOUT TELSTRA’S SERVICE
THE INFORMATION JOURNEY INFORMATION FACTORY
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
BUSINESS EVENTS occur & raw data is created
1
PROCESSES to extract, transform and load data are
applied by computer systems in the Enterprise Data Warehouse
3
FINISHED INFORMATION in the form of reports, self-serve
access, analytics access, etc. is delivered to the business to
enable decision making
4
TRUST QUALITY is critical to
data & information usage & business
adoption
5
Subassemblies of INFORMATION such as employee & product
hierarchy, market analysis, etc. may be combined
2
CORPORATE DASHBOARD
SELF SERVE ANALYTICS
Accurate
Complete
Timely
Trusted
Relevant Understood Engineered
THE CHALLENGE DATA TRUST
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
It’s hard to manage and trust data when you don’t know exactly where its come from or what quality it is
SOURCE SYSTEMS LEGACY REPORTING PLATFORMS
CANNED REPORTIN
G
SQL DATA EXTRACT
BUSINESS CHALLENGES
Reconciliation overhead
Data quality Timeliness
Manual Collation
Reliability
Continuous improvement culture
Trust in data
THE SOLUTION A DATA QUALITY FIREWALL
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
PROPOSED SOLUTION
Introduction of a Data Quality Firewall into Enterprise Data Warehouse to provide proactive detection and alarming of data quality issues from receipt of data from source systems to presentation in the BI and Analytics layer
ANTICIPATED RESULT
Improve the communication of data health to all EDW users: Provide a business centric view Provide an IT operational centric view
Accurate Complete Timely
Trusted
Relevant Understood Engineered
Key behavioural elements valued by
users & considered the ‘tipping point’ for
EDW adoption & usage
FIREWALL MEASURES
Systematic measure based data analysis
By design, not accident, the data is structured
LEVELS OF MATURITY
ENHANCE THROUGH CONTINUOUS IMPROVEMENT
PROFILE, MEASURE & MONITOR TO ENABLE YOU TO MANAGE YOUR DATA
NO VISIBILITY OF DATA LINEAGE OR QUALITY
DATA MANAGEMENT - ALARM, FIX, ENRICH, SUSPEND OR BLOCK DATA
DATA QUALITY FIREWALL FRAMEWORK OVERVIEW
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
You Can't Manage What You Don't Measure
INITIAL DIMENSIONS OF DATA QUALITY
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
Correct references to related entities (Referential Integrity, Entity Integrity)
Entities exist only once within a data set
Conformance to business rules for the entity (excluding Referential Integrity)
Data is not missing and is of sufficient breadth and depth for its specified use
Time elapsed since real world entity was recorded or business event occurred
Data is sufficiently up-to-date for the task at hand
INTEGRITY Accurate
UNIQUENESS Accurate
VALIDITY Accurate
COMPLETENESS
Complete
CURRENCY Timely
TIMELINESS Timely
INITIAL DATA QUALITY MEASURE MATRIX
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
TIER 0 SOURCE TO STAGING
TIER 1 STAGING TABLES
TIER 2 STAGING TO ADS
TIER 3 ADS TABLES
TIER 4 ACCESS LAYER
INTEGRITY N/A N/A
UNIQUENESS N/A
VALIDITY N/A N/A
COMPLETENESS N/A
CURRENCY N/A N/A
TIMELINESS N/A N/A N/A
HOW IT WORKS SIMPLE VERSION
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
Outputs
DQ Rules Engine Items To Be Tested
EDW
Schedule
DATA QUALITY FIREWALL PROJECT APPROACH
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
Framework Integration into SDLC
Tool Selection & Installation Dashboard Retrofit
COMPLETED Sept 2010
COMPLETED Nov 2010
COMPLETED Jan 2011
COMPLETED Jan 2012 IN PROGRESS
The framework explains the firewall concept and purpose. It outlines the high level DQ measures & control points (Tiers) that will be implemented. Framework complete, business case approved & socialised.
Input DQ controls into the Software Development Lifecycle to address new initiatives / new data sources i.e. • Data Profiling • Batch Load Patterns • DQ Firewall Measures • Testing, etc
1. Ascertained options from the Gartner Magic Quadrant for Data Quality Tools
2. Approached top 4 vendors to respond informally to our problem statement & DQ Firewall Framework
3. Completed tool selection process
4. Worked through procurement process
5. SAS Dataflux software installed
1. SAS Dataflux training held in Jan 2011
2. Working with Teradata / SAS to deliver foundation build (Feb–Mar 2011)
3. Deliver the 1st EDW Currency Dashboard in May 2011
4. Deliver Tiers 0 – 4 by Jan 2012
1. Construct DQ Tier Control measures for all data within the EDW
2. Create alarms and automated resolution workflow
3. Understand current pervasive DQ issues and build rules to ensure that the issue is proactively identified
DATA MANAGEMENT A HOLISTIC APPROACH
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012
The Data Quality Firewall won’t deliver trust in isolation
THANK YOU FOR YOUR ATTENTION
A JOURNEY TO TRUSTED DATA | RICK ANDREWS | 21 MAY 2012