Redman-Toronto-ten habits- May2012 © Navesink Consulting Group, 2000- 2012 T.C. Redman, Page 1 The Ten Habits of Those with the Best Data Thomas C. Redman, Ph.D. Navesink Consulting Group at the Toronto DAMA May 17, 2012 [email protected]
Dec 26, 2015
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T.C. Redman, Page 1
The Ten Habits of Thosewith the Best Data
Thomas C. Redman, Ph.D.
Navesink Consulting Group
at the Toronto DAMA
May 17, 2012
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 2
Introduction and Summary
Those with the best data: Adopt a customer-facing definition of quality. Aim to prevent errors at the points of data creation,
rather than correcting them downstream. Follow “ten habits” that align the entire organization. Enjoy rich rewards for their troubles!
Agenda
What “the best data” looks like Thinking about quality An “non-delegatable” choice The ten habits My (evolving) views on organization Questions anytime
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 3
Market Data Vendor
Background: Financial services companies purchase market data from companies such as Reuters, Bloomberg, etc.
Lack of trust causes them to purchase basic data from multiple sources.
Bank request: Far better data, so it could reduce its vendor base and eliminate downstream costs of bad data.
Work conducted: Clear statement of customer needs. Measurement against those needs. Root causes identified and addressed, one at a time. Statistical control. In the course of day-in, day-out work.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 TCR, Page 4
Market Data Example: Results
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 TCR, Page 5
Each error not made saves an average of $500. Quickly millions!
The day-in, day-out work of data quality management is conducted at the work group level
Access Financial Assurance at AT&T
Background: AT&T expenditures for “access” about $20B/yr. Access Financial Assurance aims to ensure integrity of access
bills, through parallel “billing.”
Key Idea: Get the bill right the first time.
Work conducted: Dissatisfied middle manager, seeking a better way. Top-down deployment. Staff group defined series of deliverables, then audited (regional)
compliance. Supplier and process management. Customer needs, measurement, improvement, control.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 TCR, Page 6
Results: Access Financial Assurance
Data accuracy improved 90%. Billing errors reduced 98%. Cycle time (bill period closure) reduced 67%. AT&T costs (of financial assurance) reduced 73%
($100M/year). LEC costs (of access billing) reduced 20%.
Redman-Toronto-ten habits-May2012 TCR, Page 7© Navesink Consulting Group, 2000-2012
There is much hidden “non-value-added work” built in to accommodate bad data.
Enterprise Programme at BT*(British Telecom) Revenue: $33 billion/yr Employees: 95,000 Operates in 170 countries 22 Million customers (4 Million business)
Enterprise Data Quality Improvement Programme (10-year effort) Recognized the inherent complexity of people, process, technology issues
(e.g., data quality problems masquerading as “systems issues.”) Explicit linkage of data (quality improvement) to strategic business
objectives (e.g., business transformation). Over time, magnitude of DQ problem understood and exposed. Governance structure starting at the very top. Consolidated expertise in data quality improvement in IT. Estimated and delivered benefits vetted by Finance.
Redman-Toronto-ten habits-May2012 TCR, Page 8
*This summary largely courtesy of Nigel Turner, who led BT’s programme and is now at Trillium Software. He has vetted this summary.
© Navesink Consulting Group, 2000-2012
BT – ResultsEnterprise Data Quality Improvement Programme, cont’d Dual focus on reducing capital expenditure and the rework that results
from searching for “lost network facilities.” Problem discovery, measurement, audits, new controls (hold the gains)
enhanced by Trillium DQ tool suite. Focused on “big improvement projects” (delivered 75 over ten years). Dual focus on data clean-up and process improvement.
Redman-Toronto-ten habits-May2012 TCR, Page 9© Navesink Consulting Group, 2000-2012
More than you might think, data permeate everything. Bad data are “silent killers.
Business Benefits: > $1B (verified and conservative) Also improved customer satisfaction, better regulatory compliance, reversed brand damage and revenue leakage, and contributed to business transformation: These benefits not quantified.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 10
High Quality in Your Mind’s Eye
Redman’sFavorites
Apple products
Italian tile
C&D Heating and Cooling
Disneyland
Common Characteristics
Relatively few defects.
They are corrected in a prompt, friendly manner.
Easy-to-use
Make it easier to do the things I want to do.
Sleek design!
Trust the company!!
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 11
Contrast the I-Phone with a Data Model
Or a financial statement!
E.g.,
"Corporate structure”
"Employment"
"Membership"
etc.
PARTYPARTYRELATIONSHIP
PARTYRELATIONSHIPTYPE
ORGANI-ZATION
PERSON
from
on one side of
to
on the other side of
an example of
embodied in
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 12
Data QualityData are of high quality if they are fit for their intended uses
(by customers) in operations, decision-making, and planning (after Juran).
free of defects:- accessible- accurate- timely- complete- consistent with other sources- etc.
possess desired features:- relevant- comprehensive- proper level of detail- easy-to-read- easy-to-interpret- etc.
Data that’s fit for use
Customers are the ultimate arbiters of quality!!
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 13
Data Quality - aspirational
“Exactly the right data and information in exactly the right place at the right time and in the right
format to complete an operation, serve a customer, make a decision, or set and execute
strategy.”*
*Redman, Data Driven: Profiting from Your Most Important Business Asset, Harvard Business Press, 2008
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 14
Data Quality – day-in, day-out
Meeting the most important needs of the most important customers.
© Navesink Consulting Group, 2000-2012 T. C. Redman, Page 15
Data Quality: The Non-delegatable Choice
Redman-Toronto-ten habits-May2012
Un
man
aged
Pre
vention at s
ource
Find and fix
Eliminate The Sources Of PollutantTo Clean Up The Lake, One Must First
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 16
They recognize that, left alone, accountability shifts downstream!!!
Here’s how youdo number 3,
soncos2(x) + sin2(x) = 1
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 17
The (nearly-certain) results
Approach
Management
Focus
Typical
Error RateCost of Poor Data Quality
Find and Fix
(First-Gen)The Past
1-5% (at the field level)
20% of revenue
Prevent Future Errors (Sec-Gen)
The FutureTwo orders of
magnitude better
Reduced by two-thirds
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 18
Habit 1: Focus on the most important needs of the most important customers
Those with the best data adopt a customer-facing definition of quality.
In doing so, they recognize that: All data are not created equal. Similarly, customers,
problems, and business opportunities are not created equal.
Generally, the most important data are those needed to set and execute the company’s most important business strategies.
And they focus as much of their energies on these customers, strategies, and data.
Said differently, their data quality programs are fully aligned with business strategy.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 19
Data Doc’s Hierarchy of Needs
1. Acquire the data they need
2. Trust that data are correct
3. Understand meaning
4. Understand how data fit with
other data
5. Keep data safe from harmMany people and
organizations exhibit a
“hierarchy of “needs”
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 20
Habit 2. Process, process, process
They recognize that they create data via their cross-functional business processes
A B C D
They recognize that most errors occur “in the white space”
They think “BIG-P”
They recognize “the next guy” (serving the customer) as a customer
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 21
Data - Defined
A datum consists of three elements:
The thing of interest in the real-world
The particular of interest
(entity, attribute, value)
The value assigned to the attribute for the entity
Example: (Jane Doe, Service Record Date = July 1, 1996) Note that, as defined, data are abstract. “Customers” see them as they are
presented in tables, databases, graphs, etc.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 22
Implications…
Thus even the simplest datum arises from three distinct sources:
The model (entity, attribute)-pair is created within a modeling process, usually by IT or purchased from outside.
The data value is created (at enormous rates) by the business process.
The presentation may be created by database tools, application programs, PowerPoint presenters, etc in an application development process.
All three processes must be managed end-to-end for high-quality data to result.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 23
They use the Customer-Supplier Model to establish requirements and feedback loops
Suppliers Customers“Your Process”
inputs outputs
requirementsrequirements
feedback feedback
BIG-P process
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 24
Habit 3: They employ supplier management for external sources of data
Suppliers Customers“Your Process”
inputs outputs
requirementsrequirements
feedback feedback
They expect high-quality data from outside. And invest (time) with their suppliers to get them
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 25
Habit 4: They measure quality at the source in business terms
They define metrics with clear business implications.
Private Bank’s Customer Data:
Percent of statements with
an error
Telecom’s Access Charges:
Risk = Overbilling + Underbilling
Many organizations:
Fraction “perfect” records
(interpreted as “work” done correctly)
Time-Series Record-Level Accuracy
0.20.30.40.50.60.70.80.9
1 3 5 7 9 11 13 15 17 19 21 23 25
week
frac
tio
n r
eco
rds
com
ple
tely
co
rrec
t
They measure continuously
They get good at interpreting results
They integrate top-line DQ metrics with other business results
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 26
Habit 5: They employ controls at all levels to halt simple errors and establish a basis for moving forward
Time-Series, Record-Level Accuracy
0.20.3
0.40.50.60.7
0.80.9
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37
week
reco
rds
com
ple
tely
co
rrec
t
UCL
LCL
They employ simple edits to stop errors in their tracks:
Ex: (Title = Mrs., Sex = M) cannot be correct
They employ statistical control to identify process issues early and to look forward:
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 27
Habit 6: They have a knack for continuous improvement
Time Series, Record-Level Accuracy
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 4 7 10 13 16 19 22 25 28 31 34 37 40 42 46 48
week
reco
rds
com
ple
tely
co
rrec
t
They have a way of not just starting, but completing improvement projects, both to:
• eliminate root causes of error
• acquire new data
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 28
Habit 7: Set and achieve aggressive targets
Time-Series, Record-Level Accuracy
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 4 7 10 13 16 19 22 25 28 31 34 37 40 42 46 48 52 55
week
reco
rds
com
ple
tely
co
rrec
t
They focus not just on the level, but also on the rate of improvement
They set targets like:
• half the error rate every year
• add two significant new features every year
They decide to position themselves near the front with respect to quality in their industries
In many respects, for them planning for quality is no different than planning for revenue growth, new product development, etc.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 29
Habit 8: Formalize management accountabilities for data
I’ve told that CIO about these data problemsa million times! Why can’t
they get them right?
They recognize that responsibility for data lies with “the business,” not IT.
Some codify responsibilities in policy.
My favorite (adopted for data):
“Don’t take junk data from the guy upstream. And don’t pass junk data on to the next guy!”
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 30
Habit 9: A broad, senior group leads the effort
They know that that quality programs go as far and fast as the senior person leading the effort demands.
So a broad, committed, senior team leads the effort.
“They thought they could make the right speeches, establish broad goals, and leave everything else to subordinates... They didn’t realize that fixing quality meant fixing whole companies, a task that can’t be delegated.”
Dr. Juran, 1993
Experience so far is that “data” is even tougher than the factory floor.
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 31
They: Distinguish “I” from “IT.” They recognize that they
can’t automate their way out of a quality issue. Start small. Create early wins. Actively manage change. Avoid unwinnable battles, especially early on. Build political capital. Over time, they build data quality into:
The organization People’s psyche To new systems
Habit 10: Recognize that the “hard issues are soft” and actively manage change
Who Does the Work?*8. Formalize management
accountabilities for data
Senior Leadership:
Middle Management (Command):
10. Advance a culture that
values data and data quality
9. Broad, informed,
demanding leadership
2. Manage processes that
create data (so they do so correctly)
3. Manage “suppliers” (both
inside the Army and out) of data
7. Set and meet
aggressive targets for
improvement: top-to-bottomWork is highly
interconnected
1. Focus on the most important
needs (of customers)
6. Improvement: Find and
eliminate root causes of error
4. Measure quality levels
against customer needs
5. Deploy controls, at all
levels, to remain error-free*
Everyone who touches data = Four Basic “Steps”
*Ten habits of those with the best data from Redman, Data Driven: Profiting from Your Most Important Business Asset, Harvard Business Press, 2008.
Taken together, the tasks define
an overall “Management
System for Data Quality”
Redman-Toronto-ten habits-May2012 TCR, Page 32© Navesink Consulting Group, 2000-2012
The ten habits reinforce one another*9. Broad senior
leadership
8. Data Policy
3. Supplier Mgmt
1. CustomerNeeds
10.ManageData Culture
5. Control4. Measurement 6. Improvement 7. QualityPlanning
DefinesAccountabilities
via
MustAdvance
DeployedTo
Supports
Supports
APlatform
For
LeadsTo
Identify"gaps"using
UnderliesEverything
Responsiblefor meeting
Responsiblefor meeting
ToBetterMeet
DeployedTo
MonitorConformance
Using
Underlies
Everything
SetTargets
For
2. Process Management
*This figure adapted from Redman, Data Quality: The Field Guide, Digital Press, Boston, 2001Redman-Toronto-ten habits-May2012 T. C. Redman, Page 33© Navesink Consulting Group, 2000-2012
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 34
The Ten Habits apply to all data, in all industries and government
Market, product, and people (customer and employee) data. Intelligence, scientific and logistics data. Health care data.
Data created internally or gathered from external sources.
Meta-data, master data, enterprise data. Data to be stored on paper, in operational systems, in
warehouses, enterprise systems. Client statements, 10-Ks, prospectuses. Data only seen by computers and data that convince
people to trust industries and companies (or not).
Fundamental Organization Unit for Data Quality
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 35
*Quality Improvement facilitator is a permanent role, supporting a series of project teams, which disband when their projects complete
Leadership
Tech SupportManager
Supplier Management
Process Management
Requirements Team
Control Team
Measurement Team
Customer Team
Improvement Teams*
Control Team
Measurement Team
Current “Best” Overall Organization Structure for Data Quality*
Data Council• Leadership• Data Policy• Define process and supplier structure• Advance Data Culture
Process B(metadata)
Supplier D
*This figure adapted from Redman, Data Quality: The Field Guide, Digital Press, Boston, 2001
Chief Data Office
• Day-in, day-out leadership• Secretary to Council• Metadata process owners• Training• Deep expertise• Supplier Program Office
Chief Information TECH Office• Technical Infrastructure• Security/Privacy impl.• Build DQ features into new systems• (One-time) data cleanups
Process AManage and
improve data, following agreed
process mgmt methods
Supplier CManage and
improve data, following agreed supplier mgmt
process)
… Other improvement projects …
“PROCESS VIEW:” These structures overlaid on current organization chart
Improvement project team
Follow agreed methodto complete assigned
project
Redman-Toronto-ten habits-May2012 T. C. Redman, Page 36© Navesink Consulting Group, 2000-2012
Primary responsibility for DQ
Primary responsibility for DQ
Federated Org Structure for Data QualityFederated Org Structure for Data Quality
Org HeadOrg Head
-
• Audit
-
• Audit
AuditAudit
•Policy Deployment•DQ Transparency•Common Methods
•Policy Deployment•DQ Transparency•Common Methods
Chief Data OfficeChief Data OfficeDepartment LeadershipDepartment Leadership
Senior Data BoardSenior Data Board
DQ Policy
Dpt DQ Team
Dpt DQ Team
Redman-Toronto-ten habits-May2012 TCR, Page 37© NCG and DBP, 2012
CreatorsCreators CustomersCustomers
Department Organization for Data Quality Department Organization for Data Quality
Note: DQ facilitator leads efforts to understand customer needs, conduct improvement projects, etc. Ideally, reports into functional management, with a dotted line to the data team, as pictured
Department Head Org’s Data Board
IT: Systems, databases, metadata repository, tools
MetricsTeam
TrainingTeam
ControlTeam
Metadata/Stds team
BusinessCase Team
FacilitationTeam
Head of Data Program
Data Creation
rqmtsrqmts
feedbackfeedback
input output
Support Tech
DataSupplier
DataCustomer
coordFunctional Activities
DQ facilitator
Dept DataTeam
ServicesTeam
Dept Data Committee
SupplierMgmt Team
Redman-Toronto-ten habits-May2012 TCR, Page 38© Navesink Consulting Group, 2000-2012
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 39
Final Remarks
Those with the best data: Adopt a customer-facing definition of quality. Aim to prevent errors at the points of data
creation, rather than correcting them downstream.
Follow “ten habits” that align the entire organization.
Enjoy rich rewards for their troubles!
Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 40
What Did He Say?
Questions?
Thomas C. Redman, Ph.D.