Michael Barger
Benchmarking For Best Practice
What is Benchmarking?• A tool for continuous improvement
• Compliments Lean and Six Sigma activities
• Lean and Six Sigma projects develop improved processes
• Benchmarking finds best processes already developed
Benchmarking’s Roots• The term “benchmarking” was
originally a land surveyor’s term used in topographical surveys
• A benchmark was a distinctive mark made on a rock, a wall, or building that served as a reference point in determining one’s current position and altitude
• This was a sighting point from which additional measurements could be made
Definitions• Benchmarking: A strategic and analytic
process of continuously measuring an organization’s products, services and practices against a recognized leader
• Best Practices: A process or a practice that causes performance as measured by a specific metric to be superior
Best Practices Benchmarking• Best Practice Benchmarking is straight forward
for an organization that already uses metrics to track processes
• Best Practice Benchmarking can be used instead of a Six Sigma Project or can compliment one
Types of Benchmarking• Process benchmarking - investigates business
processes
• The goal to identify the best practices from other benchmark firms
• Activity analysis used to benchmark cost and efficiency
• Often applied to back-office processes
• Financial benchmarking - performing comparative financial analysis to assess overall competitiveness and productivity
• Performance benchmarking - competitive analysis by comparing products and services against target firms
Types of Benchmarking (continued)
• Product benchmarking – focused on product improvement
• Can involve reverse engineering which is taking apart competitors products to find strengths and weaknesses
• Strategic benchmarking - observes how others compete
• Usually not industry specific
• Best to look at other industries
• Functional benchmarking - focus on a single function to improve its operation
• Complex functions such as human resources, finance, accounting and IT are unlikely to be directly comparable
• May need to focus on sub-processes to make valid comparison
The Four Phase Benchmarking Model• Phase I: Plan
• Phase II: Do
• Phase III: Act
• Phase IV: Track
I: The Plan Phase
The Plan Phase• Select the process to be benchmarked
• Select and prepare the benchmarking team
• Identify benchmarking partners• Research information sources for best
practices• Rank potential partners [1-50]• Select the final partners [1-5]• Know & use the benchmarking guidelines for
ethical conduct
Establishing a Benchmarking Project• Motivation
• Team selection and building
• Framing the problem
• Select the process to be benchmarked
• Senior management buy-in
• Resource allocation
• Identify benchmarking partners
Motivation• Justification: Where is the pain?
• Resources
• Return on investment
• Result of current path
Building the Team• Establish the team and the role of the team to
align with the overall goals and objectives
• Identify formal and informal roles
• Team building process
Team Building Process• Careful member selection (skills, motivation,
personality, availability)
• Establish Ground Rules: The team’s role and responsibilities
• The organization’s role and responsibilities
• Other key issues in the team building process
• Develop a Team Charter
• Adopt the characteristics of a high performing team
A High Performing Team• Clear vision
• Full participation
• Clear Communication
• Process improvement plan
• Clearly defined roles
• Collaborative behavior
• Well-established Decision Process
• Established Norms
• Team Management Process
Team Introductions• Name
• Job title & position
• Professional/Career Goals
• General Background information (hobbies, marital status, family information, sports etc.)
• Knowledge/experience with benchmarking activities
• Two-minute interviews & briefings by colleagues
Formal Roles• Champion
• Team leader
• Team member
• Facilitator
• Senior management representative
• Information manager
• Technical expert
Team Process Roles• Team Leader
• Facilitator
• Scribe
• Timekeeper
Roles can be rotated
Team Documents• Team Charter
• Working Agreement
• Code of Conduct
• Team Roles and Processes
• Communications Plan
Communication Plan• Communication Plan is essential
• Establishes roles and responsibilities
• Addresses both internal and external information
• Documents how information will be communicated
(meetings, issues, deliverables, resources, project status, reports, etc.)
• Documents to whom information will be communicated
Communication Plan (Continued)
• Informs the leaders and others on what to expect in terms of how the teams BM project may impact other projects-efforts within the organization
• Creates an a sense of shared commitment & ground rules within team
• Encourages a culture of openness, trust and acceptance, and provides an opportunity to exchange information, ideas and information in a timely manner
Frame the Problem• Develop a problem statement
• Use an Ishikawa Diagram or 5 Why methods
• Establish a root cause
• Avoid presupposing the solution
• Establish a clear vision of the outcome
• Build consensus
Stakeholders• Identify who the stakeholders are in your
organization
• Reach a group consensus
• From the group consensus identify which stakeholders will support your benchmarking team
• Plan to brief these individuals on the approach, progress and the benchmarking findings at critical milestonesThese individuals can help determine the
success of the benchmarking effort.
Develop an Information Briefing• Vision
• Plan
• Methods
• Expected results
Selecting Benchmarking Partners• Research information sources for possible
partners• Dun and Bradstreet• Business lists• Personal knowledge• Current business partners
• Rank potential partners [1-50]• Establish selection criteria• Poll or survey the potential partners• Score information against selection criteria
• Select the final partners [1-5]
Partner Selection CriteriaSample criteria for identifying benchmarking partners:
• Is the benchmarking partner comparable financially (similar revenues, sales, profits)?
• Is the benchmarking partner comparable size (similar number of employees)?
• Does the partner engage in comparable functions (similar work process, methods, practices)?
• Does the partner have comparable outputs (similar products & services)?
Partner Selection Criteria (Continued)
• Does the partner have similar customer expectations?
• Does the partner have comparable organization and divisional structures?
• Does the partner have comparable inputs (supplies & ingredients)?
• Is the partner part of a comparable market sector (public, private, government)?
• Does the partner have comparable logistics (similar set-up & workflow)?
II: The Do Phase
The Do Phase• Determine the data collection plan
• Collect and analyze the data
• Train team members in skills & tools as necessary
• Determine performance gaps and strengths
• Analyze performance gaps & strengths
• Produce a benchmarking report
How will Data Be Collected?• What metrics will provide the data to compare
current and benchmarked processes?
• What kind of information or measurement is necessary? (accuracy, quality, customer satisfaction, speed, etc.)
• What data exists on the internal process?
• How should the team collect the internal data?
• Who on the team will collect the data?
How will the Data Be Collected? (Cont.)
• How will the team check its results?
• How much time will be needed to collect the data?
• How will the data be consolidated and analyzed?
• How will the satisfaction of the stakeholders be evaluated during the process?
• What method(s) should be used to collect data from the benchmarking partners?
Research Methods• Internet-based information searches
• Interviews/focus groups
• Archival data
• Observation
• Surveys
• Selecting and establishing metrics
• Internal experts
Some Examples of Metrics• Productivity, by transactions per unit
• Accuracy, by the error rates
• Speed, by the cycle time
• Product stability, by the number of engineering change orders per month
• Process financial contribution, by the value-to-cost ratio
• Product availability, by fill rate
• Product quality, by first-pass yield
• Capacity, by volume managed
• Service, by the on-time delivery
Considerations for Metrics• Reliability - consistent measurements
• Validity – measuring the right things
Reliability of MetricsEUI* for Five Manufacturing Plants for Three
MonthsPlant Month 1
Month 2
Month 3
Mean
A 5.5 6.2 3.6 5.1
B 3.2 5.0 2.9 3.7
C 5.1 4.3 5.8 5.1
D 1.3 4.2 4.5 3.3
E 5.1 2.0 1.1 2.7*Energy Use Intensity (Kwh/sf/month)
Note that the best plant cannot be determined. Analysis of Variance indicates that most of the variation is due to measurement error.
Validity of Metrics• Does it make sense to the experts?
• Does it have a strong relationship with other metrics that are similar?
• Does it have weak or dissimilar relationships with metrics that are different?
• Is it free from other influences that might be causing the results?
Sample Considerations• Sample Selection
• Nearest Neighbor Bias
• Success Bias
• Selection Bias
Sample Selection• Sample should be representative
• Processes should be comparable
• Should represent a range of practices
• Objective: Discover what differentiates best from the rest
Nearest Neighbor Bias
Look beyond organizations that you already know about. Include organizations that are unfamiliar, but relevant, if possible.
“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.”
-Donald Rumsfeld
Success Bias• Avoid selecting only highly successful firms
• A range of success is needed
• Differentiation is the key
Selection Bias• We are often unaware that we are sampling
the survivors
• This produces a biased sample
• The results may be misleading
• Finding results for organizations that did not survive can help understand what is differentiating a best practice
Data Collection• Establish the purpose for the data
• Define the type of data to be collected
• Determine when and how will data be collected
• Determine who will collect the data
• Control Charts
• Multi-attribute Utility Analysis
• Triangulation
• Errors to Avoid
• Check Sheets
• Histograms
• Pareto Analysis
• Scatter Diagrams
• Fishbone Analysis
Process and Tools
Some Pitfalls to Avoid• Small sample bias
• Attribution bias
• Correlation vs. causation
• Confirmatory Bias
• Groupthink
• Hidden Profiles
Small Sample Bias• Conclusions drawn from a small sample of
observations
• The observations may be outliers or random fluctuations
• Larger samples are needed to draw stable conclusions
Attribution Bias• An incorrect conclusion that A is causing B
because it stands out
• Example: innovation is attributed to a strategic planning process when the planning process has little to do with it
• Collecting more data and seeking the root causes can help avoid this bias
Correlation vs. Causation• Two variables are related but A does not
necessarily cause B
• Need to have a better root cause explanation than simply observing the relationship
• Seek out a clear rational explanation for the relationship to ensure causality
Confirmatory Bias• Once an initial idea is accepted we tend to
seek out only confirmatory information
• We avoid or disregard disconfirmatory information
• Can lead to invalid conclusions
• Facilitator and team should be aware of this tendency
Groupthink• A feature of groups, particularly when they are
cohesive
• Need a “devil’s advocate” to challenge too ready acceptance of ideas and conclusions
• Self-censorship and majority view does not necessarily indicate unanimous consensus
Hidden Profiles• Team members often bring different
information to meetings
• In many cases not all of the relevant information is shared
• Good facilitation can overcome this bias
Determine Gaps and Strengths• Analyze the gaps/strengths in your current
business process against your benchmarking partners and determine areas to target for improvement
• Do a performance gap analysis with a detailed comparison of the current process to the best-in-class
• List the best practices and the strengths where benchmarking partners display superior performance showing parity where there are no significant differences
Determine Performance Gaps (Cont.)
• Describe where your internal practices are superior to the benchmarked partners
• Produce the analysis necessary for the benchmarking report and prepare to make recommendations to the process owners based on that analysis
• Determine reasons for the gaps projecting any future competitive gaps
III: The Act Phase
ACT Phase - Implementation• Develop an Action Plan for implementation
• Communicate the Action Plan
• Execute the Action Plan
• Recalibrate benchmarks
Transfer/Implementation• Strategically and tactically plan how you will
incorporate best practice within your organization and record, analyze and evaluate findings
Implementation & Transferability Plan• Build awareness and communication
• Ensure that there are sufficient resources
• Build relationships between the source and the recipient of the knowledge
• Assure Timely implementation
• Link to a strategic plan
Develop an Action Plan
Plan How to:
• Achieve desired results
• Measure the results
• Monitor feedback on the process changes
• Identify tasks necessary to implement the process changes
• Identify necessary training
Obtain Stakeholder Approval The following should be asked before implementation:
• Is the plan clear?
• Does the plan show how gaps in performance will be closed?
• Will it lead to sufficient changes?
• Has the management supported the team and recognized its accomplishments?
• Is the organization ready to implement successfully the changes to the process?
• Is the process in place to implement the plan?
• Are there sufficient resources for the plan?
Implementation of Best Practices• Build awareness and communication
• Ensure sufficient time and resources
• Build relationships between the source and the recipient of the knowledge
• Adhere to a suitable schedule
• Strategically and tactically plan the implementation of the best practice
• Record, analyze, and evaluate findings
IV: The Track Phase
Track Phase – Monitoring the Implementation
• Review the Effectiveness of the Best Practice Implementation
• Modify the Action Plan as Necessary to Improve Implementation
• Review the Effectiveness of the Best Practice
• Continually Monitor Means for Improvement
Evaluation of Implementation• Follow-up Surveys
• Quasi-Experimental Designs
• Single-group interrupted time series design
• Interrupted time series design with control groups
Recalibrate Benchmarking• Need for advancement still exists
• Reset targets
• New values become internal measurements for the next benchmarking effort
• Establish a new process baseline
• Continue to monitor your current best practice against others
Monitor the Benchmarked Process • Confirm strategic alignment
• Continually monitor customer satisfaction
• Determine if any additional world-class process has emerged
Repeat the Cycle• Conduct ongoing visits with benchmarking
partners
• Consider recalibrating annually
• Perform the steps of the benchmarking model
Continuous
Improvement
PLAN
DO
ACT
TRACK