© 2009 The MITRE Corporation. All rights reser Approved for Public Release. Case #09-3713. Approved for Public Release. Case #09-3713. September 11, 2009 Kevin Buck and Michele Steinbach Applying Best Practices for Performance Management Improvement
Dec 25, 2015
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
September 11, 2009Kevin Buck and Michele Steinbach
Applying Best Practices for Performance Management
Improvement
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Agenda Purpose of this briefing
The challenge: delivering on the strategic vision
Case study: the challenges for a federated Research and Development (R&D) enterprise
A performance management improvement decision framework
Case study: performance management recommendations
So why focus on performance management?
2
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Purpose
3
The purpose of this briefing is to describe how a government enterprise that is multi-faceted and multi-cultural can tailor and apply performance management principles to better manage their portfolios and achieve their vision.
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Delivering on the Strategic Vision Federal agencies are under increasing pressure to
demonstrate:– Delivery to stakeholders– A solid plan for achieving strategic goals
Articulating the strategic vision is not enough– Agency leaders must actively steer the enterprise toward its desired
end-state Moving from a description of the future to making that future
a reality involves:– Mapping out the transition plan, – Executing that plan, and – Monitoring plan execution to ensure the enterprise arrives at the
correct destination
4
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Case Study: Federated R&D EnterpriseKey Performance Management Challenges
Limited span of direct performance control– Some aspects of the portfolio are directly within the financial control and
authority of portfolio manager, while some aspects of the portfolio are notDecentralized portfolio
– The portfolio spans numerous organizational boundaries and performance management cultures
Constrained information flow– Information security requirements, the classified nature of specific data, and
different cultures constrain information flow across the enterprisePerformance management resources are constrainedNumerous reporting requirements
– Enterprise is subject to many performance-management related regulations and other compliance requirements
Effectiveness/efficiency of data collection/analysis systems
5
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Key AssumptionsMany government organizations wish to improve
performance– Application of performance management techniques can
improve performance
There is no “one size fits all” solutionPerformance management solutions must be tailored to a
government enterprise’s specific goals and circumstancesThere are cost, risk, and benefit tradeoffs that must be
considered in determining if, when, and how performance management improvements should be undertaken
6
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Purpose of a Performance Management Improvement Decision FrameworkFramework objective:
– Support performance improvement by reducing or eliminating challenges that will hinder the ability of a government enterprise to improve as effectively and efficiently as possible.
7
To achieve this objective, the framework links:– Performance to performance management
– Improvement recommendations to a best-of-breed performance management process
– Improvement recommendations to an enterprise’s specific goals and circumstances
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
8
TailorTailor
DiagnoseDiagnose:Compare
improvement goals to current
performance management
maturity, challenges, and enablers
RecommendRecommend:
Identify customized
recommendations and prioritize next
steps
Key Framework Elements
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
9
Why the Need to “Diagnose”?
Improvement LandscapeImprovement Readiness
Improvement Goals
Performance Management Maturity
Improvement Fertile GroundChange Capacity
Processes, People, Technology
Stakeholder Engagement
Shaped by Regulatory, Political, and Competitive Environment
Past Failures and Successes
Enablers and Constraints
Integration of Supporting Processes
Policies and Regulations
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Your improvementgoals
Enablers and Constraints
Improvement Readiness
Prioritization ofimprovement
recommendations
Performance Management Maturity
Diagnostic Elements
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
11
Category Ad Hoc Repeatable Defined Managed Optimizing
Enterprise Goals and Priorities
Enterprise goals not clearly defined.
Enterprise goals established but not clearly mapped to mission and vision.
Enteprise goals mapped to mission and vision. There is some traceability of strategic goals to some stakeholder needs.
Effectiveness of goals measured and reviewed. Key stakeholder needs are directly linked to strategic goals.
Enteprise goals prioritized, refined and improved based on results. Stakeholder needs are prioritized and directly influence enterprise goals.
MetricsMetrics defined after the fact or ad hoc.
Metrics are defined and measurable but primarily represent what can, rather than what should, be measured.
Metrics are clearly defined, measurable, and accepted. Some metrics can be directly traced to some stakeholder needs.
Effectiveness of metrics measured and reviewed. Key stakeholder needs are directly linked to metrics.
Metrics refined and improved based on effectiveness. Stakeholder needs are prioritized, key stakeholder needs are directly linked to metrics, and metric effectiveness is regularly reassessed by rigorously revisiting stakeholders and their needs.
TargetsPerformance targets not defined.
Performance targets defined.
Performance targets well established and accepted. Some targets are influenced by some stakeholder needs.
Performance targets established based on rigorous benchmarking process. Key stakeholder needs are directly linked to targets.
Performance targets refined and improved based on results. Targets are consistently reassessed based on interaction with stakeholders and a rigorous revisiting of stakeholder needs and priorities.
Strategic Alignment
Metrics not aligned to strategic goals
Metrics loosely aligned to strategic goals
Metrics clearly aligned to strategic goals. Some metrics and strategic goals are influenced by some stakeholder needs.
Metrics measured and reviewed for alignment to strategic goals. Review includes consideration of key stakeholder needs.
Metrics / goals refined and improved based on alignment reviews. Consistent interaction with key stakeholders influences refinement of metrics/goals.
Activity Alignment
Metrics do not directly measure program activities.
Metrics loosely aligned to program activities.
Metrics clearly aligned to program activities. Some activities are influenced by some stakeholder needs.
Metrics measured and reviewed for alignment to program activities. Review includes consideration of key stakeholder needs.
Metrics / program activities refined and improved based on alignment reviews and interaction with key stakeholders and reassessment of stakeholder priorities.
MeasurementMeasurement of metrics is ad hoc and inconsistent.
Measurement of metrics is conducted on a routine basis.
Measurement of metrics is institutionalized. Some efforts are undertaken to assess stakeholder satisfaction.
Measurement effectiveness tracked and measured. Stakeholder satisfaction/needs are consistently evaluated.
Measurement of metrics is constantly refined and improved, with an objective of more directly measuring stakeholder satisfaction.
Leadership
Leadership not actively engaged in performance management.
Leadership routinely engaged in performance management.
Leadership views performance management as standard business practice. Leadership acknowledges the importance of stakeholder needs to performance management.
Leadership applies formalized governance process to measure and monitor effectiveness of performance management efforts. Leadership actively promotes the alignment of stakeholder needs to performance management.
Leadership refines and improves performance management efforts. Leadership proactively promotes the direct engagement with key stakeholders as an intrinsic aspect of performance management.
Infrastructure (IT, personnel, guidance, training)
No infrastructure in place to track and manage performance management efforts.
Infrastructure in place to support performance management efforts.
Infrastructure institutionalized and sustained.
Infrastructure measured for effectiveness and efficiency.
Infrastructure refined and improved based on effectiveness. Centralized performance management knowledge repository.
Performance Management Maturity Model (PM3)
Sample Diagnostic – PM3 In Aggregate
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
12
Sample Diagnostic – PM3 SnapshotCategory Ad Hoc Repeatable Defined Managed Optimizing
Metrics
Metrics defined after the fact or ad hoc.
Metrics are defined and measurable but primarily represent what can, rather than what should, be measured.
Metrics are clearly defined, measurable, and accepted. Some metrics can be directly traced to some stakeholder needs.
Effectiveness of metrics measured and reviewed. Key stakeholder needs are directly linked to metrics.
Metrics refined and improved based on effectiveness. Stakeholder needs are prioritized, key stakeholder needs are directly linked to metrics, and metric effectiveness is regularly reassessed by rigorously revisiting stakeholders and their needs.
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
13
Sample Diagnostic – Enablers/Challenges Factor Group A: Enabling Environment
– Factor 1: Commitment to improvement?– Factor 2: Organizational Politics?– Factor 3: Organizational Stability?– Factor 4: Communication?
Factor Group B: Management Resources– Factor 5: Skills/experience?– Factor 6: Organizational design?– Factor 7: Funds for managing?– Factor 8: Training opportunities?
Factor Group C: Tools/Techniques– Factor 9: Standardized taxonomy?– Factor 10: Application of decision-making techniques/analyses?– Factor 11: Application of management evaluation techniques?– Factor 12: Data collection/sharing systems?
Factor Group D: Processes/Practices– Factor 13: Structured processes for performance management?– Factor 14: Structured processes for performance and mgmt improvement?– Factor 15: Compliance with regulations and policies?– Factor 16: Application of benchmarking for improvement?
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Sample Diagnostic – Enablers/Challenges
Factor Group A:
Enabling Environment
Factor Group B
Management Resources
Factor Group C
Tools/Techniques
Factor Group D:
Processes/Practices
Factor 1
Factor 9
Fac
tor 1
3
More vulnerable to failureLarger Capability Gaps
14
* Adapted from R. Stevens, The MITRE Corp., “Systems Engineering in the Information Age: The Challenge of Mega-Systems”, 12Jan2005Factor 1 Factor 2
Factor 3Factor 4
Facto
r 5
Fac
tor
6
Facto
r 7
Factor 8
Factor 9
Factor 10
Factor 11
Facto
r 12
Fac
tor
13F
acto
r 14
Fact
or 1
5
Factor 16
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
15
Key Framework Elements - RecommendationsImprovement Goals
• Identify “gotchas” to avoid
•Provide detailed guidance on adopting/implementing performance management process steps
• Identify techniques and tools to apply
• Identify how to communicate the need and value of performance management to stakeholders
•Determine how to align performance management efforts with Presidential direction and regulations
Priorities
??
??
??
??
??
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
16
Step 4: Develop process for monitoring and managing performance on an on-going basis
Step 1: Define strategic outcomes
Step 2: Describe current performance
Step 3: Describe the future state
Step 5: Define and implement a periodic performance management process evaluation
Key Framework Elements - RecommendationsPerformance management process
– Steps do indeed “fit most”, but …
Where do I enter this process?
Where should I focus attention and resources?
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Why focus on Performance Management? Strategic Planning
Business/Operations Planning and Management
Financial Management and Budgeting
Cost Management
Risk Management
Transition Planning and Change Management
Investment Decision-Making Processes and Management
Innovation Management
Supply Chain Management
Performance Engineering
Human Capital Management
Enterprise Architecture and Systems Engineering
17
Performance Management
Enterprise Management
Portfolio Management
Program/Project Management
Acquisition Management
Vision and Strategic Outcomes
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Managing Portfolio Performance ImprovementWe are here now
X
http://www.albrightstrategy.com/framework.html
A good road map is dynamic and evolves over time as conditions change and new discoveries are made.
Alternate paths must always be considered.
Roadmap Type Critical Activities
“Know Why” “Know What” “Know How” “To Do”
Technology • Mission analysis• Codify organizing
logic for business processes and infrastructure (e.g., EA “as is”)
• Capabilities analysis
• Performance baselining and benchmarking
• Specify “To Be” environment
• Strategic planning• Root cause
analysis.• Gap analysis
(e.g., SWOT )• Dependencies
analysis• Portfolio analysis
• Functional analysis
• Systems engineering
• Performance analysis and engineering
• Transition planning
• Analysis of Alternatives
• Portfolio analysis and optimization
• Change management
• Investment management
• Portfolio management
• Acquisition management
• Resource management
Process
People
Application or System
Roadmap elements adapted from:
X
We need to be here by xx/yy/zz
18
PfM a key element
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Role of PfM: Managing the Transition
19
FUTURE STATE
TODAY
Goal: Invest in more experimental technologies for greater innovative opportunities; focus more on long-term needs.
This is a “build” slide best viewed in
slide show view
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
20
FUTURE STATE
TODAYEnterprise Goals
Technology
Maturity
Short Term Long Term
Experimental
Mature I
I
I
III
Baseline or Current State: Portfolio dominated by mature technologies and short-term focus
Role of PfM: Managing the TransitionThis is a “build”
slide best viewed in slide show view
I = Investment
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
21
FUTURE STATE
TODAY
Interim State: Portfolio shifting toward longer-term focus, with higher-maturity technology still predominating.
Enterprise Goals
Technology
Maturity
Short Term Long Term
Experimental
Mature
I I
I
I
I
I
Role of PfM: Managing the TransitionThis is a “build”
slide best viewed in slide show view
I = Investment
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
22
FUTURE STATE
TODAY
Enterprise Goals
Technology
Maturity
Short Term Long Term
Experimental
Mature
II
I
I
II
I
End State: Investments concentrated in more experimental technologies supporting long-term enterprise goals.
Role of PfM: Managing the TransitionThis is a “build”
slide best viewed in slide show view
I = Investment
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Time
Resources
Project Phases
1.Start up
2.Development
3.Implementation
4.MaintenanceS
DI
M
Today
Future
Many
Few S
I
D
I M
RISK
Role of PfM: Optimizing Across Multiple Dimensions
Investment timing
Prioritizing varying objectives, investment purposes, and stakeholder needs that may be unrelated or interdependent
RISK
23
Capability Gap #1
Capability Gap #2
Capability Gap #5
Investment AInvestment BInvestment CInvestment DInvestment EInvestment FInvestment GInvestment Z
Stakeholders
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Case Study: Federated R&D EnterpriseAdopt a Performance Management PhilosophyThe portfolio delivers if the enterprise achieves outcomes and fulfills
stakeholder needs within expected costs and schedule– Performance management involves people, processes, and
technology that enable successful enterprise performance and its effective communication to stakeholders
Performance management success drivers:
Link performance measuring/monitoring to stakeholder needs and outcomes
Proactively focus on leading indicators of potential success/failure
Only measure/monitor what is important for characterizing progress in achieving outcomes and fulfilling stakeholder needs
Motivate desired performance by selecting measures that clearly signal what is important to the enterprise
Strike the right balance between qualitative/quantitative measures and among enterprise priorities
24
© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.
Approved for Public Release. Case #09-3713.
Summary
To achieve its goals, the enterprise must – Develop a sound strategy that supports stakeholder needs;– Create a roadmap that outlines the path toward its strategic
goals; and,– Execute that roadmap, through sound application of portfolio
and performance management principles and processes.
A performance management improvement decision framework can help the enterprise to tailor those processes to the organization’s strengths, weaknesses, resource constraints, and organizational enablers.
25