© 2015 Accenture. All rights reserved. Accenture, its logo, and 'High Performance. Delivered.' are trademarks of Accenture. Agile Day Riga 2015 Alex Birke, May 23, 2015 Essential Metrics for Agile Project Management
Aug 14, 2015
© 2015 Accenture. All rights reserved. Accenture, its logo, and 'High Performance. Delivered.' are trademarks of Accenture.
Agile Day Riga 2015
Alex Birke, May 23, 2015
Essential Metrics for Agile Project Management
Why Metrics?
“they help […] to make decisions” (Eric Ries, Lean Startup)
2© 2015 Accenture. All rights reserved.
What makes a good metric?
As many as required, as less as possible
Leading indicators over trailing indicators
Measure results, not activity
Assess trends, not snapshots
Easy to collect
4© 2015 Accenture. All rights reserved.
But wait!
What about Scrum’s metrics asthe Burndown and the Velocity!
5© 2015 Accenture. All rights reserved.
A Set of Core Metrics for Agile Project management*
Scope Cost
Schedule Productivity
� Story Point Effort Performance Indicator� Scope Volatility
� Release Slippage Risk Indicator
� Sprint BurndownPerformance Indicator
� Release BurnchartPerformance Indicator
Quality
� Sprint Productivity
� Required Productivity
© 2015 Accenture. All rights reserved.
*) Metrics are agnostic of technology or domain where Agile Project Management is used
� Running Tested Features
� Delivered Defect Rate
Sprint Productivity (P)
Definition
Productivity is measured as generated value in story points that can be completed per person day.
Calculation
# story points [SP] of a sprint
P =
# person days of a sprint
8© 2015 Accenture. All rights reserved.
Required Productivity (RP)
Definition
RP is the productivity that the team would require in the remaining sprints, to complete at least remaining “Must-Have” user stories, so that a Minimal Viable Release (MVR) can be deployed to production.
Calculation
# of SP estimate of remaining “Must-Have” stories
RP =
Total planned effort in remaining sprints
9© 2015 Accenture. All rights reserved.
Required Productivity
10© 2015 Accenture. All rights reserved.
Required Velocity =
(3 [SP] + 3 [SP] +8 [SP] + 3 [SP]
) / 3
= 5.66 [SP]
Assuming 105 hours per upcoming Sprint:
RP = 5.66 [SP] / 105 [hours] = 0.054 [SP / hours]
Release Slippage Risk Indicator (RSRI)
Definition
RSRI indicates whether at least a minimal viable release (MVR) can be deployed to production on the scheduled date.
Calculation
“Past” Sprint Productivity (P)
RSRI =
Required Sprint Productivity for release date (RP)
= 1 : as planned
RSRI < 1 : delayed
> 1 : ahead of plan
11© 2015 Accenture. All rights reserved.
Release Slippage Risk Indicator
12© 2015 Accenture. All rights reserved.
Required Productivity = ( 3 [SP] + 3 [SP] + 8 [SP] + 3 [SP] ) / (3 [Sprints] * 105 [hours] ) =
= 0.054 [SP / hours]
Past Productivity = 5 [SP] / 90 [hours] = 0.055 [SP / hours]
Release Slippage Risk = 1.02
Release BurnChart Performance Indicator (RBPI)
Definition
RBPI indicates if a project is on schedule by showing the variation in the amount of completed work compared to amount of work planned.
Calculation
Story points fully completed so far in the release
RBPI =
Story points planned so far in the release
= 1 : as planned
RBPI < 1 : delayed
> 1 : ahead of plan
13© 2015 Accenture. All rights reserved.
0 20 60160
280400
560
720820
9201015
11101210
1330
490
0
200
400
600
800
1000
1200
1400
1600
1800
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Wo
rk (
Sto
ry P
oin
ts)
Sprint #
Release Burn Up Chart
Must-Have Scope Must-Have + Should-Have ScopePlanned Dev Complete or Accenture Scope CompleteIdeal Burn-Up Required Burn
2 new “Billing” epics added“Summary Reports” added
“Rating” epics de-scoped
In the example above, at the end of Sprint 7: RBPI = 490 / 720 = 0.68,
which indicates that the amount of user stories done was less than expected.
Release BurnChart Performance Indicator
© 2015 Accenture. All rights reserved.
Sprint Burndown Performance Indicator (SBPI)
Definition
SBPI shows the deviation in completed work compared to work planned, for the current sprint.
Calculation
Ideal remaining work
SBPI =
Actual remaining work
= 1: as planned
SBPI < 1: delayed
> 1: ahead of plan
15© 2015 Accenture. All rights reserved.
Sprint Burndown Performance Indicator = 120 [hours] / 160 [hours] = .75
(i.e. -25% deviation)
Sprint Burndown Performance Indicator= 40 [SP] / 50 [SP] = .80
Fig. 1: Task Burndown (effort based) Fig. 2: User Story Burndown (Story Point based)
Sprint Burndown Performance Indicator
© 2015 Accenture. All rights reserved.
Scope Volatility (SV)
Definition
Scope Volatility depicts the amount of change in size of the release scope, comparing the release scope size measured at start of the release and after the last completed sprint.
Calculation
Current Size of Must-Have Scope – Initial Size of Must-Have Scope
SV = * 100
Initial Size of Must-Have Scope
> 0: Scope creep
SV < 0: Scope drop
= 0: Planned scope size retained (often corridor)
17© 2015 Accenture. All rights reserved.
11431228
Must-Have Scope 1237
1273 1330
0
200
400
600
800
1000
1200
1400
1600
1800
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Wo
rk (
Sto
ry P
oin
ts)
Sprint #
Release Burn Up Chart
Must-Have Scope Must-Have + Should-Have ScopePlanned Dev Complete or Accenture Scope CompleteIdeal Burn-Up Required Burn
2 new “Billing” epics added
“Rating” epics de-scoped
In the example above, Scope Volatility = (1273 [SP] – 1143 [SP] ) / 1143 [SP] * 100 = .11
Scope Volatility
© 2015 Accenture. All rights reserved.
Story Point Effort Performance Indicator (SPEPI)
Definition
SPEPI (aka CPI, sprint-wise) indicates if the ongoing project release is currently on budget, depicting deviation in planned effort per story point to the actual effort per story point.
Calculation
Planned Effort to be spent in the release so far
Planned Story points delivered in the release so far
SPEPI =
Actual Effort spent on the release so far
Actual Story points delivered in the release so far
= 1 : as planned
SPEPI < 1 : cost overrun
> 1 : under budget
19© 2015 Accenture. All rights reserved.
Story Point Effort Performance Indicator
“per”
20
Value
Effort / money
SPEPI = (875 / 630 ) / ( 957 / 490 ) = 0.71, which indicates a cost overrun.
© 2015 Accenture. All rights reserved.
Running Tested Features* (RTF)
Definition
RTF depicts the variance of working (running) features over total features built to date.
Calculation
# completed user stories that still pass all acceptance tests
RTF = x 100
total # of completed user stories to date
21© 2015 Accenture. All rights reserved.
*) “A Metric leading to Agility”, Ron Jeffries
Running Tested Features
22© 2015 Accenture. All rights reserved.
Sprint 6 RTF rate was 71%.
This indicates that the remaining 29% of the user stories built and tested are not working and cannot be deployed to production.
Delivered Defect Rate (DDR)
Definition
DDR indicates the effectiveness of the review and testing activities, thus ensuring that fewer defects are identified on the delivered product (increment).
Calculation
Defects identified after Done-ness of user stories
DDR =
Total effort spent in all tasks till date
Alternative metric: Defect Density is # of defects identified in completed user stories per size of Done user stories.
23© 2015 Accenture. All rights reserved.
Delivered Defect Rate
24© 2015 Accenture. All rights reserved.
DDR Sprint 1 Sprint 2 Sprint 3 Sprint 4
Defects Identified 7 12 19 25
SPE Effort 400 831 1262 1685
Defect Rate 0,018 0,014 0,015 0,015
Engineering effort includes effort from Agile Lifecycle tool for design, build, test, defect fix tasks. Includes PO & SM time as % of completed stories
Count only the defects logged after the story is marked complete by the developer
In the example above, DDR trend is stable. Therefore the delivered quality is stable.
A Set of Core Metrics for Agile Project management*
Scope Cost
Schedule Productivity
� Story Point Effort Performance Indicator� Scope Volatility
� Release Slippage Risk Indicator
� Sprint BurndownPerformance Indicator
� Release BurnchartPerformance Indicator
Quality
� Sprint Productivity
� Required Productivity
© 2015 Accenture. All rights reserved.
*) Metrics are agnostic of technology or domain where Agile Project Management is used
� Running Tested Features
� Delivered Defect Rate
Some further sometimes helpful metrics
26
% Stories accepted
Stakeholder Involvement Index
% Changed Scope
Average Time to Market
(Retrospective) Process improvement
Customer Satisfaction
Test Automation
Coverage
Employee Engagement
Epic Progress Report
Questions & Answers
27© 2015 Accenture. All rights reserved.