Measuring Scrum

Scrum Metrics for Hyper-Productive Teams:

How They Fly like Fighter Aircraft

By Jeff Sutherland and Scott Downey

Scrum Metrics for Hyper-Productive Teams:

How They Fly like Fighter Aircraft

By Jeff Sutherland and Scott Downey

What’s AheadGoal

Tuning the Scrum Meetings

Metrics: Math and Purpose

The RoboScrum Workbook

What’s AheadGoal


Metrics: Math and Purpose

The RoboScrum Workbook

Jeff SutherlandChairman, Scrum Training Institute

CEO Scrum, Inc. & Sr. Advisor to OpenView Venture Partners

Agile Coach for OpenView Venture Partners portfolio companies

CEO/CTO/VPE for 11 Software Companies Created first Scrum at Easel Corp. in 1993 Rolled out Scrum in the next 7 Companies Achieved Hyper-Productive State in All

Companies.

Signatory of the Agile Manifesto

Founder of the Agile Alliance

http://[email protected]

Jeff SutherlandChairman, Scrum Training Institute

CEO Scrum, Inc. & Sr. Advisor to OpenView Venture Partners

Agile Coach for OpenView Venture Partners portfolio companies

CEO/CTO/VPE for 11 Software Companies Created first Scrum at Easel Corp. in 1993 Rolled out Scrum in the next 7 Companies Achieved Hyper-Productive State in All

Companies.

Signatory of the Agile Manifesto

Founder of the Agile Alliance


Scott Downey

Head Agile Coach, Napster

Formerly Head Agile Coach, MySpace 68 Delivery Teams

Created Immersive Scrum Training System for Rapid Bootstrapping of Hyper-Productive Teams

Average Improvement was over 600%

Average Boot-Up took only 4.5 Weeks

Co-Teaching CSM with Jeff since 2007

Shock Therapy: A Bootstrap for Hyper-Productive Scrum at Agile ‘09 with Jeff and Björn Granvik


Scott Downey

Head Agile Coach, Napster

Formerly Head Agile Coach, MySpace MySpace had 68 Delivery Teams

Created Immersive Scrum Training System for Rapid Bootstrapping of Hyper-Productive Teams

Average Improvement was over 600%

Average Boot-Up took only 4.5 Weeks

Co-Teaching CSM with Jeff since 2007

Shock Therapy: A Bootstrap for Hyper-Productive Scrum at Agile ‘09 with Jeff and Björn Granvik


GoalTo develop and standardize a set of

Minimally-Invasive Metrics

and a Fully Portable Language for team comparison across an Enterprise.

that will help Scrum Masters

Evaluate and Advise Teamswhile providing Rich Insights about Team performance

INVEST

INVEST Mnemonic Refresher

mmediately Actionable

egotiable

aluable

stimable

ized to Fit

estable

Thanks to Bill Wake for the acronym. See www.xp123.com.

Creating a Culture of Hyper-Productivity


Creating a Culture of Hyper-Productivity



Sprint Planning Meetings Anchoring the Scale A New Perspective on INVEST Value, Voice and Visibility

Daily Stand-Up Meetings The Fourth Question

Tuning the Scrum MeetingsSprint Planning Meetings

Anchoring the Scale A New Perspective on INVEST Value, Voice and Visibility

Daily Stand-Up Meetings The Fourth Question

Sprint Planning Meetings

Keystone: A central cohesive source of support and stability;The architectural piece that locks other pieces into position; That which provides structural integrity.

Sprint Planning MeetingsTuning the Scrum Meetings

Priority 1


Priority 2

Priority 3

DeliveryTeam

SPO

That’s the most importan

t.

Priority 2

Priority 1

DeliveryTeam


Priority 3

SPODelivery

Team

That’s the

easiest.

Priority 1

Priority 2

Priority 3

DeliveryTeam


SPO

Scrum

Master

That’s a 3.

3

Priority 1


SPO

Priority 2

Priority 3

3

DeliveryTeam

DeliveryTeam


SPO

Priority 23

Priority 3

Priority 1

385


3

Sprint Planning MeetingsTuning the Scrum Meetings100

%66%33%

167%

267%

433%

1 2 3 5 8 13


PBL SBL Bank

1 2 3 5 8 13 21 34 55

215133

8

100%

66%33%167%

267%

433%

1 2 3 5 8 13 34 55

SPO

DeliveryTeam

Do you Approve?

Yes.Do you

Approve?Yes.

Do you Approve?

No.

Do you Approve?

Yes.Do you

Approve?Yes.

!@#$%^&


PBL SBL Bank

1 2 3 5 8 13 21 34 55

13

I N V E S T

100%

66%33%167%

267%

433%

8

3

5

1 2 3 5 8 13 21 34 55

SPO

DeliveryTeam

Yes.Can you do this?

Yes.Can you do this? No, it’s not

Actionable.

Can you do this?

Yes.Can you do this?

5

3

8

#$%@!#$

21

700%

Sprint Planning MeetingsPro

du

ct B

ack

log

Team’sCommitment

Pla

nnin

g M

eetin

g


AutomaticTasking

Sprint Planning Meetings

Pla

nnin

g M

eetin

g


Goal: Accept the LARGESTchunk of work that passesthe INVEST mnemonic, thenstop digesting it and get it done.

Spr

int B

ackl

og

Our Amazing Product

API Work

Front End Work

Value, Voice and Visibility

Database Work

Technologists’Perspective

Database Work

API Work

Front End Work

Value, Voice and VisibilityCustomer’s Perspective

Featu

re 1

Feature 2

Feature 3


Value, Voice and VisibilityUser Story 1

User Story 2


User Story 2


User Story 2

User Story 2.1

User Story 2.2User Story 2

User Story 1

Value, Voice and VisibilityCustomer’

s Perspectiv

e

User Story 1

User Story 2.1


User Story 2.2

User Story 1

User Story 2.1


Product Backlog Sprint BacklogUser Story 2.2

SPO

SM

Info

rmati

on R

adia

tor

The Fourth Question

SPO

SM

Info

rmati

on R

adia

tor

The Fourth Question3

2. Compared to our Keystone, how many Story Points did WE Achieve yesterday on Priority 1?

2. What will WE Achieve Today on Priority 1?

3. Is anything blocking or jeopardizing OUR progress on Priority 1?

1. What did you Achieve Yesterday on Priority 1?

2. What will you Achieve Today on Priority 1?

3. Is anything blocking or jeopardizing your progress on Priority 1?

1. What did WE Achieve Yesterday on Priority 1?

3. What will WE Achieve Today on Priority 1?

4. Is anything blocking or jeopardizing OUR progress on Priority 1?

SPO

SM

Info

rmati

on R

adia

tor

The Fourth Question3

MetricsThe Math and Purpose of Each

MetricsThe Math and Purpose of Each

Metrics

1.Velocity2.Work Capacity3.Focus Factor4.Adopted Work5.Found Work6.Targeted Value Increase7.Accuracy of Estimation8.Accuracy of Commit

1:1

1:1

Metrics: VelocityI, as a… Scrum Product Owner who is trying to create

an accurate Roadmap for future releases

…need… a reliable metric on which to base my assumptions about the rate of the team’s progress

…so that… I, with our Leadership, can make well-informed tradeoffs and commitments based on the reality of our company’s capabilities.

Formula: ∑Original Estimates of All Approved Cards

1:1

Priority 18 Original Estimate: 8 Story Points

Original Estimate: 5 Story Points


Estimate 8

3

5

11

Velocity =

SPO

Metrics: Velocity

Priority 23

Priority 3

5

Metrics: Work CapacityI, as a… Scrum Master who is trying to coach a Team

toward Hyper-Productivity

…need… a way to measure how much work the Team can do in a given Sprint, whether it results in an Approved Card or not,

…so that… I can quantify the Team’s entire capability, ask

intelligent questions about distractions and take action to optimize the conversion of Effort into Value.

Formula: ∑All Work Reported During the Sprint

1:1

Priority 23

Priority 3

5

Metrics: Work Capacity

What have we achieved today on Priority 1?

Item 1Item 2

321 5 813

Priority 18




Work Reported : 8 Story Points

Work Reported: 2 Story Points

Work Reported : 7 Story Points

EstimateReported

8

32

57

11 17

Velocity=Work

Capacity=

SPO8

Metrics: Work Capacity

Priority 23

Priority 3

5

Priority 18

Priority 23

Priority 3

5

Velocity & Work Capacity

20

10

08

18

20

10

08

25

20

10

09

01

20

10

09

08

20

10

09

15

20

10

09

22

20

10

09

29

0

5

10

15

20

25

30

35

40

Hyper-Productivity Begins Original Velocity Velocity Work Capacity

Sprint Name

Sto

ry P

oin

ts

Metrics: Focus FactorI, as a… a member of Leadership who is not a member

of a Scrum Delivery Team

…need… a way to measure how much of each Team’s

bandwidth results in deployable product, in a

cross-team comparable way,

…so that… I can actively help sub-optimized Teams,

intelligently allocate resources and reward

our Teams for their hard work.

Formula: Velocity ÷ Work Capacity

1:1

WorkCapacityVelocity

11 17

64.7%

Metrics: Focus Factor

W. Edwards Deming

Attempts to force

non-deterministic

systems

to operate at greater

than

80% efficiency will

cause

short bursts of

stabilization

followed by extreme

periods

of destructive and

unpredictable

variations

from that goal.

Focus Factor

20

10

08

18

20

10

08

25

20

10

09

01

20

10

09

08

20

10

09

15

20

10

09

22

20

10

09

29

0%

20%

40%

60%

80%

100%

Sprint Name

Velc

ocit

y ÷

Work

Capacit

y

Metrics: Adopted WorkI, as a… a Scrum Master, who is trying to coach a team

toward more accurate Commitments during each Sprint Planning Meeting,

…need… a metric that clearly shows if the Team has a tendency to under Commit and is consistently having to pull work forward from the Product Backlog before the end of the Sprint

…so that… I can encourage the Team toward higher Commitments during the Sprint Planning Meetings without the risk of pushing them to failure.

Formula: ∑Original Estimates for Work Pulled Forward

Original Commitment 1:1

Metrics: Adopted Work

3

1

5

2

8135

8

3

5

PBLOriginal Commitment:

Adopted Work:

Total Commitment:

0

0

0

3

0

3

4

0

4

9

0

9

11

0

11

19

0

19

19

13

32

SBL In Prog

DONE

Adopted Work = 13 ÷ 19 = 68.4%

Metrics: Adopted Work

Metrics: Found WorkI, as a… a Scrum Master, who is trying to help my Team make

more accurate and reliable Commitments in Sprint Planning,

…need… a clear way to measure the likelihood of unexpected work based on a Card’s Original Estimate

…so that… I can offer advice to the Team on making achievable Commitments and provide them fair warning when they start to accept a Card that will probably

surprise them.

Formula: ∑Total Work Reported per Card – Original Estimate

Original Commitment1:1

3

Found Work: 3

Metrics: Found Work

3

1

5

2

8135

8

3

5

PBL SBL In Prog

DONE

Total Commitment: 19

Original Commitment:19


Metrics: Found + Adopted Work

-- -- -- 20100818 20100825 20100901 20100908 20100915 20100922 201009290.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

140.00%

Win Win Win Win Win Win Win Win Win WinLoss Loss Loss Loss Loss Loss Loss Loss Loss Loss0.00% 0.00% 0.00%24.14%

43.24% 35.00% 34.62%

89.66%100.00%

64.86%

0.00% 0.00% 0.00%

37.93%

75.00%

25.00%

100.00%

19.23%13.64%

31.03%

Win/Loss by SprintShowing Last 10 Sprints. To Win, the

Team must achieve 80%+ of their Commit and Total Surprise Work can

be 20% at most.

Achievement Total Surprise

Sprint Name

Perc

enta

ge (

Based o

n O

rigin

al C

om

mit

ment

for

Each S

pri

nt)

Metrics: Targeted Value IncreaseI, as a… a Scrum Product Owner who is trying to evaluate the

efficacy of the product directions I have chosen

…need… a reliable way to measure the increased value contribution of the Team sprint-over-sprint

…so that… I can compare the Team’s rate of value contribution

increase to the changes in revenue we are generating and adjust our direction if the value

that I estimated isn’t being realized.

Formula: Current Sprint’s Velocity ÷ Original Velocity

1:1

Metrics: Targeted Value IncreaseSprint Number Velocity

Original Velocity

Current Velocity

2010.05.05 8 2010.12.05 13 2010.19.05 21 2010.26.05 37 2010.26.05 42

TVI: 8 ÷ 8= 100%13 = 162.5%21 = 237.5%33 = 412.5%42 = 525%

Metrics: Targeted Value Increase

20

10

08

18

20

10

08

25

20

10

09

01

20

10

09

08

20

10

09

15

20

10

09

22

20

10

09

29

0

5

10

15

20

25

30

35

40

Velocity vs. Work Capacity Trend

Hyper-Productivity Begins Original Velocity Velocity Work Capacity

Sprint Name

Sto

ry P

oin

ts

Metrics: Accuracy of EstimationI, as a… a Scrum Product Owner who is interested in creating

reliable roadmaps, including Optimistic, Likely and

Pessimistic release dates for larger initiatives,

…need… a metric that tracks the margin of error on the Team’s Original Estimates

…so that… I can multiply their good-faith estimates by this

factor and create more realistic date projections.

Formula: 1 – (Estimate Delta ÷ Total Commit)

1:1

3

1

35

2

8

SBL In Prog

DONE

Found Work: 3

Metrics: Accuracy of Estimation

135

8

3

5

PBLOriginal Commitment:19


35

2

8

3

1

SBL In Prog

DONE

Found Work: 3


135

8

3

5

PBLOriginal Commitment:19


= 0.4090- 0.4090

=

35

2

8

3

1

SBL In Prog

DONE

Found Work: 3

Metrics: Accuracy of Estimation Original Commitment:19


Estimate Delta: 9Total Commitment: 22

Actual2

2

8

5

7

Delta

1

1

3

3

1

1 0.59159%We want

about

80%!


35

2

8

3

1

DONE

Actual2

2

8

5

7

Delta

1

1

3

3

1


18 19 20 21 22 23 24 25 26 27

…3 5 8 13 21…

34 5 67 8 9 1011 12 13 14 15 16 17… …

Metrics: Accuracy of CommitI, as a… a Scrum Product Owner, who is concerned about the

accuracy of my Roadmaps

…need… a metric that informs me of the margin of error when

the Team commits to a body of work

…so that… I can use this margin of error to predict reliable

dates, and know when it is safe to lobby for a higher

Commitment at each Planning Meeting.

Formula: (∑Original Estimates) ÷ (∑Original Estimates + ∑Adopted Work + ∑Found

Work)

1:1

Metrics: Accuracy

20100818 20100825 20100901 20100908 20100915 20100922 201009290%

20%

40%

60%

80%

100%

Accuracy of Commitment

Sprint Name

Com

mit

ment

÷ A

ctu

al D

e-

livery

20100818 20100825 20100901 20100908 20100915 20100922 201009290%

20%

40%

60%

80%

100%

Accuracy of Estimation

Sprint NameOri

gin

al Esti

mate

s ÷

Actu

al

Eff

ort

RoboScrum(a.k.a. “The Awesome Spreadsheet of

Awesomeness”)

(a.k.a. “The Awesome Spreadsheet of Awesomeness”)

RoboScrum

RoboScrumRoboScrum gives you grounds

for QUESTIONS but not for ACCUSATIONS!!

Not an Excel Expert (Google + F1)

Offered as an Example Only

Do Not Guarantee Perfection

Constantly Evolving during Daily Use

Fully Acknowledge Improvements can (and will!) be Made

No Macros (for Ease of Portability)

Use at Your Own Risk

No Support Contract is either Expressed or Implied

RoboScrum

RapidScrum.com

Password: rapidscrum

Measuring Scrum

Business

scrum meetingstuning

scrum meetingspriority

scrum meetingsspopriority

scrum meetingscreating

scrum meetings100

scrum masters

hyperproductive scrum

scrum meetings metrics