Kanban Metrics in practice for leading Continuous Improvement

for leadingContinuous Improvement

Kanban Metricsin practice

@BattistonMattia

About me

● from Verona, Italy

● software dev & continuous improvement

● Kanban, Lean, Agile “helper”

● Sky Network Services

Mattia Battiston@BattistonMattia

mattia.battiston@gmail.com

Ciao!

Why are we here?

OUR EXPERIENCE

WHY

HOW

IMPROVING

LESSONS LEARNT

FORECASTING

Kan...what?

a little knowledge of Kanban helps(limiting WIP, lead time, value vs waste, queues, batches, etc.)

Why do we need metrics?

#1: drive continuous improvement #2: forecast the future

But I thought metrics were bad....

Typical problems:gaming

dysfunctions

Good vs Bad metrics

● look at improving the whole system ● reward/punish individuals

“95% performance is attributable to the system, 5% to the people”

W. Edwards Deming

● feedback about state of reality ● used as target

● leading (let you change behaviour) ● lagging (tell you about the past)

● all metrics must improve ● local optimisations

Our system

Iteration-Based

On-demand

Direct

How do we collect the data?

The SpreadsheetInputs: story details; start time and duration of each state

Public version: https://goo.gl/0A9QSN

For you to copy, reuse, get inspired, etc.

All the maths you need

● Min, Max

Normal: data is distributed around a central valuee.g. height of UK population

Skewed: data has a long tail on one side (positive or negative)e.g. income of UK population (positive skew)Lead time of stories follows skewed distribution

● Average (mean)avg(1,2,2,2,3,14) = (1+2+2+2+3+14)/6 = 4

● Median: separates the high half from the low half. Less impacted by outliersmedian(1,2,2,2,3,14) = 2

● Mode: value that occurs more frequentlymode(1,2,2,2,3,14) = 2

● Standard Deviation: measures the amount of dispersion from the average. When high, values are spread over a large range.

stdev(1,2,2,2,3,14) = 4.5; stdev(1,2,2,2,3,5) = 1.2;● Percentile: percentage of elements that fall within a range

50% perc(1,2,2,3,7,8,14) = 3; 80% perc(1,2,2,3,7,8,14) = 7.8;

● Normal Distribution vs Skewed Distribution:

Cumulative Flow DiagramDescription: Each day shows how many stories are in each state

n. s

torie

s

days

Cumulative Flow DiagramIdeal CFD: thin lines growing in parallel at a steady rate -> good flow!

Cumulative Flow Diagram● Objective: retrospect (but needs a good facilitator)

CFD used for Retrospective

● Objective: demonstrate effectiveness of changes

changed WIP limit in DEV from 3 to 2

Cumulative Flow Diagram

● Objective: decide what you should work on today● Objective: forecasting: rough info about lead time, wip, delivery date (although

they’re easier to use when tracked separately)

WIP

Lead Time

Throughput

Delivery Date

CFD Patterns

(taken from CFD article by Pawel Brodzinski)

growing lines: indicate large WIP + context switching. action: use WIP limits

stairs: indicates large batches and timeboxesaction: move towards flow (lower WIP,

more releases, cross-functional people)

flat lines: nothing’s moving on the boardaction: investigate blockers, focus on finishing, split in

smaller stories

single flat line: testing bottleneckaction: investigate blockers, pair with testers,

automate more

typical timeboxed iterationdropping lines: items going backaction: improve policies

metrics forDelivery

Time

Control ChartDescription: For each story it shows how long it took. Displays Upper and Lower control limits; when a story falls out of limits something went wrong and you should talk about it.

stories

lead

tim

e (d

ays)

Cycle/Lead Time stats + HistoryDescription: Stats to get to know your cycle time and lead time. They let you predict “how long is the next story likely to take?”. Visualize trends of improvement

Lead Time distribution

lead time (days)

n. s

torie

s th

at t

ook

that

long

Description: For each lead time bucket (#days), how many stories have taken that long.Useful to show as a percentage to know probability.

WEIBULL DISTRIBUTION

50%

80%

Story HealthDescription: Indicates if the story is in good health or if we should worry about it. Based on lead time distribution

50-80% >90%80-90%0-50%0-4 gg 5-7 gg 8-10 gg >10 gg

Cycle Time vs Release Prep. Time

stories

days

Description: For each story shows how long it spent in the iteration and in release preparation (Context specific). Used to discuss cost vs value of release testing.

metrics forPredictability

Iteration Throughput

iteration

no. s

torie

s co

mpl

eted

Description: Number of stories that get done for each iteration

Rolling Wave ForecastingDescription: visualise in the backlog the likelihood of stories getting done in the next 2 weeks

Arrivals RateDescription: how often a story is started, aka pulled into our system (arrival). This is how often you can change your mind about what to do next

Points vs Lead Timele

ad t

ime

(day

s)

story points

Description: Shows low correlation between estimated points and actual lead time

Disney StationsDescription: Like queueing at Disneyland. “How long in here? How long from here?”

Task TimeDescription: Shows how long tasks usually take (context specific). Gives you an idea of how long a story will take based on n. of tasks

metrics forQuality

Bugs percentageDescription: Percentage of bugs over stories. Also expressed as “1 bug every X stories”

metrics forContinuous

Improvement

Flow EfficiencyDescription: Shows how long stories have spent in queues - nobody was working on them. Shows how much you could improve if you removed waiting time.

Time in status

time

spen

t in

sta

te (

days

)

story

Description: for each story visualise how long it spent in each status (absolute and percentage). Shows trends of where stories spend more time

Retrospective

ResourcesBooks

Metrics● Data driven coaching - Troy Magennis● Seven Deadly Sins of Agile Measurement - Larry Maccherone● The Impact of Lean and Agile Quantified - Larry Maccherone● Kanban at Scale: A Siemens Success Story - Bennet Vallet● FocusedObjectives@Github - Troy Magennis● Visual feedback brings key Agile principles to life - Bazil Arde

n● How visualisation improves Psychological Safety - Bazil Arden

Forecasting● Cycle Time Analytics - Troy Magennis● Top Ten Data and Forecasting Tips - Troy Magennis● Forecasting Your Oranges - Dan Brown● Using Maths to work out Potentially Deliverable Scope - Ba

zil Arden● Forecasting Cards - Alexei Zheglov

Story Points● Story Points and Velocity: The Good Bits - Pawel Brodzi

nski● No correlation between estimated size and actual time

taken - Ian Carroll

Lead Time● Analyzing the Lead Time Distribution Chart - Alexei Zheglov● Inside a Lead Time Distribution - Alexei Zheglov● Lead Time: what we know about it, how we use it - Alexei Z

heglov● The Economic Impact of Software Development Process Cho

ice - Troy Magennis

More● Flow Efficiency - Julia Wester● Cumulative Flow Diagram - Pawel Brodzinski

Thank you!

@BattistonMattia

mattia.battiston@gmail.com

really, really appreciated! Help me improve