Gauging the ROI of Your Agile Organization Lee Cunningham Director, Enterprise Agile Enablement VersionOne June 2014 Improving
Gauging the ROI of Your Agile Organization Lee Cunningham Director, Enterprise Agile Enablement VersionOne
June 2014
Understanding Improving
© 2014 VersionOne 2
About me…
• Lee Cunningham • Lean-agile practitioner, coach,
advisor • Some traditional hats along the way:
Software Engineer, Development Manager, QA Manager, Release Manager, Project/Program Manager
• USAF • SPC, PMP, PMI-ACP, CSM • [email protected]
© 2014 VersionOne 3
14% ROI standards “well defined”
ROI? Compared to what?
33% ROI standards “not defined” or “poorly-defined”
Standish Group, DARTS 2007
© 2014 VersionOne 4
Is ROI even being measured?
64% “rarely” or “never” perform post-project value evaluation
Standish Group, DARTS 01-2013
© 2014 VersionOne 5
…yet
72% report that their project evaluations are “somewhat accurate”
Standish Group, DARTS 01-2013
© 2014 VersionOne 6
So, what they have told us is…
I HARDLY EVER EVALUATE MY PROJECTS --
BUT WHEN I DO, MY EVALUATIONS ARE SOMEWHAT
ACCURATE.
STAY CLUELESS, MY FRIENDS…
© 2014 VersionOne 7
A word about statistics
http://tylervigen.com/
© 2014 VersionOne 8
The Product Owner?
© 2014 VersionOne 9
Takeaways
• Agile is about business -- not software.
• We need to be able to talk about agile practices in terms that senior organizational leadership would appreciate.
© 2014 VersionOne 10
“Agile Organization”?
For our discussion, we’re defining “Agile Organization” as a system formed around an agile value stream, whether a portfolio, program, or a team.
© 2014 VersionOne 11
Cycle Time & Lead Time
© 2014 VersionOne 12
ROI =
Revenue – [Operating Expense + Total Variable Expenses] Investment
How much we are paid for it How much we are paid for it – How much it cost to make it How much we are paid for it – How much it cost to make it What it cost for the raw materials used to make it
© 2014 VersionOne 13
Why projecting ROI is hard to do
Revenue – [Operating Expense + Total Variable Expenses] Investment
© 2014 VersionOne 14
Challenges to calculating Revenue
• New Revenue: new revenue from new customers
• Incremental Revenue: new revenue from existing customers
• Retained Revenue: revenue from existing customers we would have lost had we not undertaken this
• Operational Efficiencies: reduced costs resulting from having undertaken this
© 2014 VersionOne 15
Basic “Concept to Cash” Value Stream
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
© 2014 VersionOne 16
The essence of throughput economics
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
Until we’ve gotten paid, all we’ve done is spend money.
Frequency and Value
© 2014 VersionOne 17
And we can’t forget….
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
rework -$
Defects -$
© 2014 VersionOne 18
Mapping to ROI components
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
rework -$
Defects -$
Investment Revenue Operating and Variable Expenses
© 2014 VersionOne 19
If we want to increase ROI…
Revenue – [Operating Expense + Total Variable Expenses] Investment
Increase the Numerator
…and/or Decrease the Denominator
© 2014 VersionOne 20
If we want to increase ROI…
Revenue – [Operating Expense + Total Variable Expenses] Investment
Focus here
…and/or Decrease the Denominator
Expenses and Investment can only be reduced so far until Revenue is no longer possible.
Increase the Numerator
Revenue, at least hypothetically, has no ceiling.
© 2014 VersionOne 21
“Revenue? But we don’t sell our software – it’s for
an internal customer…”
© 2014 VersionOne 22
Factors affecting ROI components
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
rework -$
Defects -$
Investment Revenue Operating and Variable Expenses
• Effort expended in ideation • Effort expended in developing Epics, Stories • Quality of Epics, Stories
• Effort expended in analysis & design • Quality of Epics, Stories • Degree of team empowerment • Maturity of engineering practices & discipline • Delivery/deployment capability • Volume of escaped defects • Quality of support process*
• Release frequency • Customer-perceived value • Quality of Epics, Stories • Volume of escaped defects
© 2014 VersionOne 23
What to measure and
where
© 2014 VersionOne 24
What to Measure What that indicates
“Concept to team-ready” cycle time • How much the development, grooming, and maintenance of features and stories is costing us
• How long we’re allowing the value of the features and stories to decay
Story/Feature acceptance rate The effectiveness of the backlog generation and grooming process
What to measure: Investment
© 2014 VersionOne 25
What to Measure What that indicates
“Team-ready-to-Deployed” cycle time • How much the process of taking a team-ready story or feature to the customer is costing us
• How long we’re allowing the value of the features and stories to decay
Escaped Defect rate • How much it is directly costing us to fix the defects
• The opportunity cost of fixing those defects
What to measure: Expenses
© 2014 VersionOne 26
What to Measure What that indicates
“Concept to cash” cycle time (total lead time)
How much the complete ideation-to-payment value stream is costing us
Cycle-time by story/feature value How long the more valuable stories & features are being allowed to decay in value
Release frequency • How long revenue-generating opportunities are being deferred
• How long direct customer feedback is being deferred
Escaped defect rate Risk exposure to retained, incremental, and new revenue
What to measure: Revenue
© 2014 VersionOne 27
Cycle time shows up a lot as something to measure. Why?
Key: Cycle Time
• Revenue
• Inventory (epics, stories in the backlog)
• Feedback
The value of all of these decays with time:
© 2014 VersionOne 28
The best single indicator of system ROI is the trend in lead time
associated with that value stream…
…because all of the lower-level things we measure (e.g., defect
rate, internal cycle times) directly affect lead time.
Key: Cycle Time
© 2014 VersionOne 29
No matter what we might wish, the capability of the system today is what it is.
Key: Measure trends, not snapshots in time
Instead of just asking “Why is it taking us x days to deliver a feature?”
ask “Are we delivering things that are truly valuable to our customers faster now than we were 3 months ago?”
© 2014 VersionOne 30
So how can we improve ROI?
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
Decrease Investment
Increase Revenue
Decrease Operating and Variable Expenses
• Competent and committed Product Ownership • Effective and efficient backlog development and grooming skills
• Trained and Empowered teams • Mature and disciplined agile engineering practices • Robust delivery/deployment capability • Solid definition of “Done” • Robust support process*
• Value focus • Frequent Releases to customer
Decrease Lead Time!
© 2014 VersionOne 31
Suggested reading for thought leadership aggregation
© 2014 VersionOne 32
Q&A
© 2014 VersionOne 33
On Behalf of Everyone at VersionOne…
Thanks!