10/30/2013 1 Michael Mah Managing Partner QSM Associates, Inc. 75 South Church Street Pittsfield, MA USA 01201 413-499-0988 Fax 413-447-7322 e-mail: [email protected]Website: www.qsma.com Blog: www.optimalfriction.com When Agile Becomes a Quality Game Changer; What Data Says from Recent Agile Benchmark Research (#2)
32
Embed
When Agile Becomes a Quality Game Changer; What Data Says ...uploads.pnsqc.org/2013/slides/key2_Mah_slides.pdf · industries, waterfall, Agile, offshore/outsourced, in-house, new
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
“Agile projects can be considered more successful in the sense that they deliver more functionality with fewer defects.”
- Kent Beck
(#4)
10/30/2013
3
(#5)
(#6)
10/30/2013
4
(#7)
Rayleigh Curve Defect Rate
(#8)
10/30/2013
5
(#9)
(#10)
Background
“We don’t need no stinking metrics” - Jim Highsmith
“Without metrics, you’re just someone with another opinion” - Michael Mah
10/30/2013
6
(#11)
(#12)
10/30/2013
7
(#13)
The QSM SLIM Database
QSM maintains the world’s largest benchmarking database of 10,000+ completed software projects collected worldwide. We put industry productivity statistics on the desktop.
The QSM SLIM database contains projects in all industries, waterfall, Agile, offshore/outsourced, in-house, new development, and maintenance.
SLIM tools enable managers to measure and estimate Agile and/or waterfall projects, and determine ROI.
(#14)
British Telecom
SAP
Microsoft
Intel
AT&T/BellSouth
Nationwide
Motorola
VerizonWireless
Roche Diagnostics
Fiserv Corp
IBM Global
Misys Healthcare
JPMorganChase
Boeing
Bank of New York Mellon
Lockheed Martin
Progressive Insurance
DirecTV
The QSM SLIM Database
10/30/2013
8
(#15)
(#16)
10/30/2013
9
(#17)
(#18)
10/30/2013
10
(#19)
(#20)
Agile Metrics Capture – Velocity etc.
10/30/2013
11
(#21)
Agile vs Waterfall - Schedule
C&T Duration (Months) vs Effective SLOC
10 100 1000
Effective SLOC (thousands)
1
10
100
C&
T D
uration (Months)
Agile 1
Agile 3
Agile 2
Traditional 1
Traditional 2
Agile 1
Agile 3
Agile 2
Traditional 1
Traditional 2
All Sy stems Special Project QSM 2002 Scientific Av g. Line Sty le 1 Sigma Line Sty le
Elan 3.2 Release
6 mos
Industry Average
10 mos
FASTER
(#22)
Agile vs Waterfall - Quality
FEWER BUGS
10/30/2013
12
(#23)
This Data Says: Kent was CORRECT
“Agile projects can be considered more successful in the sense that they deliver more functionality with fewer defects.”
- Kent Beck
(#24)
10/30/2013
13
(#25)
Case Study: Co-Located XP - Follett Software
Team size
24 Developers 7 Testers 3 Customers 3 Project
Leaders Code Base
1,000,000 lines of code
7,000 automated unit test
10,000 automated acceptance test
(#26)
10/30/2013
14
(#27)
(#28)
10/30/2013
15
(#29)
Project Sketch – Core Metrics
(#30)
Input to SLIM
Size
Time
Defects
Effort
10/30/2013
16
(#31)
Industry
Average
Current
Performance
Delta
Project Cost
$3.5 Million
$2.2 Million
-$1.3M
Schedule
12.6 months
7.8 months
-4.8 mos
QA Defects
242
121
-50%
Staffing
35
35
n/a
Follett vs. Industry Average
(#32)
Distributed SCRUM – BMC Software
10/30/2013
17
(#33)
Project Sketch – Core Metrics
(#34)
Industry
Average
Current
Performance
Delta
Project Cost
$5.5 Million
$5.2 Million
-$.3M
Schedule
15 months
6.3 months
-8.7 mos
QA Defects
713
635
-11%
Staffing
40
92
+52
BMC vs. Industry Average
10/30/2013
18
(#35)
Agile Assessment — Schedule
SCHEDULE
10 100 1,000
PROJECT SIZE (thousands)
1
10
100
C&
T D
ura
tion (M
onth
s)
Agile Companies Company B SCRUM Company A XP QSM 2005 Business Avg. Line Style 1 Sigma Line Style
Faster Schedules
(#36)
Agile Assessment – Quality
BUGS
10 100 1,000
Effective SLOC (thousands)
1
10
100
1,000
10,000
Erro
rs (S
ysIn
t-Del)
Agile Companies Company B SCRUM Company A XP QSM 2005 Business Avg. Line Style 1 Sigma Line Style
Fewer Defects
10/30/2013
19
(#37)
This Data also Says: Kent was CORRECT
“Agile projects can be considered more successful in the sense that they deliver more functionality with fewer defects.”
- Kent Beck
(#38)
10/30/2013
20
(#39)
The Columbus Agile Benchmark Study
(Columbus vs the World)
(#40) footer
SLIM-
Estimate:
Size, Schedule,
Cost & Quality
Estimating
SLIM-DataManager
Software Project Metrics Repository
SLIM-Control:
Variance
Analysis
&
Adaptive
Forecasting SLIM-Metrics:
Industry
Benchmarking
& Process
Improvement
SLIM-MasterPlan: Incremental
Development & Project
Aggregation
10/30/2013
21
(#41)
(#42)
10/30/2013
22
(#43)
(#44)
10/30/2013
23
(#45)
(#46)
Agile Captures the Right Metrics for SLIM
Velocity/Burndown
Headcount
Stories and Point Sizing
Bugs
10/30/2013
24
(#47)
(#48)
10/30/2013
25
(#49)
Speed
Time-to-Market
1 10 100 1,000
New + Modified Size (thousands)
1
10
100
Month
s
All Sy stems QSM Business Av g. Line Sty le 1 Sigma Line Sty le
Faster Schedules
(#50)
10/30/2013
26
(#51)
Bugs
Bugs During QA
1 10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Defe
cts
All Sy stems QSM Business Av g. Line Sty le 1 Sigma Line Sty le
Fewer Defects
(#52)
Short Feedback Loops
Paired programmers
Instantaneous code reviews
Accelerated learning and execution
Face to face communication channel
10/30/2013
27
(#53)
Transparency
“Transparency is a great floodlight. People who thrive in political maneuvering hate SCRUM…”
- Ken Schwaber
(#54)
Avoiding Burnout
XP = Sustainable pace
40 Hour Work Weeks
Prevent productivity collapse for overworked teams
10/30/2013
28
(#55)
High-bandwidth Communication
The best teams have “wide-open pipes”
Domain knowledge moves among the team
Information flows rapidly and accurately
(#56)
10/30/2013
29
(#57)
But wait, there’s more…
(#58)
10/30/2013
30
(#59)
(#60)
New Agile Benchmark Trends
Agile Trends - Iterations/Build Phase
Time-to-Market
10 100 1,000
New + Modified Size (thousands)
1
10
100
Mo
nth
s
Effort
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Pe
rson
-Mo
nth
s
Average Staff
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
He
ad
cou
nt
Bugs During QA
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
De
fects
10/30/2013
31
(#61)
(#62)
10/30/2013
32
(#63)
For Additional Information
Michael Mah Managing Partner QSM Associates email: [email protected] website: www.qsma.com blog: www.optimalfriction.com twitter: @michaelcmah Tel: 1 413-499-0988 Andrea Gelli QSM Associates Switzerland 8032 Zurich T +41 44 555 9126 email: [email protected]