Page 1
1© 2017 Rogue Wave Software, Inc. All Rights Reserved.
1
Confronting the mission-critical software testing challengeEpisode 3:
What if you could eliminate the hidden costs of development?
Alan McKellarV.P. software development
Walter CapitaniProduct manager, Klocwork
Page 2
2© 2017 Rogue Wave Software, Inc. All Rights Reserved.
2
Presenter
Alan McKellarV.P. software developmentRogue Wave [email protected] : @AlanMcKellar
Walter CapitaniProduct manager, KlocworkRogue Wave [email protected] : @walter_capitani
Page 3
3© 2017 Rogue Wave Software, Inc. All Rights Reserved.
3
1. What are “hidden costs”?2. Code reviews3. A bug’s life4. Issue crowdsourcing5. Wait times6. Klocwork static code analysis7. Q&A
Agenda
Page 4
4© 2017 Rogue Wave Software, Inc. All Rights Reserved.
4
What are “hidden costs”?
Page 5
5© 2017 Rogue Wave Software, Inc. All Rights Reserved.
5
What everyone else says
Page 6
6© 2017 Rogue Wave Software, Inc. All Rights Reserved.
6
What we’re talking about today
A different perspective on things that we know happen every day
Time/resources consumed but not identified, tracked, or acted upon
Page 7
7© 2017 Rogue Wave Software, Inc. All Rights Reserved.
7
Poll #1Which of the following hidden costs has the largest impact on your organization?
• Open source software costs• Server downtime• Support issues / customer escalations• Lack of skills• Delayed or rushed releases
Page 8
8© 2017 Rogue Wave Software, Inc. All Rights Reserved.
8
Code reviews
Page 9
9© 2017 Rogue Wave Software, Inc. All Rights Reserved.
9
Not enough code reviews
“Further analysis revealed that individual inspection performance varied by a factor of 10 in terms of faults found per unit time and individuals
found on average about 53% of the faults.”
1. DZone / Agile Zone, August 22, 20142. “Testing the value of checklists in code inspections,” Hatton, 2007
1
2
Page 10
10© 2017 Rogue Wave Software, Inc. All Rights Reserved.
10
Why the reluctance?
• Expensive– Multiple people working on the same module
• Developers would rather create than review– Find other ways of “reviewing”
Yet we all know early detection is cheaper to fix
Page 11
11© 2017 Rogue Wave Software, Inc. All Rights Reserved.
11
Static code analysis
if(i = j) j++;
if(i == j) j++;
Defect: Assignment operator used in
conditional statement
Assignment operator replaced with intended comparison operator
Vulnerable Code
Fixed Code
Page 12
12© 2017 Rogue Wave Software, Inc. All Rights Reserved.
12
• 80% of defects are introduced in development• Each defect found in test costs 50x to fix
Why the reluctance?
Page 13
13© 2017 Rogue Wave Software, Inc. All Rights Reserved.
13
A bug’s life
Page 14
14© 2017 Rogue Wave Software, Inc. All Rights Reserved.
14
Much more than fixing code
Impact on stakeholdersSales
Forced to avoid selling the feature
Sales
Spending time on the phone
Development
Fixing issues rather than creating new features
Marketing
Can’t talk about it
Support
Another brick in the wall
Marketing
Impact to brand image
C-suite
Applying pressure!
Page 15
15© 2017 Rogue Wave Software, Inc. All Rights Reserved.
15
Tools like Klocwork shorten cycle times, making it easier to meet delivery times.
Page 16
16© 2017 Rogue Wave Software, Inc. All Rights Reserved.
16
Issue crowdsourcing
Page 17
17© 2017 Rogue Wave Software, Inc. All Rights Reserved.
17
How many people does it take to fix a bug?“I found a
bug!”
“Now, how do I fix it?”
“I can help.”
“Have you tried this?”
“I’ve seen this before.”
“Is it fixed yet?”
MANAGER
Page 18
18© 2017 Rogue Wave Software, Inc. All Rights Reserved.
18
Be faster than Googling it
• Takes time to understand and translate results to your specific situation
• No validation that the “answer” is best for you
“Using Klocwork is WAY FASTER than Googling it!”- Walter Capitani, Feb. 2017
Page 19
19© 2017 Rogue Wave Software, Inc. All Rights Reserved.
19
Test environment vs. real world
Page 20
20© 2017 Rogue Wave Software, Inc. All Rights Reserved.
20
Wait times
Page 21
21© 2017 Rogue Wave Software, Inc. All Rights Reserved.
21
“The silent killer”*
30 days (53%) spent waiting between phases
“Define a software delivery strategy for business innovation,” Forrester Research, Inc., July 2014
Page 22
22© 2017 Rogue Wave Software, Inc. All Rights Reserved.
22
Shull et al estimate that non-severe defects take approximately 14 hours of debugging effort after
release, but only 7.4 hours before release.
* “What we have learned about fighting defects,” Shull et al, 2002
Page 23
23© 2017 Rogue Wave Software, Inc. All Rights Reserved.
23
Poll #2For your last major customer escalation incident, how did you feel about the effort to resolve the problem?
• Less than I was willing to put in• About what I expected• More than I was willing to put in
Page 24
24© 2017 Rogue Wave Software, Inc. All Rights Reserved.
24
What could you have done instead of working on that problem we just polled?
Page 25
25© 2017 Rogue Wave Software, Inc. All Rights Reserved.
25
Klocwork static code analysis
Page 26
26© 2017 Rogue Wave Software, Inc. All Rights Reserved.
26
Check code earlier & faster• Issues identified at your desktop
– Correct code before check-in• Issues identified through Continuous
Integration– Instant feedback at scale
• SmartRank recommendation engine helps prioritize work
• Create custom checkers to meet specific needs
• Debugger-like call-stack highlights the cause of the issues
Page 27
27© 2017 Rogue Wave Software, Inc. All Rights Reserved.
27
Summary
• Ineffective code reviews• Impact of bugs on the organization• How many people does it take to fix a bug?• “The silent killer”
Page 28
28© 2017 Rogue Wave Software, Inc. All Rights Reserved.
28
Q & A
Page 29
29© 2017 Rogue Wave Software, Inc. All Rights Reserved.
29
Try Klocwork nowwww.klocwork.com/free-trial
Page 30
30© 2017 Rogue Wave Software, Inc. All Rights Reserved.
30
Available for binge watching
www.roguewave.com/scaEpisode 1: How to achieve security, reliability, and productivity in less timeEpisode 2: Static analysis works for mission-critical systems, why not yours?Episode 3: What if you could eliminate the hidden costs of development?
www.roguewave.com/webinarsCar cybersecurity: What do the automakers really think? Five ways to create more secure codeStatic analysis’ role in automotive functional safety (ISO 26262)
Page 31
31© 2017 Rogue Wave Software, Inc. All Rights Reserved.
31