- 1. W6 Special Topics 5/1/2013 11:30:00 AMYin and Yang: Metrics
within Agile and Traditional Lifecycles Presented by: Shaun
Bradshaw, Zenergy Technologies, Inc. Bob Galen, Rgalen
ConsultingBrought to you by:340 Corporate Way, Suite 300, Orange
Park, FL 32073 888-268-8770 904-278-0524 [email protected]
www.sqe.com
2. Shaun Bradshaw Cofounder of Zenergy Technologies, Inc., a QA
and agile solutions consulting firm (zenergytechnologies.com),
Shaun Bradshaw has spent the past fifteen years advising, teaching,
and mentoring clients to improve their QA and test processes with
effective testing and test management techniques. He is the
coauthor, editor, and trainer of a suite of methodologies covering
testing, test management, and test metrics. A popular speaker at
many of the major QA industry conferences, Shaun is known for his
presentations on test metrics, the S-Curve, and the Zero Bug
Bounce.Bob Galen Bob Galen is an agile coach at RGalen Consulting
and director of agile solutions at Zenergy Technologies, a North
Carolina-based firm specializing in agile testing and leading agile
adoption initiatives. Bob regularly speaks at international
conferences and professional groups on topics related to software
development, project management, software testing, and team
leadership. He is a Certified Scrum Master Practicing (CSC),
Certified Scrum Product Owner (CSPO), and an active member of the
Agile Alliance and Scrum Alliance. Bob published Scrum Product
OwnershipBalancing Value from the Inside Out, which addresses the
gap in guidance toward effective agile product management. Contact
Bob at [email protected] or [email protected]. 3.
4/16/2013The Yin & Yang Title of Software MetricsSTAREAST 2013
Presented By: Bob Galen and Shaun BradshawSlide 1Software Peace of
Mind in Action??? Introductions Bob Galen & Shaun
BradshawDirector, Agile Solutions Zenergy Technologies Determined
agile adoption coach, trainer, and consultant 10+ years of agile
experience across Software, QA/Test, and Project Management
Specialty in Agile at-Scale challenges
[email protected] 2VP of Consulting Solutions
Zenergy Technologies Experienced test manager, consultant, trainer
IT Testing strategist 15 years of multi-domain experience Agile
experience; Scrum Master
[email protected] Peace of Mind in
Action1 4. 4/16/2013??? Format For This Session Focus for this
session: QA & Testing Metrics Panel format 15 minutes for Shaun
15 minutes for BobWrite down your questions, ideas, metrics, etc.
on 3x5 cards 30 minutes of Q&A interaction and debate Slide
3Software Peace of Mind in ActionWhat is Measurement? Measurement
is the process by which numbers or symbols are assigned to
attributes of entities in the real world in such a way as to
characterize them according to clearly defined rules. Measurement
is the empirical, objective assignment of numbers, according to a
rule derived from a model or theory, to attributes of objects or
events with the intent of describing them.2 5. 4/16/2013???
Software Metrics Traps Karl Wiegers 1. 2. 3. 4. 5. 6. 7. 8. 9.
10.http://www.processimpact.com/articles/mtraps.htmlLack of
Management Commitment Measuring Too Much, Too Soon Measuring Too
Little, Too Late Measuring the Wrong Things Imprecise Metrics
Definitions Using Metrics Data to Evaluate Individuals Using
Metrics to Motivate, Rather than to Understand Collecting Data That
is Not Used Lack of Communication & Training Misinterpreting
Metrics DataSlide 5Software Peace of Mind in Action??? Case Study
Large, traditional financial / insurance organization 100 year
company, traditionally staffed IT organization Regulatory support
Large-scale project; joint with other consulting firmsSlide
6Software Peace of Mind in Action3 6. 4/16/2013??? Waterfall is
Real Not every project can be broken down into 2-week chunks Not
every company wants to be agile Scale of the project impacts
everything Well know were done when were done is NOT an acceptable
answer in many contexts Waterfall and AgileSlide 7Software Peace of
Mind in Action??? Monitoring Phases & Transitions The
overriding goal was to plan a series of maturation phases Testing
would serve as the guide between phases Qualifying progress
Verifying functionality Measuring variation / defects Guiding next
stepsSlide 8Software Peace of Mind in Action4 7. 4/16/2013???
Risk-Based TestingAll testing was focused towards what Was
delivered Worked Was planned Was considered Higher Priority
features Was considered Higher Risk itemsSlide 9Software Peace of
Mind in Action??? Key Metrics Exit Criteria Test Execution by
priority Have we tested enough? Did we cover our plans?Overall pass
rates Is the software working? Where are we Blocked? Are there any
Patterns to the failuresPercent of defects corrected by severity
Are we catching up on repairs? Is the software maturing at a
sufficient rate to support the plan? Slide 10Software Peace of Mind
in Action5 8. 4/16/2013??? Customer Override Basically, the
customer has the final authority to adjust Metrics Analysis
(Re)Actions Goals So, agility from a leadership & governance
perspectiveSlide 11Slide 12Software Peace of Mind in ActionSoftware
Peace of Mind in Action6 9. 4/16/2013TransitionSlide 13Software
Peace of Mind in Action??? Agile Metrics Agile methods are
generally not metrics centered, Theyre much more team-centered and
progress or outcome centered Rarely do you want to examine the
individual; leave that to the team Exceptions experienced
coachesAvoid Metrics Dysfunction metrics Lines of code produced
Bugs found Estimate vs. Actual Slide 14Software Peace of Mind in
Action7 10. 4/16/2013??? Big 4 Agile Metrics 1. 2. 3. 4.Value
Delivered Velocity & Throughput Quality Team Morale &
SatisfactionSlide 15Software Peace of Mind in Action???
Value-Delivered Prioritization Always deliver the highest priority
work first Earned value its DoneCustomer Acceptance User Story
sign-off Active customer demonstration; embrace changesROI Usage
Sales & usage targets; confirm Did we make the right decisions
on value? Adjustments? Slide 16Software Peace of Mind in Action8
11. 4/16/2013??? Velocity & Throughput Productivity matters in
agile teams First, the software is only accepted if it meets clear
Done-Ness criteria Velocity the number of Story Points produced per
Sprint Throughput the number of Days that a story was InProgress,
relative to its size; also a risk playContinuous improvement
Velocity UP; Throughput - DOWN Slide 17Software Peace of Mind in
Action??? Quality Test Automation Continuous Integration &
Continuous Deployment Every check-inrun automation;
Stop-the-LineDefect Escapes Escaping your Done-Ness Escaping your
Sprint Escaping into the wildCustomer EscapesPairing, inspections,
RCA Continuous Improvement downward trending Slide 18Software Peace
of Mind in Action9 12. 4/16/2013??? Team Morale & Satisfaction
Self-directed; Team autonomy; Listened to Attrition is analyzed;
New hire onboarding Sustainable pace Periodic surveys Team overall
health & satisfaction Feedback on adjustments Agile valuesMake
the survey and the actions Transparent Make it clearpeople are
central to your successSlide 19Software Peace of Mind in Action???
Thats it Aligned with Goals Sprint Goals & Release Goals
Organizational Goals & Company GoalsAttributes: Theyre all
importantperhaps People First There are no fixed targets Keep them
simple; everyone is aware of the goalMeasure as a team Stop the
individual metrics dysfunction Slide 20Software Peace of Mind in
Action10 13. 4/16/2013TransitionSlide 21Software Peace of Mind in
ActionNow Its YOUR Turn Questions? Debate!Slide 22Software Peace of
Mind in Action11 14. 4/16/2013??? Questions for your Consideration
What healthy metrics have you personally experienced? What about
stories of dysfunctional metrics? Challenges youve seen in moving
from Waterfall to Agile or vice versa? If you only had 1-3 metrics
to measure software in your own company, what would they be?Slide
23Software Peace of Mind in Action??? Questions for your
Consideration Example of what we could discuss Estimation vs.
Actuals Planning Poker & Relative estimationAnyone use
GQM?Slide 24Software Peace of Mind in Action12 15. 4/16/2013Any
other questions?Thank you!Slide 25Software Peace of Mind in
Action13