rent Session Presented by: Paul Holland Te s Brought to you by: 340 Corporate Way, Suite Orange Park, FL 32073 888‐2 T9 Concur 4/8/2014 2:00 PM “Agile Test Management and Reporting‐ Even in a Non‐Agile Project” sting Thought 300, 68‐8770 ∙ 904‐278‐0524 ∙ [email protected]∙ www.sqe.com
13
Embed
Agile Test Management and Reporting—Even in a Non-Agile Project
Whether you have dedicated test teams or testers distributed over Scrum teams, you have the challenge of planning, tracking, and reporting their testing not only in a meaningful way but also in a way that can adapt to the rapidly changing environment of software development projects. Many commonly used planning methods do not allow for flexibility, and reporting often relies on horribly flawed metrics including number of test cases executed or test pass percentage. Paul Holland explains a planning, tracking, and reporting method he developed during his last five years as a test manager at Alcatel-Lucent. Paul describes how he uses powerful “high-tech” tools like whiteboards and spreadsheets to create easy-to-understand visual representations of his group’s testing. Learn how you can create status reports that provide the details that upper management seeks. These status reports are effective in both waterfall and agile environments—and will stand up to management scrutiny.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
rent Session
Presented by:
Paul Holland Te s
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073 888‐2
T9 Concur4/8/2014 2:00 PM
“Agile Test Management and Reporting‐ Even in a Non‐Agile Project”
An independent software test consultant and teacher, Paul Holland has more than sixteen years of hands-on testing and test management experience, primarily at Alcatel-Lucent where he led a transformation of the testing approach for two product divisions, making them more efficient and effective. As a test manager and tester, Paul focused on exploratory testing, test automation, and improving testing techniques. For the past five years, he has been consulting and delivering training within Alcatel-Lucent and externally to companies such as Intel, Intuit, Progressive Insurance, HP, RIM, and General Dynamics. Paul teaches the Rapid Software Testing course for Satisfice. For more information visit testingthoughts.com.
Agile Test Management and Reporting –Even in a non-Agile Project
Paul HollandConsulting Tester and Teacher
at Testing Thoughts
My Background• Independent S/W Testing consultant since Apr 2012• 16+ years testing telecommunications equipment y g q p
and reworking test methodologies at Alcatel-Lucent• 10+ years as a test manager• Presenter at STAREast, STARWest, Let’s Test,
EuroSTAR and CAST• Keynote at KWSQA conference in 2012• Facilitator at 35+ peer conferences and workshopsFacilitator at 35 peer conferences and workshops• Teacher of S/W testing for the past 5 years• Teacher of Rapid Software Testing
• Over the past 10 years I have spoken with many people regarding Software Testing I cannotpeople regarding Software Testing. I cannot directly attribute any specific aspects of this talk to any individual but all of these people (and more) have influenced my opinions and thoughts on metrics:– Cem Kaner, James Bach, Michael Bolton, RossCem Kaner, James Bach, Michael Bolton, Ross
Collard, Doug Hoffman, Scott Barber, John Hazel, Eric Proegler, Dan Downing, Greg McNelly, Ben Yaroch
1. Measure and/or compare elements that are inconsistent in size or compositioninconsistent in size or composition– Impossible to effectively use for comparison– How many containers do you need for your
possessions?
– Test Cases and Test StepsTest Cases and Test Steps• Greatly vary in time required and complexity
– Bugs• Can be different severity, likelihood - i.e.: risk
2. Create competition between individuals and/or teamsand/or teams– They typically do not result in friendly competition– Inhibits sharing of information and teamwork– Especially damaging if compensation is impacted
– Number of xxxx per tester – Number of xxxx per feature
Impact of Using Bad Metrics• Gives Executives a false sense of test coverage
– All they see is numbers out of contexty– The larger the numbers the better the testing– The difficulty of good testing is hidden by large “fake”
numbers• Dangerous message to Executives
– Our pass rate is at 96% so our product is in good shapeCode co erage is at 100% o r code is completel– Code coverage is at 100% - our code is completely tested
– Feature specification coverage is at 100% - Ship it!!!• What could possibly go wrong?
– Work to be done (divided into two sections)Work to be done (divided into two sections)– Work in Progress– Cancelled or Work not being done– Completed work
• Red stickies indicate issues (not just bugs)• Create a sticky note for each half day of work (or
mark # of half days expected on the sticky note)• Prioritize stickies daily (or at least twice/wk)• Finish “on-time” with low priority work incomplete
• All of these items are optional – add your own p yelementsUse what makes sense to your situation– Charter Title (or Test Case Title)– Estimated Effort– Feature area
Tester name– Tester name– Date complete– Effort (# of sessions or half days of work)
INP vs. REIN ARQ 6 7 5 jbright 01/06/2012 01/10/2012
To translate the files properly, had to install Python solution from Antwerp. Some overhead to begin testing (installation, config test) but was fairly quick to execute afterwards
INP vs. REIN + SHINE ARQ 12 12
Traffic delay and jitter from RTX ARQ 2 2 0 ncowan 12/05/2011 12/05/2011
Took longer because was not behaving as expected and I had to make sure I was testing correctly. My expectations were wrong based on virtual noise not being exact.
Weekly Report
• A PowerPoint slide indicating the important issues (not a count but a list)issues (not a count but a list)– “Show stopping” bugs– New bugs found since last report– Important issues with testing (blocking bugs,
equipment issues, people issues, etc.)Risks (updates and newly discovered)– Risks (updates and newly discovered)
– Tester concerns (if different from above)– The slide on the next page indicating progress