Testing in Agile Projects: Where Things Stand Today Brian Marick [email protected] ght © 02004 by Brian Marick. Permission granted to reproduce verbatim. mages copyright www.arttoday.com.
Testing in Agile Projects:Where Things Stand Today
Brian [email protected]
Copyright © 02004 by Brian Marick. Permission granted to reproduce verbatim.Some images copyright www.arttoday.com.
What Are Agile Projects?
• An attitude toward change
• An attitude toward software
• Some attitudes toward people
The Manifesto for Agile Software Development, www.agilemanifesto.org
Changing Requirements Are Swell
Oh! Let’s add this!
Sure!Now that I see my
feature, I don’t like it. How should wechange it?
Software Can Be Soft
• Programs can become better, cleaner, and more capable
• They become changeable by being successfully changed– not mainly by planning for change
• Frequent new requirements “train” the code and the coders
“Agile methods, the Emersonian worldview, and the dance of agency”www.visibleworkings.com/papers/agile-methods-and-emerson.html
People
• Written documentation is a poor substitute for continuous conversation
• Generalists trump specialists• Teams can self-organize• Trust
– “… if you ask for help, someone has to help you” - Lisa Crispin
Agile Methodologies
• Extreme Programming
• Scrum
• DSDM
• Evolutionary delivery and staged delivery share many characteristics
See “Reading” slides at the end
The Import
Kaner, Bach, and Pettichord, Lessons Learned in Software Testingwww.context-driven-testing.com/wiki/scribble.cgi
These attitudes toward change, software, and people are the context for testing in Agile projects
Four Types of Testing
Geoffrey Moore, Crossing the Chasm, p. 19
Programmer Testing
I need an object that records eachtime segment
Test-driven designUnit testing
Programmer Testing
The test comes first
Programmer Testing
Code is written topass the test
Programmer Testing
Not done? Write another test…
Programmer Testing
And the code to pass it.All earlier tests continue to pass
Programmer Testing
Code awkward? Fix it now.Tests continue to pass
(Refactoring)
Martin Fowler, Refactoring
Programmer Testing
Eventually, the jelly is cooked, nailed down,and ready for further change
Status
Why was this decision made?
I need an object that records eachtime segment
Because of a Customer Representative
…I want to scribble notes aboutwhat I’m doing…
FIT Tests
Makes senseto the
customerrepresentative
Reminder,not
requirement
Easy to run
fit.c2.com
Test Results
Browser-Friendly
TestsFirst
Tests Make Change Smooth
smallchange
smallchange
smallchange
Has anythingbroken?
Has anythingbroken?
Has anythingbroken?
Tests Inform Programmers
• There are no explicit requirements or specifications– so tests cannot check code against them
• Tests serve same goal as requirements or specifications– they provoke programmers to
write the right programNo way!
Requirements for Test Notation
• Provoking the right code
• Improving product conversation– tests are something to talk about– ground conversation in the concrete– forging a common vocabulary
• Making possibilities more noticeable– explaining to someone else supplements
trying out working software– concreteness sparks ideas
The Tester As Participant
How can we best be concrete? What “goes without saying”?
What should the product not do?Who’s being overlooked?What bugs seem likely?
“Tester” As Job Title
How can we best be concrete? What “goes without saying”?
What should the product not do?Who’s being overlooked?What bugs seem likely?
Status
acceptance tests, customer tests, whole-product tests, business-facing tests…
Mid-Course Observation
Business Facing
Technology Facing
These tests primarilysupport programming(as well as the whole
team’s understanding)
Is This Really Testing?
• Checked examples– for discussion– for confident implementation– where are the bugs?
• Change detectors– for confident implementation– where are the bugs?
What About When the Examples Are Bad Examples?
(incomplete, misleading)
Extending the Model
Examples that usebusiness
terminology
Examples that useinterior
terminology
Business Facing
Technology Facing
Su
pp
ort
Pro
gra
mm
ing
Extending the Model
Examples that usebusiness
terminology
Examples that useinterior
terminology
?
?
Business Facing
Technology Facing
Su
pp
ort
Pro
gra
mm
ing
Critiq
ue P
rod
uct
Critiquing the Product
• What resource do we newly have?– the working product, including new code
• Exploratory testing– “simultaneous learning, test design, and
test execution” - James Bach
• Doing what?– diverse users and their scenarios– imaginative end-to-end testing– some opportunistic feature testing
www.satisfice.com/articles.shtml
Exploratory Bug Finding
A Quadrant Entry
Examples that usebusiness
terminology
Examples that useinterior
terminology
User-centeredbug reports
?
Business Facing
Technology Facing
Su
pp
ort
Pro
gra
mm
ing
Critiq
ue P
rod
uct
Status
Still Not Addressed
• What about security bugs, configuration bugs, performance problems, bugs revealed under load, usability problems (like suitability for color-blind people), etc. etc. etc.?– difficult to specify by example– whole-product, but not central to domain
These Are Technology Issues
• Understanding of implementation more important than understanding of a particular domain
Examples that usebusiness
terminology
Examples that useinterior
terminology
User-centeredbug reports
?
Business Facing
Technology Facing
Su
pp
ort
Pro
gra
mm
ingC
ritiqu
e Pro
du
ct
The Good News
My Take on the State of the Practice
Examples that usebusiness
terminology
Examples that useinterior
terminology
User-centeredbug reports
“ility” bugreports
Business Facing
Technology Facing
Su
pp
ort
Pro
gra
mm
ing
Critiq
ue P
rod
uct
Summary: Testing in Agile Projects
Reading (1)• Agile development in general
– www.agilealliance.org
– Agile Software Development, Alistair Cockburn
– “Agile methods, the Emersonian worldview, and the dance of agency”, Brian Marick, www.visibleworkings.com/papers/agile-methods-and-emerson.html
• Extreme Programming– Extreme Programming Explained, Kent Beck
– www.xprogramming.com/xpmag/whatisxp.htm
• Scrum– Agile Software Development with Scrum, Schwaber and Beedle
– www.mountaingoatsoftware.com/scrum
• DSDM– DSDM: Business Focused Development, DSDM Consortium
– www.dsdm.org
Reading (2)
• Agile from a tester’s point of view– www.testing.com/agile– Testing eXtreme Programming, Lisa Crispin and Tip House– [email protected]– www.testing.com/cgi-bin/blog
• Programmer testing– Test-Driven Design by Example, Kent Beck– Test-Driven Development: A Practical Guide, Dave Astels– Pragmatic Unit Testing, Hunt and Thomas– [email protected]
Reading (3)
• Exploratory testing– www.satisfice.com/articles.shtml– www.testingcraft.com/exploratory.html
• Context-driven testing– Lessons Learned in Software Testing, Kaner, Bach, and
Pettichord– www.context-driven-testing.com/wiki/scribble.cgi
• Miscellaneous– FIT: fit.c2.com (see also fitnesse.org)– Crossing the Chasm, Geoffrey Moore– Refactoring, Martin Fowler (et. al.)