Belgium Testing Days - Making Test Automation Work in Agile Projects

Making Test

Automation Work in

Agile Projects

Belgium Testing Days 2012

Lisa Crispin

With Material from Janet Gregory

Introductions: Experience, Goals

Form groups: New to automation One to two years experience w/ automation More than two years experience

Talk with at least two other people Tell each other your learning goals for today Note the most interesting one you hear

About me…

Programmer, tester, agile team member

Introduction - You

Main role on team? Programming, automation experience? Using agile approach? What have you automated? (Test,

CI,deployment...)

Takeaways

Foundation for successful test automation “Whole Team” approach What/when to automate Identifying, overcoming barriers Choosing, implementing tools First steps We won’t do any hands-on automation, but will demo

some examples

Exercise: Your Learning Goals

Write one interesting learning goal you heard from another participant at the start of the class on a sticky note. Write your number one learning goal for

today on a sticky note. Put the sticky notes on your table group

“learning goals” sheet Group similar ones together

Let‟s start by defining “agile”

Agile teams:

Deliver business value

frequently

at a sustainable pace

while adapting to the changing

needs of the business

Source: Elisabeth Hendrickson

The key is “sustainable pace”

Technical debt slows us down

The Death

Spiral

High technical debt +

insufficient automation = even

less time

Barriers to Test Automation

What‟s holding you back?

Exercise: Your barriers

Individually and silently – write one barrier hindering your team per sticky note

Put these on the “impediments” wall chart for your table group

Talk to your fellow group members – do you see any patterns?

Questions?

The Whole-Team Approach

Automated tests are code Respecting the tests Collaborating Commitment to quality Return on investment

Automated tests are code

public class CalculatorFixture extends

ColumnFixture {

public String startDate;

public String endDate;

public double startBalance;

public double endBalance;

public String irrTarget;

private Calculator calculator = new

Calculator();

private Double irr;

Test code deserves the same respect as

production code! (Rodney

Dangerfield, a

comedian whose

tag line was “I

don’t get no

respect… )

Source: Gojko Adzic, StarEast 2011 keynote

Testers are especially good at…

Eliciting examples

Turning them into tests

Ensuring the right testing

gets done

Exploratory testing

Give testers time to do what we do best

The Whole-Team Approach

Experiment: Iteration 1

Pair up: one will be tester, one programmer

Sit back to back so you face away from each other

Tester gets a drawing which needs to be replicated

Tester tells the programmer what to draw, all at one time.

Programmer draws the shapes based on what the tester explained.

No talking during „coding‟!

Tester “tests” the drawing, reports “bugs” on index cards

Programmer fixes the “bugs”

How long did it take? Will the customer be happy?

Experiment: Iteration 2

Collaborate!

Tester tells programmer what to draw, watches the programmer draw, answers questions, points out „defects‟ for programmer to fix immediately

(Don‟t show the programmer the drawing, that makes it too easy, we‟re trying to simulate real coding)

How long did it take? Will the customer be happy?

Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game

Ways to collaborate

Team responsibility

Automate all regression tests

Commitment to quality What‟s your team‟s commitment? The best possible software product?

Experiment

Meaningful

Commitmen

Under-commit

Plan less work than you think you can do

Including all test automation

Learn to write maintainable tests

Get over the “hump of pain”

From Gerard Meszaros’ XUnit Test Patterns

Whole-team software development

Create a user story

Write Customer

(Q2) Tests

Start thinking how to code

Expand tests – Story Tests

Automate Q2 Tests

Exploratory testing

Customer User

Acceptance

Product owner Product owner/ Tester Tester Tester/Programmer Programmer

Pair, “Show

Exercise

•Think of an experiment to get your

whole team engaged in automating

•Share with your table group •Pick two to share with the class

Getting Over the Hump

The test automation pyramid The agile testing quadrants What should be automated What shouldn't Difficult areas

Test Automation Pyramid

Agile Testing Quadrants

Shared Understanding

Start with tests Collaborate

What Should We Automate?

Quadrant 1 tests

Unit, component, TDD

Quadrant 2 tests

API, service-level

Quadrant 4 tests

Load, performance, stress

Quadrant 3 tests?

Leverage automation where useful

What Shouldn‟t We Automate?

Quadrant 2 tests

Wizard of Oz, prototyping

Quadrant 3 tests

Usability, UAT, ET

Tests that will never fail?

Assess risk

ROI not enough

One-off tests

Where Should We Be Careful?

GUI tests

Watch ROI

End-to-End tests

Push testing down to lowest level

Remember the Pyramid

Hard to Automate?

Legacy code

Hard to automate, or just lack of skill?

“Working Effectively with Legacy Code” – Feathers

“Strangling” – Fowler, Thomas

Exercise: Low-Hanging Fruit

Agile Automation Strategy

What hurts the most

Layered approach

Applying agile principles

Small chunks/thin slices

Smart test design

Choosing the right tools

What Hurts the Most

Use retrospectives

Keep an impediment backlog

Multi-Layered Approach

Example: Learn TDD at unit level Automate GUI smoke tests

Simplicity

Address one or two needs at a time

Understand the problem first

Try simplest approach first

Work in small chunks, thin slices

Incremental & iterative

Automate a Slice at a Time

Example: 4-step UI to validate, upload profit sharing contribution data

• Thread 1: All four pages with navigation

• Thread 2: Select year, enter description on page 1, display on page 2, browse and upload file on page 2

• Thread 3: Validate data in file, display on page 3

• Thread 4: Persist data, display „success‟ message on page 4

Thin Slice Example

Exercise: Thin Slices

Here‟s our user story:

As an internet shopper, I want to create an

account so that I do not have to enter my address

and billing information each time I make a

purchase

Draw a mind map for this story on a big sheet of

Identify a basic end-to-end slice of functionality

that can be coded, tested, and automated.

If you have time, identify additional slices.

Test Design Patterns/Principles Code design patterns One clear purpose Don‟t Repeat Yourself

Iterative Feedback

Spike two different approaches Pick one to try for N # of iterations Use retrospectives to evaluate

Learn by Doing

Courage – don‟t be afraid to fail Production code practices for test code Incremental, thin slices Experiment

Questions About Automation Strategy?

Choosing Tools

Team effort Time Requirements Focus on goals, problems, not tools. Experiment

Understand the Purpose

Who‟s using the tests? What for? What‟s being automated? Existing tools, environment Who‟s doing what for automating?

What Fits Your Situation • Existing skills • Language of application under test • Collaboration needs • What‟s being automated • Life span, future use of tests

Test Drivers/Frameworks

Automation layer Fit for everyone on team Try out more than one Let‟s look at more examples

Where To Find Tools

www.softwareqatest.com/qattls1.html

www.testingfaqs.org

www.opensourcetesting.org

awta.wikispaces.com/2009ToolsList

groups.yahoo.com/group/agile-testing

http://bit.ly/AgileTestTools - aa-ftt spreadsheet

Example: My Team‟s Tool Choices

• IntelliJ Idea

• Jenkins, ant, Maven

• JUnit

• FitNesse

• Canoo WebTest

• Watir

• JMeter

• Selenium 2.0 / WebDriver with Geb framework

Exercise: Tools

Making Test Automation Work

Time to do it right

Learning culture

Testable architecture

Test data

Managing tests

Time To Do It Right

Limit scope, don‟t over-commit

Write automation task cards

Quality must be team goal

Long-term, will let you go faster

Learning Culture

OK to make mistakes

Lots of small experiments

Evolve right design

Testable Architecture

• Layered architecture • eg. UI, business logic, data access

• Ports and Adapters pattern • App can work without UI or database

• Ports accept outside events

• Adapters convert for human or automated

Test Data

Avoid database access when possible

Setup/Teardown Independent, rerunnable tests

Canonical data Refresh before each test run

Customizable data for ET

Production-like data Get customers to provide example data

Managing Automated Tests

Tests as Documentation

Continuous Integration

Reporting results

Metrics

Tests as Documentation

Understandable

Who will really use them?

Once passing, must always pass

Any Example Can Become a Test

Given/Then/When Example

Scenario: Valid name search returns results

GIVEN that Kant is a supervisor with employees

AND Kant has an employee named Smith

WHEN Kant navigates to the employee name

search page

AND enters the value “S”

THEN Kant will see a search result that includes

Continuous integration/testing…

…short feedback loop

Keep tests passing

Stop the line to fix problems

Test Management Tools

Manage tests, code together

Some tools have own management

What problem are you trying to solve?

Exercise: Tests as Documentation

Key Success Factors

Whole Team Approach

Thin Slices

Short Feedback Loops

Experiments

Specification by Example

Exercise: Breaking Barriers

Remember

It‟s a team problem!

Tackle automation problems with diversity

Try small experiments

Baby steps – start simple

Questions? “Aha” Moments?

Agile Testing: A Practical Guide for Testers and Agile

By Lisa Crispin and Janet Gregory

www.agiletester.ca

Experiences of Test Automation

Dorothy Graham and Mark Fewster

Beautiful Testing: Leading Professionals Reveal How

They Improve Software

Edited by Tim Riley, Adam Goucher

Includes chapter by yours truly

Test Patterns

Xunit Test Patterns: Refactoring Test Code

By Gerard Meszaros

Specification by Example

How successful teams deliver the right

software

Gojko Adzic

Case studies from > 50 teams

Agile Test Automation Resources

dhemery.com/pdf/writing_maintainable_automated_accep

tance_tests.pdf

lisacrispin.com janetgregory.ca gokjo.net exampler.com agile-testing@yahoogroups.com testobsessed.com testingreflections.com pairwith.us

Belgium Testing Days - Making Test Automation Work in Agile Projects

gojko adzic

sticky note

automation

programmer

tester

tests

team

tools

Technology

Agile Td Automation

Test automation in an agile context - Sogeti...

Building QA Automation Using Agile - Synerzip€¦ ·...

Gherkin for test automation in agile

Final Paper Automation in Agile Testing - STC...

Making Test Automation Work in Agile...

Agile Test Automation - WordPress.com · Agile Test...

Strategies for agile software test automation

AGILE AND AUTOMATION CONCLAVE 2018 · AGILE AND AUTOMATION....

Agile Automation - · PDF fileAgile Automation What does...

Atd agile automation

Making Test Automation Work in Agile Projects - Agile...

Test Automation Strategies for the Agile World

Agile Test Automation Strategy • Motivation –The Agile.....

Automation Dashboard - Symbio · Automation Dashboard •.....

Agile Automation Strategies and Frameworks