When Testers Feel Left Out in the Cold

nt Session

Presented by:

Hans Buwalda

Brought to you by:

340 Corporate Way, Suite Orange Park, FL 32073 888‐2

W1 Concurre4/9/2014 10:30 AM

“When Testers Feel Left Out in the Cold”

LogiGear

300,68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com

http://starcanada.techwell.com/�

Hans Buwalda LogiGear

Hans Buwalda has been working with information technology since his high school years. In his thirty year career, Hans has gained experience as a developer, manager, and principal consultant for companies and organizations worldwide. He was a pioneer of the keyword approach to testing and automation, now widely used throughout the industry. His approaches to testing, like Action Based Testing and Soap Opera Testing, have helped a variety of customers achieve scalable and maintainable solutions for large and complex testing challenges. Hans is a frequent speaker at STAR conferences and is lead author of Integrated Test Design and Automation: Using the Testframe Method.

http://members.techwell.com/sme-profiles/hans-buwalda�

2/5/2014

1

STARCANADA 2014Toronto, Canada

"Test Management" Track Session W1

Wednesday, April 9, 10.30 – 11.30

© 2014 LogiGear Corporation. All Rights Reserved

Hans BuwaldaLogiGear

hans @ logigear.com

Who is your speaker

Software testing company, around since 1994Testing and test automation services:

www.logigear.comwww.testarchitect.com

LogiGear Corporation

Testing and test automation services:− consultancy, training− test development and automation services− "test integrated" development services− focus on big and complex testing projects

Products:− TestArchitect™, TestArchitect for Visual Studio™− integrating test development with test management and automation− based on modularized keyword-driven testing


Dutch guy, in California since 2001Background in math, computer science, managementSince 1994 focusing on automated testing− keywords, agile testing, big testing

Hans Buwalda hans @ logigear.comwww.happytester.com

2/5/2014

2

Scope of this talk

Introduction to the management aspects specific for a test projectNot a project management co rseNot a project management courseA personal view, based on experiences with my own methodCollection of ideas and experiences, not necessarily a complete pictureInput and discussions welcome (also talk with each other!)


Input and discussions welcome (also talk with each other!)The ideas are coming from mixed European, American and Asian experiences, apply with care

The Challenges for a Test Process

testing should be fun

testing sho ld be effecti etesting should be effective

testing should be efficient

testing should be under control


2/5/2014

3

Keywords, Action Words

number name quantitynew product P-9009 Sledge Hammer 5

number quantityadd quantity P-9009 20

number name quantitynew product P-9009 Sledge Hammer 5

number quantityadd quantity P-9009 20

actions, each with a keyword and arguments read from top

to bottom

fragment from a test with actions

Common in automation tools nowadays (but with different styles)

Identify tasks for both test development and automation

q yadd quantity P-9009 3add quantity P-9009 6

number quantitycheck quantity P-9009 34

q yadd quantity P-9009 3add quantity P-9009 6

number quantitycheck quantity P-9009 34

"34" is the expected value here

to bottom


y p

The test developer creates tests using actions

Each action consists of a keyword and arguments

The automation task focuses on automating the actions

Each action is automated only once

Test Module Plan

Test Module 1 Test Module 2 Test Module N

Overview

Tests

Objectives

Tests Tests

Actions

. . .Objectives Objectives


AUTOMATION

user passwordlog in jdoe StarEast

first last brand modelrent car John Renter Ford Escaperent car John Renter Chevrolet Volt

last totalcheck bill Renter 140.42

interaction test business testwindow control value

enter log in user name jdoeenter log in password StarEast

window control property expectedcheck prop log in ok button enabled true

car guy

2/5/2014

4

Discussing costs and benefits


Examples of costs and benefits

Involvement of users and business specialists

Preventation of incidentsShorter time to market

Costs? Benefits?

Need for investmentsBuild up of a test organisationLicense costsTraining, coachingMaintenance of (much) testware...

Increased quality and certaintyLesser dependence on users and expertsElimination of dull workImproved documentation...


Business Benefits• Time-to-Market• Quality-to-Market• Control

2/5/2014

5

What can make testing miserable . . .

Discussions about costs and benefits

Resistance

Commitment

Politics

Dependence

Unrealistic expectations


Unrealistic expectations

Difficulty

Motivation

Practical issues and problems

Resistance is Normal

“Let's reconsider” “I don't understand this”

WHEN THEY SAY : THEY COULD THINK :

Let s reconsiderNow is not the timeAll the time new objections“Fine, but are we ready for this?”Saying nothingSaying yes, acting noThe method is good but in

I don t understand this“I didn't expect all this” “This is going to cost me my job”I'm the star here, I don't need the competition“We can't achieve this”We will become too


The method is good, but in this specific case ….

We will become too dependent on those guysNow they will find out how bad we are testing

2/5/2014

6

GETTING COMMITMENT...


Management Commitment

Offer solutions not additional problems

Testing is often not popular . . .Nobody wants an extra task (extra problem)

Bug are bad

Testing is good

Offer solutions, not additional problemsTell managers that a good tested system creates a positive image (not only negative reasons for testing)Present/show what you're doing (glass box)Try to get clarity about policies and directions for testingKeep in mind: managers want things to be under control


Keep in mind: managers want things to be under control − give clear and timely information about (1) progress and (2) results

Use outsiders and/or books/articles to make your caseTry to find some bugs . . .

2/5/2014

7

Commitment, specifics

“No time, no money, ...”− back to the problem− you should not become the problem owner!you should not become the problem owner!

“It is so expensive/it is so difficult”− testing is expensive and difficult− test automation is difficult

“The others should do the testing”figure this out


− figure this out− you can't deal with this yourself (strategic context mismatch)− "doing things right" versus "doing the right things"

General vagueness− hidden problems and conflicts

Politics . . .

Who is to blame when things go wrongSystems are sometimes released while not finishedTesters tend to get the bullet

Notify the responsible managers early about what is going to happenMake clear written down procedures, especially for the test execution phaseNot every bug is equally important (pick your battles)


Make the test process transparentMaintain an atmosphere of cooperation and communication with allAsk for help, sound the alarm, − don’t underestimate the “politics of failure”, it can hurt you

2/5/2014

8

The Life Cycle of an IT Project

An (Old) Joke, Still Valid

EnthusiasmDisappointment! ! ! PANIC ! ! !Looking for guilty ones ("blame-storming")


Punishing the innocentRewarding those who had nothing to do with it

The 3 questions for a bug

Ask these question, in the given order

1. Is it a problem?

2. What is the root cause?


3. Why did we not catch it?

2/5/2014

9

Dependencies . . .

Testing and automation is dependent on many factors:• Working systems• Test environments (like virtual machines)

S ifi ti i t d i k l d

Make clear arrangements with everybody about everything, as early as possible− written down

• Specifications, requirements, domain knowledge• People• ...


Keep in touch with the rest of the projectMake the high level test products as early as possible

Dependencies, specifics

“The system under test isn't available”− The automation will be the first in trouble− Discuss this (early)Discuss this (early)− Allocate resources when really needed

“The system under test doesn't work”− The automation will be the first in trouble− Create a “health check” (smoke test to see if all

functionalities that are relevant to the automation work)

“Th i i ”


“There is no test environment”− Start planning this as early as you can− Make clear how important this is− “No test environment, no test”

2/5/2014

10

Expectations Around AutomationThe impression can arise that automated testing (in particular with keywords) is just a push of a button . . .

even if nobody said so!

Message: testing is difficult, automated testing is even more difficult

Make clear what is happening

Communicate manage expectations

. . . even if nobody said so!


Communicate, manage expectations

Don't put too low figures in a test plan− let somebody else lower them (in writing)

Ask managers and other people involved what they expect

Difficult to Keep People Motivated

Watch the signals

Motivation of team members can erode when time progressesHappens to both test developers and automation engineers

Watch the signals

Make sure that the right people are assigned− Automation engineers should have a programming background− Testers should have a testing and/or business background

Avoid “monks work” (Dutch expression for tedious repetitive work)

Differentiate the work

Avoid isolation of the test group


Be prepared to switch roles

Consider agile approaches

Be ready for this problem in advance

Create operational and professional communication structures− for example special interest groups to regularly discuss professional topics

2/5/2014

11

Complexity: The Work is Difficult

Making tests is supposed to be "difficult":• Finding bugs of others• Making concrete examplesAutomating them is difficult as well

Don't be over-confident, use coaching from experienced peers or outsidersConsider reference visits to other projects or sitesTalk about the work, even create special interest groupsKeep in touch with others doing the same work

Automating them is difficult as well


Keep in touch with others doing the same workOrganize interactions (meetings!), for example with:− users− developers− auditors

Delegate tasks (better lazy then crazy...)

Practical Issues and ProblemsDoes the test tool work here ? ?Do we have authorizations on the test environment ? ?Where do we keep our test products ? ? Which tests do we have ? ?

Be prepared for this, automated testing is a “cloud of details”Put suitable team members in leading roles, and delegate responsibilities to themF l t t j t t t t i f ll ti j b

Questions, chores, details, . . .


For larger test projects test management is a full time job, make sure you get this timeTry to find easier ways, organize things better, or automate more− “If you have a difficult task, ask a lazy man. He will find an easier way”

2/5/2014

12

Some final remarks

Changing a process can mean:− changing of work processes− a learning curvea learning curve− risks and benefits

It is not the same as: buy a tool and all will be well

Changing any process is more than anything an


organisational change and should be managed as such

Thank you for listening

…and...

keep cool, even in the cold


When Testers Feel Left Out in the Cold

Technology

logigear corporation

test process testing

test project

test automation services

test developer

test module plan test

agile testing

effective testing