Graham Thomas - Software Testing Secrets We Dare Not Tell - EuroSTAR 2013

Graham ThomasIndependent Software Testing Consultant

Software Testing Secrets

That We Dare Not Tell !

www.eurostarconferences.com

@esconfs#esconfs

Abstract

These questions are important because they drive at the very heart of what we are doing in the software testing industry today.

2

I have worked in IT for over 30 years, in software testing for over 20 years, so you might think that I know a lot about IT and in particular software testing. Well, I am going to share some dark secrets about software testing with you, which we dare not tell, and do that in the form of questions.

This session has been designed to be a highly interactive discussion which many people might find challenges their basic understandings. I will act as facilitator, give an introduction to each question, then actively moderate the debate and if needed act as arbiter. Come along, expect to be involved, and if you have a view then please share it. Help to drive forward the discussion –and the software testing industry.

A bit about you

A feel for the audience today Current Role

Above Program Test Manager

Program Test Manager or Test Manager

Test Leader

Test Analyst or Test Execution

New to Testing

Industry Experience 15 years +

Over 10

Over 5

Done a year or more

Still got the L plates on

Industry exposure

Industry body

Workgroup

Contributor

Blog

Speaker

Author

Conferences

First time at EuroSTAR

Other conferences and seminars

Workshops or meetings

3

Secretive Air Force Space Plane X37B Lands

4

1. What is the purpose of Software Testing?

5

What non-testers think

I was explaining this presentation to my doctor the other day.

I said “It is questioning the purpose of Software Testing.”

He said “ That is easy, it is to find all the bugs.”

I said “Not necessarily.”

He gave me a funny look, and I realised that this was a longer conversation that we had time for!

6

7

8

2. Just how effective is the way we test –and how do we know? (Trad, V-Model, Structured Testing, agile or any other form of testing for that matter?)

Effectiveness

GBP / USD per bug

Overall cost by requirement / use case /story

Organisational cost – If I do this I can’t then do that

Can I afford to test?

How long will it take

Why is it now taking longer

How do I measure it?

Can’t I have it now?

And what will I get for all that

The Time / Cost / Quality Dice

How will I know I have got all of that?

Do I need all that?

What happens if I don’t get all of that?

9

Discuss

10

3. If checking isn't software testing, then why is it that ‘checking’ is what our stakeholders are paying us to do?

11

Drivers for Testing

Checking

Prove it works

Show me it working

Coverage

Understand what I will get

Quantify the risk of delivery / non-delivery

Testing

Michael Bolton quoting Laurent Bossavit“A Testers Role is to see through the veil of conflicting illusions!”

Agile Testing is ‘Exploratory Learning’

Discovery

12

Discuss

13

4. If software testing is so difficult, demanding and challenging, then why is it that we keep on assigning the least skilled or experienced to perform it?

14

Conundrum

And even if we don’t know;

what the purpose of software testing is,

how effective our software testing is,

or agree on how we go about software testing,

surely we should at least be able to draw together some attributes for the kind of people we need to perform software testing?

15

Discuss

16

5. Why do software testers spend so much of their time running tests that do not find bugs?

17

Tools Scripting Execution Analysis Resources

Automation Paradox

To successfully automate tests they all have to pass. However what is the point of running tests that don’t find any bugs?

Tools to automate testing If we had tools that could test software we wouldn’t need software

testers!

Though, how would we know whether the tools were working or not?

Automating testing does not save any time, and in most cases takes longer than manual testing!

The latest trends in test optimisation are to stop running regression suites that take time effort and resource but no longer find bugs!

Inefficiencies

18

Discuss

19

To Conclude

As an industry, collectively:

“We do not know, or agree on the purpose of software testing”

“We do not know how efficient or effective our test models are, regardless of type”

“We don’t even agree on what it is we should do when we are testing software”

“We don’t understand what skills and experience we need to perform software testing”

“And finally when we do testing, an awful lot of it doesn’t actually make any difference”

I think we should either:

“Keep these secrets very, very quiet”

“Or tell everyone . . . . . .and set about finding the answers!”

20

Let me think about that for a minute

21

Contact Details

Graham Thomas

Independent Software

Testing Consultant

graham@badgerscroft.com

+44 7973 387 853

www.badgerscroft.com

22