Software Test Automation SQuAD February 2010 Scott M. Allman [email protected].

Software Test Automation

SQuADFebruary 2010

Scott M. [email protected]

2http://xkcd.com/627

Classic

3

Outline

•This is an interactive session•What I am NOT covering today•Stages of software testing and tools for that stage•General observations on the tools•Your stories and questions about software test automation tools

4

Not Covered Today

•Commercial tools•Art of downloading and installing•Comparisons of different tools that play similar roles•People problems - technical problems

5

Universal and Easy Tools

•OpenOffice Freeware substitute for MS Office– http://www.openoffice.org/

•Yahoo Widgets– ScreenShot

•JEdit - Language sensitive editor– http://www.jedit.org/

•XAMPP– http://www.apachefriends.org/en/xampp-windows.html

6

7

8

9

Often Heard Reasons To Not Automate Software Testing

• I am not a programmer• Test automation tools do not apply to our

system• If I automate the testing I might lose my job• It takes too much time and not worth the

investment• Can you think of other reasons?

10

Stages of Software Testing

1.Design the experiments2.Create the data and inputs3.Set up the lab4.Execute tests5.Analyze results6.Report results7.Retrospect

11

Design the Experiments

•Few tools to help with experimental designs•Familiar: ECP and Boundary Value Analysis•FitNesse – a universal test runner–User inputs to tests–Worksheet inputs

12

FitNesse

•Installation– http://www.fitnesse.org/

•Key features– Wiki hierarchy of tests– History feature– Run Fitnesse

•Your stories?

13

Create the Data

•Generating data – XML•Generating loads•Mock objects for tough scenarios•Your Stories?

14

Set up the Lab

•Ensure tests run the same–Ant–Scripting languages

15

Execute the Tests

•Unit Testing - JUnit•Ant, again•Your stories ?

16

Analyze Results

•Visual capture of the screen and save the image•Chainsaw - Logs, custom logs, logs of all types

17

Chainsaw example

18

Chainsaw Log Analyzer

• Installation– http://logging.apache.org/chainsaw/index.html

• Replaces scripts – tail, grep• Originally designed for log4j but will work with

any well structured log• Run the Chainsaw

19

Report Results

•Installation– http://blog.testlink.org/

•Testlink– Reports are available anytime, anywhere– Archived results– Also has other test manager features

•Run TestLink

20

Retrospect

•How good are the tests ?•Coverage analyzers offer some data•Installation

– http://emma.sourceforge.net/•Emma Coverage Report

21

Summarize

•All tests need designs – Worksheets, TestLink•Data and inputs – simulators, mock object•Set up the lab – ant, TestLink•Execute the tests – Nunit, scripting languages•Analyze results – Chainsaw, screen capture•Report results – TestLink•Retrospect – coverage analyzers

22

Algorithm: Open Source Tools

Here is a simple approach:1.Find it2.Download and install it3.Learn it4.Adapt it5.Adopt it6.Try getting with FindBugs (for Java)

23

Your Stories

•Tools•Recommended tools?•Failures and disappointments ?•Success stories ?

24

Sources for the Tools– http://xkcd.com/627– http://www.openoffice.org/– http://www.jedit.org/– http://www.apachefriends.org/en/xampp-windows.html– http://www.fitnesse.org/– http://logging.apache.org/chainsaw/index.html– http://blog.testlink.org/– http://emma.sourceforge.net/

•My thanks to SQuAD for the opportunity to present today•[email protected]

25

Questions?