Top Banner
© ASERT 2006-2010 Make your Testing Groovy Dr Paul King, ASERT, Australia [email protected], paulk_asert Full: http://www.slideshare.net/paulk_asert/make-tests-groovy
202

Make Your Testing Groovy

May 10, 2015

Download

Technology

Paul King

Using the Groovy dynamic language for primarily functional / acceptance / customer / BDD testing with a forward looking perspective. Also considers polyglot options. The techniques and lessons learned can be applied to other kinds of testing and are also applicable to similar languages. Drivers and Runners discussed include: Native Groovy, HttpBuilder, HtmlUnit, WebTest, Watij, Selenium, WebDriver , Tellurium, JWebUnit, JUnit, TestNG, Spock, EasyB, JBehave, Cucumber, Robot Framework and FitNesse/Slim. Also looks at JMeter, ScalaCheck, Choco, AllPairs and ModelJUnit
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Make Your Testing Groovy

© A

SE

RT

2006-2

010

Make your Testing Groovy

Dr Paul King, ASERT, Australia

[email protected], paulk_asertFull: http://www.slideshare.net/paulk_asert/make-tests-groovy

Page 2: Make Your Testing Groovy

Topics

Introduction

• Web Drivers

• Test Runners

• Non-web Drivers

• Other Tools

• Going Beyond

• Further Information

QCON 2010 - 2

© A

SE

RT

2006-2

010

Page 3: Make Your Testing Groovy

QCON 2010 - 3

© A

SE

RT

2006-2

010

What is Groovy?

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Java – boiler plate code+ mostly dynamic typing+ closures+ domain specific languages+ builders+ metaprogramming+ GDK library

Page 4: Make Your Testing Groovy

QCON 2010 - 4

© A

SE

RT

2006-2

010

What is Groovy?

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Productivity features of Ruby

But Java-like syntax(very low learning curve)

Leverage enterprise Java features

Suitable for non-core developers

Page 5: Make Your Testing Groovy

QCON 2010 - 5

© A

SE

RT

2006-2

010

What is Groovy?

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Productivity features of Ruby

But Java-like syntax(very low learning curve)

Leverage enterprise Java features

Suitable for non-core developers

Page 6: Make Your Testing Groovy

QCON 2010 - 6

© A

SE

RT

2006-2

010

What is Groovy?

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Productivity features of Ruby

But Java-like syntax(very low learning curve)

Leverage enterprise Java features

Suitable for non-core developers

Page 7: Make Your Testing Groovy

QCON 2010 - 7

© A

SE

RT

2006-2

010

What is Groovy?

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Productivity features of Ruby

But Java-like syntax(very low learning curve)

Leverage enterprise Java features

Suitable for non-core developers

Page 8: Make Your Testing Groovy

Growing Acceptance …

A slow and steady start but now gaining in

momentum, maturity and mindshare

Now free

Page 9: Make Your Testing Groovy

Growing Acceptance …

A slow and steady start but now gaining in

momentum, maturity and mindshare

Now free

Page 10: Make Your Testing Groovy

… Growing Acceptance …

© A

SE

RT

2006-2

010

QCON 2010 - 10http://www.leonardoborges.com/writings

What alternative JVM language are you using or intending to use

Page 11: Make Your Testing Groovy

… Growing Acceptance

© A

SE

RT

2006-2

010

QCON 2010 - 11

http://pollpigeon.com/jsf-grails-wicket/r/25665/

Page 12: Make Your Testing Groovy

… Growing Acceptance …

© A

SE

RT

2006-2

010

QCON 2010 - 12

Page 13: Make Your Testing Groovy

QCON 2010 - 13

© A

SE

RT

2006-2

010

The Landscape of JVM* Languages

Java bytecode calls

for static types

Dynamic features call

for dynamic types

mostly

dynamic

typing

*Java Virtual Machine

Page 14: Make Your Testing Groovy

QCON 2010 - 14

© A

SE

RT

2006-2

010

Groovy Starter

System.out.println("Hello, World!"); // supports Java syntaxprintln 'Hello, World!' // but can remove some syntax

String name = 'Guillaume' // Explicit typing/awarenessprintln "$name, I'll get the car." // Gstring (interpolation)

def longer = """${name}, the caris in the next row.""" // multi-line, implicit type

assert 0.5 == 1/2 // BigDecimal equals()assert 0.1 + 0.2 == 0.3 // and arithmetic

def printSize(obj) { // implicit/duck typingprint obj?.size() // safe dereferencing

}

def pets = ['ant', 'bee', 'cat'] // native list syntaxpets.each { pet -> // closure support

assert pet < 'dog' // overloading '<' on String} // or: for (pet in pets)...

Page 15: Make Your Testing Groovy

A Better Java...

QCON 2010 - 15

© A

SE

RT

2006-2

010

import java.util.List;import java.util.ArrayList;

class Erase {private List removeLongerThan(List strings, int length) {

List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {

String s = (String) strings.get(i);if (s.length() <= length) {

result.add(s);}

}return result;

}public static void main(String[] args) {

List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {

String s = (String) shortNames.get(i);System.out.println(s);

}}

}

This code

is valid

Java and

valid Groovy

Based on an

example by

Jim Weirich

& Ted Leung

Page 16: Make Your Testing Groovy

A Better Java...

QCON 2010 - 16

© A

SE

RT

2006-2

010

import java.util.List;import java.util.ArrayList;

class Erase {private List removeLongerThan(List strings, int length) {

List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {

String s = (String) strings.get(i);if (s.length() <= length) {

result.add(s);}

}return result;

}public static void main(String[] args) {

List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {

String s = (String) shortNames.get(i);System.out.println(s);

}}

}

This code

is valid

Java and

valid Groovy

Based on an

example by

Jim Weirich

& Ted Leung

Page 17: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 17

© A

SE

RT

2006-2

010

import java.util.List;import java.util.ArrayList;

class Erase {private List removeLongerThan(List strings, int length) {

List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {

String s = (String) strings.get(i);if (s.length() <= length) {

result.add(s);}

}return result;

}public static void main(String[] args) {

List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {

String s = (String) shortNames.get(i);System.out.println(s);

}}

}

Do the

semicolons

add anything?

And shouldn‟t

we us more

modern list

notation?

Why not

import common

libraries?

Page 18: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 18

© A

SE

RT

2006-2

010

class Erase {private List removeLongerThan(List strings, int length) {

List result = new ArrayList()for (String s in strings) {

if (s.length() <= length) {result.add(s)

}}return result

}

public static void main(String[] args) {List names = new ArrayList()names.add("Ted"); names.add("Fred")names.add("Jed"); names.add("Ned")System.out.println(names)Erase e = new Erase()List shortNames = e.removeLongerThan(names, 3)System.out.println(shortNames.size())for (String s in shortNames) {

System.out.println(s)}

}}

Page 19: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 19

© A

SE

RT

2006-2

010

class Erase {private List removeLongerThan(List strings, int length) {

List result = new ArrayList()for (String s in strings) {

if (s.length() <= length) {result.add(s)

}}return result

}

public static void main(String[] args) {List names = new ArrayList()names.add("Ted"); names.add("Fred")names.add("Jed"); names.add("Ned")System.out.println(names)Erase e = new Erase()List shortNames = e.removeLongerThan(names, 3)System.out.println(shortNames.size())for (String s in shortNames) {

System.out.println(s)}

}}

Do we need

the static types?

Must we always

have a main

method and

class definition?

How about

improved

consistency?

Page 20: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 20

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) {def result = new ArrayList()for (s in strings) {

if (s.size() <= length) {result.add(s)

}}return result

}

names = new ArrayList()names.add("Ted")names.add("Fred")names.add("Jed")names.add("Ned")System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())for (s in shortNames) {

System.out.println(s)}

Page 21: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 21

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) {def result = new ArrayList()for (s in strings) {

if (s.size() <= length) {result.add(s)

}}return result

}

names = new ArrayList()names.add("Ted")names.add("Fred")names.add("Jed")names.add("Ned")System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())for (s in shortNames) {

System.out.println(s)}

Shouldn‟t we

have special

notation for lists?

And special

facilities for

list processing?

Is „return‟

needed at end?

Page 22: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 22

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) {strings.findAll{ it.size() <= length }

}

names = ["Ted", "Fred", "Jed", "Ned"]System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())shortNames.each{ System.out.println(s) }

Page 23: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 23

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) {strings.findAll{ it.size() <= length }

}

names = ["Ted", "Fred", "Jed", "Ned"]System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())shortNames.each{ System.out.println(s) }

Is the method

now needed?

Easier ways to

use common

methods?

Are brackets

required here?

Page 24: Make Your Testing Groovy

...A Better Java...

QCON 2010 - 24

© A

SE

RT

2006-2

010

names = ["Ted", "Fred", "Jed", "Ned"]println namesshortNames = names.findAll{ it.size() <= 3 }println shortNames.size()shortNames.each{ println it }

Page 25: Make Your Testing Groovy

...A Better Java

QCON 2010 - 25

© A

SE

RT

2006-2

010

names = ["Ted", "Fred", "Jed", "Ned"]println namesshortNames = names.findAll{ it.size() <= 3 }println shortNames.size()shortNames.each{ println it }

[Ted, Fred, Jed, Ned]3TedJedNed

Page 26: Make Your Testing Groovy

Grapes / Grab

QCON 2010 - 26

© A

SE

RT

2006-2

010

// Google Collections example@Grab('com.google.collections:google-collections:1.0')import com.google.common.collect.HashBiMap

HashBiMap fruit =[grape:'purple', lemon:'yellow', lime:'green']

assert fruit.lemon == 'yellow'assert fruit.inverse().yellow == 'lemon'

Page 27: Make Your Testing Groovy

Better Design Patterns: Immutable...

• Java Immutable Class– As per Joshua Bloch

Effective Java

QCON 2010 - 27

© A

SE

RT

2006-2

010

public final class Punter {private final String first;private final String last;

public String getFirst() {return first;

}

public String getLast() {return last;

}

@Overridepublic int hashCode() {

final int prime = 31;int result = 1;result = prime * result + ((first == null)

? 0 : first.hashCode());result = prime * result + ((last == null)

? 0 : last.hashCode());return result;

}

public Punter(String first, String last) {this.first = first;this.last = last;

}// ...

// ...@Overridepublic boolean equals(Object obj) {

if (this == obj)return true;

if (obj == null)return false;

if (getClass() != obj.getClass())return false;

Punter other = (Punter) obj;if (first == null) {

if (other.first != null)return false;

} else if (!first.equals(other.first))return false;

if (last == null) {if (other.last != null)

return false;} else if (!last.equals(other.last))

return false;return true;

}

@Overridepublic String toString() {

return "Punter(first:" + first+ ", last:" + last + ")";

}

}

Page 28: Make Your Testing Groovy

...Better Design Patterns: Immutable...

QCON 2010 - 28

© A

SE

RT

2006-2

010

public final class Punter {private final String first;private final String last;

public String getFirst() {return first;

}

public String getLast() {return last;

}

@Overridepublic int hashCode() {

final int prime = 31;int result = 1;result = prime * result + ((first == null)

? 0 : first.hashCode());result = prime * result + ((last == null)

? 0 : last.hashCode());return result;

}

public Punter(String first, String last) {this.first = first;this.last = last;

}// ...

// ...@Overridepublic boolean equals(Object obj) {

if (this == obj)return true;

if (obj == null)return false;

if (getClass() != obj.getClass())return false;

Punter other = (Punter) obj;if (first == null) {

if (other.first != null)return false;

} else if (!first.equals(other.first))return false;

if (last == null) {if (other.last != null)

return false;} else if (!last.equals(other.last))

return false;return true;

}

@Overridepublic String toString() {

return "Punter(first:" + first+ ", last:" + last + ")";

}

}

• Java Immutable Class– As per Joshua Bloch

Effective Java

boilerplate

Page 29: Make Your Testing Groovy

...Better Design Patterns: Immutable

QCON 2010 - 29

© A

SE

RT

2006-2

010

@Immutable class Punter {String first, last

}

Page 30: Make Your Testing Groovy

What we will cover

QCON 2010 - 30

© A

SE

RT

2006-2

010

Unit Testing

Mock/interaction testing

State-based testing

Integration Testing

Acceptance Testing

Web drivers

Non-web drivers

Test runners

Techniques

Testing DSLs

ATDD/BDD

Data-driven

Logic-driven

Model-driven

Performance

testing

All-pairs &

combinations

Gpars

Page 31: Make Your Testing Groovy

Groovy's Value Add for Testing

• Unit testing– Built-in asserts, support for JUnit 3&4 and TestNG,

GroovyTestCase with shouldFail and other methods

– Built-in mocking and compatible with Java mocking

• Integration testing– Metaprogramming allows various kinds of IOC like

intercepting and hooking up of components

– Wealth of GDK methods for Ant, Processes, Files,

Threads, etc. make the automating part much simpler

• Acceptance Testing and Generally– Allows creation of English-like testing DSLs using

Closures, builders, metaprogramming

– Simpler syntax great for non hard-core testers

– Grapes make tests easier to share QCON 2010 - 31

© A

SE

RT

2006-2

010

Page 32: Make Your Testing Groovy

Unit Testing Value Add (Details)

• State-based testing– Asserts, PowerAsserts,

GroovyTestCase, support for JUnit 3 & 4 and TestNG

– Easy to do data-driven style tests with improved file

manipulation capabilities

• Mocking/interaction-based testing– Built-in metaprogrammed mocking

– Compatible with popular Java mocking frameworks

and specialised Groovy mocking frameworks:

JMock, EasyMock, JMockit, Mockito, GMock, Spock

• Coverage– Cobertura and Clover support at source code level

– Other Java tools

QCON 2010 - 32

© A

SE

RT

2006-2

010

assert (3 - 1) * 2 == 'cat'.size()

| | | |

2 4 false 3

Page 33: Make Your Testing Groovy

Groovy‟s Power Assert

QCON 2010 - 33

© A

SE

RT

2006-2

010

def pets = ['dog', 'cat', 'koala', 'goldfish']assert pets.findAll{ it.size() > 3 }[0] == 'goldfish'

def pets = ['犬', '猫', 'コアラ', '金魚']assert pets.findAll{ it.size() > 1 }[0] == '金魚'

Assertion failed:

assert pets.findAll{ it.size() > 1 }[0] == '金魚'| | | || [コアラ, 金魚] | false[犬, 猫, コアラ, 金魚] コアラ

Assertion failed:

assert pets.findAll{ it.size() > 3 }[0] == 'goldfish'| | | || [koala, goldfish] | false[dog, cat, koala, goldfish] koala

Page 34: Make Your Testing Groovy

Don't Forget: Key Testing Practices • Use testing DSL’s

– Use Acceptance Test/Behavior Driven Development (ATDD/BDD)

• Look to move up the testing stack– It used to be all about the driver

– Now the driver is hidden in the framework or tool stack

• Apply good testing practices– Pareto analysis, bug clusters, mutation testing, test early

– All pairs/equivalence partitions/orthogonal array testing

– Risk-based test selection, coding for testability, use CI

– Boundary value analysis, defensive programming

• Plug and play testing tools– Run different drivers with different runners and different tools

• Complement automated tests with exploration

• Expand testing scope– Test environment readiness, test deployments QCON 2010 - 34

© A

SE

RT

2006-2

010

Page 35: Make Your Testing Groovy

Groovy and Testing Tool Spectrum*

QCON 2010 - 35

© A

SE

RT

2006-2

010

Database

Drivers

DbUnit

DataSets

SqlUnit

groovy.sql

JPA

JDO

BigTable

JDBC

SOAP /

REST

Drivers

GroovyWS

XML-RPC

CXF

Axis2

JAX-WS

JAX-RS

Utilities

AllPairs, Combinations

Polyglot languages

Logic programming

Threads, Parallel /

Concurrency libraries

Data-driven libraries

Networking libraries

XML Processing

Read/write files /

Excel / Word / CSV

Reporting, Logging

Other

Drivers

FEST

Email

FTP

AntUnit

Telnet

SSH

ExecWindowLicker

Tools

iTest2, SoapUI, Twist,

IDEs, JMeter, Text

editors, Recorders,

Sahi, Build Tools, CI

Web

Drivers

WebTest

WebDriver

JWebUnit

Tellurium

Selenium

HtmlUnit

Watij

HttpBuilder

Cyberneko

Runners

Native Groovy, JUnit, TestNG, Spock, EasyB,

JBehave, Cucumber, Robot Framework, SLIM

* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually

Page 36: Make Your Testing Groovy

Topics

• Introduction

Web Drivers

• Test Runners

• Non-web Drivers

• Other Tools

• Going Beyond

• Further Information

QCON 2010 - 36

© A

SE

RT

2006-2

010

Page 37: Make Your Testing Groovy

QCON 2010 - 37

© A

SE

RT

2006-2

010

Concept

Driver

Runner<webtest name="myTest">

<steps>

<invoke

description="get Login Page"

url="login" />

<verifyTitle

description="we should see the login title"

text="Login Page" />

</steps>

</webtest>

Web Server

HTTP Request / Response

HTTP Request / Response

Read

Script

Manual

Automated

Page 38: Make Your Testing Groovy

Driver Category• Real browser invoker

– Runs on platform

supported by real

browser

– May need multiple

platforms, e.g. IE6/IE7

– Uses actual JavaScript

engine

– Can be easier to use

with test recorders

– Automation

capabilities differ

across browsers

– Can typically get to all

aspects of browser

• Browser Emulators

– Can simulate multiple

browsers

– Less platform

restrictions

– Good for CI

– Easier to not download

images, resources

– Ability to optimise

JavaScript interactions

– More extensible

– Ability to disable

JavaScript

– Scope for parallelism

QCON 2010 - 38

© A

SE

RT

2006-2

010

Page 39: Make Your Testing Groovy

QCON 2010 - 39

© A

SE

RT

2006-2

010

Application under Test

Page 40: Make Your Testing Groovy

Native Groovy...

• Access URLs

• Built-in XML parsing

• Built-in friendly regular expression syntax

• Even for advanced cases, there is friendly

access to low-level things:– Sockets, Processes

– Databases and other things

– Files

• Huge range of Java libraries– PDF

– Reading, writing Excel

QCON 2010 - 40

© A

SE

RT

2006-2

010

Page 41: Make Your Testing Groovy

...Native Groovy...

• Useful URL methods

• Simple enough for GAE– For public sites

– Can share test scripts easily

– No setup required

QCON 2010 - 41

© A

SE

RT

2006-2

010

def html = new URL('http://localhost:8080').text

assert html.contains('<title>Welcome to SimpBlog</title>')

html.find(~'<title>(.*)</title>') { all, title ->assert title == 'Welcome to SimpBlog'

}

Page 42: Make Your Testing Groovy

...Native Groovy...

• Built-in XML Parsing

QCON 2010 - 42

© A

SE

RT

2006-2

010

def page = newXmlSlurper().parse('http://localhost:8080/viewPost?id=1')

assert page.body.h1.text().contains('Tis the season')assert page.body.h3[1].text() == 'Category: Home'assert page.body.h3[2].text() == 'Author: Bart'assert page.body.table.tr.td.p.text() ==

"Aren't we forgeting the true meaning of Christmas? You know, the birth of Santa."

Page 43: Make Your Testing Groovy

...Native Groovy

• Easy access to Java libraries

QCON 2010 - 43

© A

SE

RT

2006-2

010

@Grab('nekohtml:nekohtml:1.9.6.2')import org.cyberneko.html.parsers.SAXParser

def parser = new XmlSlurper(new SAXParser())def page = parser.parse('http://localhost:8080/viewPost?id=1')assert page.BODY.H1.text().contains('Tis the season')assert page.BODY.H3[1].text() == 'Category: Home'assert page.BODY.H3[2].text() == 'Author: Bart'assert page.BODY.TABLE.TR.TD.P.text() ==

"Aren't we forgeting the true meaning of Christmas? You know, the birth of Santa."

Page 44: Make Your Testing Groovy

• Builder for Http interactions– Flexible: bogus posts, response codes, JSON, non-

HTML

HttpBuilder

© A

SE

RT

2006-2

010

@Grab(group='org.codehaus.groovy.modules.http-builder',module='http-builder', version='0.5.0-RC1')

import groovyx.net.http.*import static groovyx.net.http.ContentType.URLENC

def http = new HTTPBuilder('http://localhost:8080')def postBody = [title:'Bart was here (and so was HttpBuilder)',

content:'Cowabunga Dude!', author:'1', category:'3']http.post(path:'/addPost', body: postBody,

requestContentType: URLENC) { resp, html ->assert resp.contentType == 'text/html'assert resp.status == 200assert html.BODY.H1.text().matches('Post.*: Bart was here.*')assert html.BODY.H3[1].text() == 'Category: Home'assert html.BODY.H3[2].text() == 'Author: Bart'assert html.BODY.TABLE.TR.TD.P.text() == 'Cowabunga Dude!'

}QCON 2010 - 44

Page 45: Make Your Testing Groovy

HtmlUnit• 100% Java-based headless browser emulator

– Can test any Web site: Java, .Net, PHP, Rails, ...

• Open Source– Apache 2 license

– Hosted at SourceForge

– 7 committers (3 very active)

– Very mature

• Useful for:– Integration and acceptance testing

– Screen scraping, deployment automation, ...

• Used by other drivers:– Canoo WebTest , JWebUnit , WebDriver , JSFUnit , Celerity

• Special features:– Easy ajax mode, emulation of multiple browsers

QCON 2010 - 45

© A

SE

RT

2006-2

010

Page 46: Make Your Testing Groovy

HtmlUnit Features...• Support for the HTTP and HTTPS protocols

• Support for cookies

• Ability to specify whether failing responses from

the server should throw exceptions or should be

returned as "error" pages

• Support for submit methods POST and GET– As well as HEAD, DELETE, ...

• Ability to customize the request headers being

sent to the server

• Support for HTML responses– Wrapper for HTML pages that provides easy access to all

information contained inside them

– Support for submitting forms and clicking links

– Support for walking the DOM model of HTML documentsQCON 2010 - 46

© A

SE

RT

2006-2

010

Page 47: Make Your Testing Groovy

...HtmlUnit Features• Proxy server support

• Support for basic and NTLM authentication

• Excellent JavaScript support– jQuery 1.2.6: Full support

– MochiKit 1.4.1: Full support

– GWT 1.6.4: Full support

– Sarissa 0.9.9.3: Full support

– MooTools 1.2.1: Full support

– Prototype 1.6.0: Very good support

– Ext JS 2.2: Very good support

– Dojo 1.0.2: Good support

– YUI 2.3.0: Good support

QCON 2010 - 47

© A

SE

RT

2006-2

010

Page 48: Make Your Testing Groovy

HtmlUnit: Testing New Blog Post...

QCON 2010 - 48

© A

SE

RT

2006-2

010

@Grab('net.sourceforge.htmlunit:htmlunit:2.6')import com.gargoylesoftware.htmlunit.WebClient

def client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')// check page titleassert 'Welcome to SimpBlog' == page.titleText

// fill in blog entry and post itdef form = page.getFormByName('post')form.getInputByName('title').

setValueAttribute('Bart was here (and so was HtmlUnit)')form.getSelectByName('category').getOptions().find{

it.text == 'Home' }.setSelected(true)form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()

...

Page 49: Make Your Testing Groovy

...HtmlUnit: Testing New Blog Post

QCON 2010 - 49

© A

SE

RT

2006-2

010

...

// check blog post detailsassert result.getElementsByTagName('h1').item(0).

textContent.matches('Post.*: Bart was here.*')def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home'assert h3headings.item(2).textContent == 'Author: Bart'

// expecting:// <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.getFirstChild()assert para.textContent == 'Cowabunga Dude!'

Page 50: Make Your Testing Groovy

QCON 2010 - 50

© A

SE

RT

2006-2

010

Canoo WebTest• Description– Open source tool for automated testing of web applications

– Declarative approach in XML or testing DSL in Groovy

– Has Test Recorder

– Excellent reporting options

– Ant-based under the covers

<target name="login" >

<testSpec name="normal" >

&config;

<steps>

<invoke stepid="get Login Page"

url="login.jsp" />

<verifytitle stepid="we should see the login title"

text="Login Page" />

<setinputfield stepid="set user name"

name="username"

value="scott" />

<setinputfield stepid="set password"

name="password"

value="tiger" />

<clickbutton stepid="Click the submit button"

label="let me in" />

<verifytitle stepid="Home Page follows if login ok"

text="Home Page" />

</steps>

</testSpec>

</target>

Page 51: Make Your Testing Groovy

Canoo WebTest Features• Strongly encourages declarative testing

– Supports testing DSLs, test structuring and reuse through

macrodefs and imports for XML flavor &

methods and closures for Groovy flavor

• Extensive support for HTML pages– Including JavaScript

• Also supports other MIME types– Generically as binary streams

– Special support for PDF, Excel, Emails

• Ant heritage provides easy IDE/CI hooks

• Excellent Documentation

• Excellent Community

• Eats own dog food– high quality codebase

QCON 2010 - 51

© A

SE

RT

2006-2

010

Page 52: Make Your Testing Groovy

Canoo WebTest Steps...

QCON 2010 - 52

© A

SE

RT

2006-2

010

Page 53: Make Your Testing Groovy

...Canoo WebTest Steps...

© A

SE

RT

2006-2

010

QCON 2010 - 53

Page 54: Make Your Testing Groovy

...Canoo WebTest Steps...

QCON 2010 - 54

© A

SE

RT

2006-2

010

Page 55: Make Your Testing Groovy

...Canoo WebTest Steps

QCON 2010 - 55

© A

SE

RT

2006-2

010

Page 56: Make Your Testing Groovy

WebTest: Testing New Blog Post...

QCON 2010 - 56

© A

SE

RT

2006-2

010

<webtest name="Testing Posting a new Blog Entry">

<invoke url="http://localhost:8080/" description="Home Page"/>

<verifyTitle text="Welcome to SimpBlog"/>

<group description="Post New Blog Entry">

<clickLink label="New Blog Entry"/>

<setInputField name="title"

value="Bart was here (and so was WebTest)"/>

<setSelectField name="category" text="School"/>

<setInputField name="content" value="Cowabunga Dude!"/>

<clickButton name="btnPost"/>

</group>

...

Page 57: Make Your Testing Groovy

...WebTest: Testing New Blog Post...

QCON 2010 - 57

© A

SE

RT

2006-2

010

...

<group description="Check Blog Post">

<verifyElementText type="h1" regex="true"

text="Post.*: Bart was here.*"/>

<verifyXPath xpath="//h3[2]/text()" text="Category: School"/>

<verifyXPath xpath="//h3[3]/text()" text="Author: Bart"/>

<verifyElementText type="p" text="Cowabunga Dude!"/>

</group>

<groovy>

println "Test run at: ${new Date()}"

</groovy>

</webtest>

Page 58: Make Your Testing Groovy

...WebTest: Testing New Blog Post...

QCON 2010 - 58

© A

SE

RT

2006-2

010

ant.webtest(name: 'Test SimpBlog') {invoke url: "http://localhost:8080/",

description: "Home Page"verifyTitle text: "Welcome to SimpBlog"group description: "Post New Blog Entry", {clickLink label: "New Blog Entry"setInputField name: "title",

value: "Bart was here (and so was WebTest with Groovy)"setSelectField name: "category", text: "School"setInputField name: "content", value: "Cowabunga Dude!"clickButton name: "btnPost"

}group description: "Check Blog Post", {verifyElementText type: "h1", regex: "true",

text: "Post.*: Bart was here.*"verifyXPath xpath: "//h3[2]/text()", text: "Category: School"verifyXPath xpath: "//h3[3]/text()", text: "Author: Bart"verifyElementText type: "p", text: "Cowabunga Dude!"

}}

Page 59: Make Your Testing Groovy

...WebTest: Testing New Blog Post...

QCON 2010 - 59

© A

SE

RT

2006-2

010

Page 60: Make Your Testing Groovy

...WebTest: Testing New Blog Post

QCON 2010 - 60

© A

SE

RT

2006-2

010

Page 61: Make Your Testing Groovy

QCON 2010 - 61

© A

SE

RT

2006-2

010

Firefox Recorder

Page 62: Make Your Testing Groovy

Watij

• Description– Java API that provides control and automation of

Internet Explorer

– Supports actions like navigating, clicking links, filling

out forms, etc.

– Also supports more complex actions like file

downloading and uploading, popup windows and

dialogs, and screen captures

• Special Features– Ability to work with IE interactively

– Can attach to an existing browser session

– Special browser commands, e.g. ie.fullScreen(true)

– Handles child browsers and popup dialogs

QCON 2010 - 62

© A

SE

RT

2006-2

010

Page 63: Make Your Testing Groovy

Watij Features

• Finders– tag(String tagName)

– attribute(String name, String value)

– index(int index)

– text(String text)

– name(String value)

– value(String value)

– caption(String value)

– id(String value)

– title(String value)

– alt(String value)

– src(String value)

– action(String value)

– method(String value)

– url(String value)

– href(String value)

– xpath(String expression)QCON 2010 - 63

© A

SE

RT

2006-2

010

Page 64: Make Your Testing Groovy

Watij: Testing New Blog Post...

QCON 2010 - 64

© A

SE

RT

2006-2

010

import watij.runtime.ie.IEimport static watij.finders.SymbolFactory.*import static watij.finders.FinderFactory.*

def ie = new IE()ie.start('http://localhost:8080/postForm')

// check page titleassert ie.title() == 'Welcome to SimpBlog'

// fill in query form and submit itie.textField(name, 'title').

set('Bart was here (and so was Watij)')ie.textField(name, 'content').set('Cowabunga dude!')ie.selectList(name, "category").

option(text, "Home").select()ie.button(name, 'btnPost').click()

...

Page 65: Make Your Testing Groovy

...Watij: Testing New Blog Post...

QCON 2010 - 65

© A

SE

RT

2006-2

010

...

// check entered post is being displayedassert ie.htmlElement(tag, 'H1').text().

matches('Post.*: Bart was here.*')def h3headers = ie.htmlElements(tag, 'H3')assert h3headers.get(1).text() == 'Category: Home'assert h3headers.get(2).text() == 'Author: Bart'

// try a more advanced finder// content is at: //TABLE/TBODY/TR/TD/Pdef row = ie.htmlElement(xpath('//TABLE/TBODY/TR'))assert row.cell(0).htmlElement(tag, 'P').text() ==

'Cowabunga dude!'

ie.close()

Page 66: Make Your Testing Groovy

...Watij: Testing New Blog Post

QCON 2010 - 66

© A

SE

RT

2006-2

010

Page 67: Make Your Testing Groovy

Selenium...

• Description– Tools to help

automate testing

for web-based

applications

– Support for

running tests on

multiple browser

platforms

• Components– Selenium Core

– Selenium IDE

Selenium RC

– Selenium Grid

QCON 2010 - 67

© A

SE

RT

2006-2

010

Source: http://seleniumhq.org/projects/remote-control/

Our focus

Page 68: Make Your Testing Groovy

...Selenium

QCON 2010 - 68

© A

SE

RT

2006-2

010

Source: http://seleniumhq.org/docs/01_introducing_selenium.html

Page 69: Make Your Testing Groovy

Selenium: Testing New Blog Post...

QCON 2010 - 69

© A

SE

RT

2006-2

010

import com.thoughtworks.selenium.DefaultSeleniumimport org.openqa.selenium.server.SeleniumServer

// start auxiliary serverdef server = new SeleniumServer()server.start()

// uncomment one of below//def browser = "*iexplore"//def browser = "*firefox3" // if Firefox already in your path//def browser = "*firefox3 C:/Program Files/Mozilla Firefox/firefox.exe"def browser = "*firefox3 C:/Program Files (x86)/Mozilla Firefox/firefox.exe"

def selenium = new DefaultSelenium("localhost", 4444, browser,"http://localhost:8080")

selenium.start()

...

Page 70: Make Your Testing Groovy

...Selenium: Testing New Blog Post

QCON 2010 - 70

© A

SE

RT

2006-2

010

...

// post blogselenium.open "/postForm"selenium.type "title", "Bart was here (and so was Selenium)"selenium.select "category", "Home"selenium.type "content", "Cowabunga Dude!"selenium.click "btnPost"selenium.waitForPageToLoad "5000"

// checksassert selenium.isTextPresent('regex:Post.*: Bart was here')assert selenium.isElementPresent('//h3[text()="Author: Bart"]')assert selenium.isElementPresent('//h3[text()="Category: Home"]')assert selenium.isElementPresent(

'//table//tr/td/p[text()="Cowabunga Dude!"]')

selenium.stop()server.stop()

Page 71: Make Your Testing Groovy

Selenium IDE...

QCON 2010 - 71

© A

SE

RT

2006-2

010

Features:

• Easy record and playback

• Intelligent field selection will use

IDs, names, or XPath as needed

• Autocomplete for all common

Selenium commands

• Walk through tests

• Debug and set breakpoints

• Save tests as HTML, Ruby

scripts, or any other format

• Support for Selenium user-

extensions.js file

• Option to automatically assert the

title of every page

Page 72: Make Your Testing Groovy

...Selenium IDE

QCON 2010 - 72

© A

SE

RT

2006-2

010

more info: http://limcheekin.blogspot.com/2009/07/behavior-driven-development-generating.html

Page 73: Make Your Testing Groovy

Selenium Other Tools...

QCON 2010 - 73

© A

SE

RT

2006-2

010

Page 74: Make Your Testing Groovy

...Selenium Other Tools...

QCON 2010 - 74

© A

SE

RT

2006-2

010

Page 75: Make Your Testing Groovy

...Selenium Other Tools

QCON 2010 - 75

© A

SE

RT

2006-2

010

Source: http://selenium-grid.seleniumhq.org/how_it_works.html

Page 76: Make Your Testing Groovy

WebDriver

• Description– Simple API to drive both real browsers

• for testing javascript heavy apps

– and a pure 'in memory' emulator solution

• for faster testing of simpler apps

• uses HtmlUnit under the covers in emulator mode

– Represents next generation of Selenium RC

• though merging into Selenium may happen under the covers

if you are a Selenium user

– Roadmap has plans to leverage some advanced

Selenium like features

• RemoteWebDriver

• FarmedWebDriver (think Selenium Grid)

QCON 2010 - 76

© A

SE

RT

2006-2

010

Page 77: Make Your Testing Groovy

WebDriver: Testing New Blog Post...

QCON 2010 - 77

© A

SE

RT

2006-2

010

import org.openqa.selenium.Byimport org.openqa.selenium.htmlunit.HtmlUnitDriver

def driver = new HtmlUnitDriver()driver.get('http://localhost:8080/postForm')assert driver.title == 'Welcome to SimpBlog'

// fill in query form and submit itdriver.findElement(By.name('title')).

sendKeys('Bart was here (and so was WebDriver)')driver.findElement(By.name('content')).

sendKeys('Cowabunga dude!')def select = driver.findElement(By.name('category'))select.findElements(By.tagName("option")).find{

it.text == 'Home' }.setSelected()driver.findElement(By.name('btnPost')).click()

...

Page 78: Make Your Testing Groovy

...WebDriver: Testing New Blog Post

QCON 2010 - 78

© A

SE

RT

2006-2

010

...

assert driver.findElement(By.tagName("h1")).text.matches('Post.*: Bart was here.*')

def h3headers = driver.findElements(By.tagName("h3"))assert h3headers[1].text == 'Category: Home'assert h3headers[2].text == 'Author: Bart'

// try a more advanced finder// content is at: //TABLE/TBODY/TR/TD/Pdef row = driver.findElement(By.xpath("//table/tbody/tr"))def col = row.findElement(By.tagName("td"))def para = col.findElement(By.tagName("p"))assert para.text == 'Cowabunga dude!'

Page 79: Make Your Testing Groovy

Tellurium

• Description– built on top of Selenium but tries to solve several

shortcomings

– "record and reply" style, difficult to refactor and

maintain, so instead define UI components

declaratively then write tests in terms of UI

– Provides many predefined UI objects for you to use

directly, such as Button, CheckBox, InputBox,

Selector, TextBox, and Table but also ability to write

your own custom UI objects

– Supports advanced locating mechanisms: composite

locator, "group locating"

– Supports testing DSL

– Supports data-driven tests

QCON 2010 - 79

© A

SE

RT

2006-2

010

Page 80: Make Your Testing Groovy

Architecture

QCON 2010 - 80

© A

SE

RT

2006-2

010

Source: http://code.google.com/p/aost/wiki/Introduction

Page 81: Make Your Testing Groovy

Tellurium Example

• Selenium:

• Tellurium UI:

• Tellurium DSL Test:

QCON 2010 - 81

© A

SE

RT

2006-2

010

selenium.type("//input[@title='Google Search']", input)selenium.click("//input[@name='btnG' and @type='submit']")

ui.Container(uid: "google_start_page",clocator: [tag: "td"], group: "true") {

InputBox(uid: "searchbox",clocator: [title: "Google Search"])

SubmitButton(uid: "googlesearch",clocator: [name: "btnG", value: "Google Search"])

}

type "google_start_page.searchbox", inputclick "google_start_page.googlesearch"

Page 82: Make Your Testing Groovy

Tellurium: Testing New Blog Post...

QCON 2010 - 82

© A

SE

RT

2006-2

010

ui.UrlLink(uid: "create_new_post",clocator: [tag:'a', text: "New Blog Entry"])

ui.Form(uid: "blogform",clocator: [tag: 'form', name:'post'], group: "true") {

InputBox(uid: "title", clocator: [name: "title"])InputBox(uid: "content", clocator: [tag:'textarea', name: "content"])Selector(uid: "category", clocator: [name: "category"])Selector(uid: "author", clocator: [name: "author"])SubmitButton(uid: "post_button",

clocator: [name: 'btnPost', value: "Create Post"])}

ui.TextBox(uid: 'main_header', clocator: [tag: 'h1'])ui.TextBox(uid: 'category_header', clocator: [tag: 'h3', position: '2'])ui.TextBox(uid: 'author_header', clocator: [tag: 'h3', position: '3'])ui.Container(uid: 'table', clocator: [tag: 'table']) {

ui.TextBox(uid: 'content_para', locator: '//tr/td/p')}

Page 83: Make Your Testing Groovy

...Tellurium: Testing New Blog Post

QCON 2010 - 83

© A

SE

RT

2006-2

010

openUrl "http://localhost:8080/"click "create_new_post"waitForPageToLoad 5000assert title == 'Welcome to SimpBlog'

// post blogtype "blogform.title", "Bart was here (and so was Tellurium)"selectByLabel "blogform.category", "Home"selectByLabel "blogform.author", "Bart"type "blogform.content", "Cowabunga Dude!"click "blogform.post_button"waitForPageToLoad 5000

// check contentsassert getText('main_header').matches('Post.*: Bart was here.*')assert getText('category_header') == 'Category: Home'assert getText('author_header') == 'Author: Bart'assert getText('table.content_para') == 'Cowabunga Dude!'shutDown

Page 84: Make Your Testing Groovy

TrUMP IDE

QCON 2010 - 84

© A

SE

RT

2006-2

010

Page 85: Make Your Testing Groovy

JWebUnit...

• Description– Java-based testing framework for web applications

– Intention is to provide a high-level "driver" Java API

– Wraps existing testing frameworks such as HtmlUnit

and Selenium with a unified, simple testing interface

– Support includes navigation via links, form entry and

submission, validation of table contents, and other

verification steps

– Includes some runner capabilities

– Useful in that it allows you to switch between

different lower level drivers without re-writing your

tests

QCON 2010 - 85

© A

SE

RT

2006-2

010

Page 86: Make Your Testing Groovy

...JWebUnit

QCON 2010 - 86

© A

SE

RT

2006-2

010

Source: http://jwebunit.sourceforge.net/

Page 87: Make Your Testing Groovy

JWebUnit: Testing New Blog Post

QCON 2010 - 87

© A

SE

RT

2006-2

010

import net.sourceforge.jwebunit.junit.*

class TestSimpBlog extends WebTestCase {void setUp() { setBaseUrl("http://localhost:8080") }

void testPostBlog() {beginAt "/postForm"assertTitleEquals "Welcome to SimpBlog"setTextField "title", "Bart was here (and so was JWebUnit)"setTextField "content", "Cowabunga Dude!"selectOption "category", "Home"clickButtonWithText "Create Post"assert getElementByXPath('//H1').textContent.matches('Post.*: Bart was here.*')

def h3headings = getElementsByXPath('//H3')assert h3headings[1].textContent == "Category: Home"assert h3headings[2].textContent == "Author: Bart"def cell = getElementByXPath('//TABLE//TR/TD')assert cell.children[0].textContent == 'Cowabunga Dude!'

}}

Page 88: Make Your Testing Groovy

"Create a Testing API"

QCON 2010 - 88

© A

SE

RT

2006-2

010

def testMethod() {// tests go here (or use SLIM, Cucumber, ...)postAndCheck('Work', 'Bart', 'Cowabunga Dude!')

}

def postAndCheck(category, author, content) {// ...// HtmlUnit or Selenium or WebDriver or ...// Details in here ...// ...

}

Page 89: Make Your Testing Groovy

Topics

• Introduction

• Web Drivers

Test Runners

• Non-web Drivers

• Other Tools

• Going Beyond

• Further Information

QCON 2010 - 89

© A

SE

RT

2006-2

010

Page 90: Make Your Testing Groovy

Native Groovy

• Groovy has a friendly ‘==‘

• Built-in assert

• Scripts are low ceremony

• By utilising @Grab are easy to share

• Many in-built testing capabilities are

accessible even from scripts

• Easy to version control or treat like

operating system scripts

• Out of the box detection of JUnit and

TestNG tests

QCON 2010 - 90

© A

SE

RT

2006-2

010

Page 91: Make Your Testing Groovy

Native Groovy

• Groovy has a friendly ‘==‘

• Built-in assert

• Scripts are low ceremony

• By utilising @Grab are easy to share

• Many in-built testing capabilities are

accessible even from scripts

• Easy to version control or treat like

operating system scripts

• Out of the box detection of JUnit and

TestNG tests

QCON 2010 - 91

© A

SE

RT

2006-2

010

Page 92: Make Your Testing Groovy

QCON 2010 - 92

© A

SE

RT

2006-2

010

Built-in JUnit 3...

– Test Runners

Page 93: Make Your Testing Groovy

...Built-in JUnit 3...

QCON 2010 - 93

© A

SE

RT

2006-2

010

...def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(

'Bart was here (and so was HtmlUnit)')form.getSelectByName('category').getOptions().find {

it.text == 'Home' }.setSelected(true)form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()

// check blog post detailsassert result.getElementsByTagName('h1').item(0).

textContent.matches('Post.*: Bart was here.*')def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home'assert h3headings.item(2).textContent == 'Author: Bart'

// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.getFirstChild()assert para.textContent == 'Cowabunga Dude!'

}}

Page 94: Make Your Testing Groovy

...Built-in JUnit 3

QCON 2010 - 94

© A

SE

RT

2006-2

010

Page 95: Make Your Testing Groovy

GroovyTestCase Tests

• Like JUnit but with some enhancements– Additional assert methods

– fewer imports

– clean shouldFail syntax

QCON 2010 - 95

© A

SE

RT

2006-2

010

class TestSimpBlogGUnit extends GroovyTestCase {def page

void setUp() {// ...

}

void testBartWasHere() {// ...

...

Page 96: Make Your Testing Groovy

QCON 2010 - 96

© A

SE

RT

2006-2

010

JUnit 4.X

import org.junit.*

class TestSimpBlogJUnit4 {def page

@Beforevoid setUp() {

// ...}

@Testvoid bartWasHere() {

// ...

• Groovy distributions from 1.7 include JUnit 4

• Automatically invokes text runner if needed

• Example uses HtmlUnit driver (not shown)

Page 97: Make Your Testing Groovy

QCON 2010 - 97

© A

SE

RT

2006-2

010

JUnit 4.X Parameterized Tests

@RunWith(Parameterized)class TestSimpBlogJUnit4DD {

def page, author, title, category, content

TestSimpBlogJUnit4DD(author, title,category, content) {

this.author = authorthis.title = titlethis.category = categorythis.content = content

}

@Parameters static data() {return [

['Bart', 'Title 1', 'Home', 'Content 1'],['Homer', 'Title 2', 'Work', 'Content 2'],['Marge', 'Title 3', 'Food', 'Content 3']

].collect{ it as String[] }}

...

Page 98: Make Your Testing Groovy

TestNG

QCON 2010 - 98

© A

SE

RT

2006-2

010

• Groovy automatically invokes text runner if run

as a script

• Example shows grouping, driver not shown

import org.testng.annotations.*

class TestSimpBlogTestNG {def page

@BeforeClassvoid setUp() {

// ...}

@Test(groups = [ "slow" ])void bartWasHere() {

// ...

Page 99: Make Your Testing Groovy

TestNG Data Driven

QCON 2010 - 99

© A

SE

RT

2006-2

010

import org.testng.annotations.*

class TestSimpBlogTestNGDD {// ...

@DataProvider(name='SimpBlogDataProvider')Object[][] data() {

return [['Bart', 'Title 1', 'Home', 'Content 1'],['Homer', 'Title 2', 'Work', 'Content 2'],['Marge', 'Title 3', 'Food', 'Content 3']

].collect{ it as Object[] } as Object[]}

@Test(dataProvider = "SimpBlogDataProvider")void bartWasHere(author, title, category, content) {

// ...

Page 100: Make Your Testing Groovy

Spock Testing Framework...

QCON 2010 - 100

© A

SE

RT

2006-2

010

Page 101: Make Your Testing Groovy

...Spock Testing Framework

QCON 2010 - 101

© A

SE

RT

2006-2

010

• Testing framework for Java and Groovy

• Highly expressive specification language– No assertion API

– No record &

replay

mocking API

– No

superfluous

annotations

– Meaningful

assert error

messages

– Extensible

– Compatible

with JUnit

reportingwise

@Speck@RunWith(Sputnik)class PublisherSubscriberSpeck {def "events are received by all subscribers"() {def pub = new Publisher()def sub1 = Mock(Subscriber)def sub2 = Mock(Subscriber)pub.subscribers << sub1 << sub2

when:pub.send("event")

then:1 * sub1.receive("event")1 * sub2.receive("event")

}}

Page 102: Make Your Testing Groovy

Spock Example...

QCON 2010 - 102

© A

SE

RT

2006-2

010

import com.gargoylesoftware.htmlunit.WebClientimport spock.lang.*import org.junit.runner.RunWith

@Speck ()@RunWith (Sputnik)class TestSimpBlogSpock {def page, subheadings, para, form, result

@Unroll("When #author posts a #category blog with content '#content' it should succeed"def "when creating a new blog entry"() {given:page = new WebClient().getPage('http://localhost:8080/postForm')form = page.getFormByName('post')

when:form.getInputByName('title').setValueAttribute("$author was here (and so was Spock)"form.getSelectByName('category').getOptions().find { it.text == category }.form.getSelectByName('author').getOptions().find { it.text == author }.setSelectedform.getTextAreaByName('content').setText(content)result = form.getInputByName('btnPost').click()subheadings = result.getElementsByTagName('h3')para = result.getByXPath('//TABLE//TR/TD/P')[0]

...

Page 103: Make Your Testing Groovy

...Spock Example

QCON 2010 - 103

© A

SE

RT

2006-2

010

...then:page.titleText == 'Welcome to SimpBlog'result.getElementsByTagName('h1').item(0).textContent.matches("Post.*: $author was here.*"subheadings.item(1).textContent == "Category: $category"subheadings.item(2).textContent == "Author: $author"

and:para.textContent == content

where:author << ['Bart', 'Homer', 'Lisa']category << ['Home', 'Work', 'Food']content << ['foo', 'bar', 'baz']

}}

// Optional use of 'and:'

Page 104: Make Your Testing Groovy

QCON 2010 - 104

© A

SE

RT

2006-2

010

EasyB

• Description: BDD, Rspec-like testing librarynarrative 'segment flown', {

as_a 'frequent flyer'i_want 'to accrue rewards points for every segment I fly'so_that 'I can receive free flights for my dedication to the airline'

}

scenario 'segment flown', {given 'a frequent flyer with a rewards balance of 1500 points'when 'that flyer completes a segment worth 500 points'then 'that flyer has a new rewards balance of 2000 points'

}

scenario 'segment flown', {given 'a frequent flyer with a rewards balance of 1500 points', {

flyer = new FrequentFlyer(1500)}when 'that flyer completes a segment worth 500 points', {

flyer.fly(new Segment(500))}then 'that flyer has a new rewards balance of 2000 points', {

flyer.pointsBalance.shouldBe 2000}

}

Page 105: Make Your Testing Groovy

EasyB Example ...

• When run will be marked as pending– perfect for ATDD

QCON 2010 - 105

© A

SE

RT

2006-2

010

scenario "Bart posts a new blog entry", {given "we are on the create blog entry page"when "I have entered 'Bart was here' as the title"and "I have entered 'Cowabunga Dude!' into the content"and "I have selected 'Home' as the category"and "I have selected 'Bart' as the author"and "I click the 'Create Post' button"then "I expect the entry to be posted"

}

Page 106: Make Your Testing Groovy

...EasyB Example...

QCON 2010 - 106

© A

SE

RT

2006-2

010

description "Post Blog Entry Feature"

narrative "for feature", {as_a "Blogger"i_want "to be able to post a blog"so_that "I can keep others informed"

}

before "posting blog", {given "we are on the create blog entry page", {

webClient = new com.gargoylesoftware.htmlunit.WebClient()page = webClient.getPage('http://localhost:8080/postForm')

}}

scenario "Bart was here blog", {

when "I have entered 'Bart was here' as the title", {form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(

'Bart was here (and so was EasyB)')}

...

Page 107: Make Your Testing Groovy

...EasyB Example...

QCON 2010 - 107

© A

SE

RT

2006-2

010

...and "I have entered 'Cowabunga Dude!' into the content", {

form.getTextAreaByName('content').setText('Cowabunga Dude!')}

and "I have selected 'Home' as the category", {form.getSelectByName('category').getOptions().find { it.text == 'Home' }.setSelected(true

}

and "I click the 'Create Post' button", {result = form.getInputByName('btnPost').click()

}

then "I expect the entry to be posted", {// check blog post detailsassert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Bart was here.*'def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home' // traditional styleh3headings.item(2).textContent.shouldBe 'Author: Bart' // BDD style

// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.firstChildassert para.textContent == 'Cowabunga Dude!'// para.shouldHave textContent: 'Cowabunga Dude!'

}}

Page 108: Make Your Testing Groovy

...EasyB Example...

QCON 2010 - 108

© A

SE

RT

2006-2

010

Page 109: Make Your Testing Groovy

...EasyB Example

QCON 2010 - 109

© A

SE

RT

2006-2

010

2 scenarios (including 1 pending) executed successfully.

Story: simp blog initial

scenario Bart posts a new blog entry [PENDING]

given we are on the create blog entry page

when I have entered 'Bart was here' as the title

when I have entered 'Cowabunga Dude!' into the content [PENDING]

when I have selected 'Home' as the category [PENDING]

when I have selected 'Bart' as the author [PENDING]

when I click the 'Create Post' button [PENDING]

then I expect the entry to be posted [PENDING]

Story: simp blog

Post Blog Entry Feature

for feature

As a Blogger

I want to be able to post a blog

So that I can keep others informed

given we are on the create blog entry page

scenario Bart was here blog

when I have entered 'Bart was here' as the title

when I have entered 'Cowabunga Dude!' into the content

when I have selected 'Home' as the category

when I click the 'Create Post' button

then I expect the entry to be posted

easyb is preparing to process 2 file(s)

Running simp blog initial story (SimpBlogInitialStory.groovy)

Scenarios run: 1, Failures: 0, Pending: 1, Time elapsed: 1.049 sec

Running simp blog story (SimpBlogStory.groovy)

Scenarios run: 1, Failures: 0, Pending: 0, Time elapsed: 1.356 sec

2 total behaviors ran (including 1 pending behavior) with no failures

easyb execution passed

Page 110: Make Your Testing Groovy

Cucumber• Description

– Loose coupling

between text spec

and step defns

QCON 2010 - 110

© A

SE

RT

2006-2

010

# language: enFeature: Addition

In order to avoid silly mistakesAs a math idiot I want to be told the sum of two numbers

Scenario Outline: Add two numbersGiven I have entered <input_1> into the calculatorAnd I have entered <input_2> into the calculatorWhen I press <button>Then the stored result should be <output>

Examples:| input_1 | input_2 | button | output || 20 | 30 | add | 50 || 2 | 5 | add | 7 || 0 | 40 | add | 40 |

# language: enFeature: DivisionIn order to avoid silly mistakesCashiers must be able to calculate a fraction

Scenario: Regular numbersGiven I have entered 3 into the calculatorAnd I have entered 2 into the calculatorWhen I press divideThen the stored result should be 1.5

Page 111: Make Your Testing Groovy

Cucumber Example...

QCON 2010 - 111

© A

SE

RT

2006-2

010

# language: en

@newpost

Feature: New Blog Post

In order to create a new blog entry

Bloggers should be able to select their name and category and enter text

Scenario: New Posting

Given we are on the create blog entry page

When I have entered "Bart was here" as the title

And I have entered "Cowabunga Dude!" as the content

And I have selected "Home" from the "category" dropdown

And I have selected "Bart" from the "author" dropdown

And I click the 'Create Post' button

Then I should see a heading message matching "Post.*: Bart was here.*"

Page 112: Make Your Testing Groovy

...Cucumber Example...

QCON 2010 - 112

© A

SE

RT

2006-2

010

Page 113: Make Your Testing Groovy

...Cucumber Example

QCON 2010 - 113

© A

SE

RT

2006-2

010

import com.gargoylesoftware.htmlunit.WebClientthis.metaClass.mixin(cuke4duke.GroovyDsl)

Given ~/we are on the create blog entry page/, { ->page = new WebClient().getPage('http://localhost:8080/postForm')

}

When(~/I have entered "(.*)" as the title/) {String title ->form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title + ' (and so was Cucumber)')

}

When(~'I have entered "(.*)" as the content') {String content ->form.getTextAreaByName('content').setText(content)

}

When(~'I have selected "(.*)" from the "(.*)" dropdown') {String option, String name ->form.getSelectByName(name).getOptions().find {

it.text == option }.setSelected(true)}

When(~"I click the 'Create Post' button") { ->result = form.getInputByName('btnPost').click()

}

Then(~'I should see a heading message matching "(.*)"') {String pattern ->// ensureThat result.getElementsByTagName('h1').item(0).textContent.matches(pattern)

assert result.getElementsByTagName('h1').item(0).textContent.matches(pattern)}

Page 114: Make Your Testing Groovy

Cucumber Data Driven Example...

QCON 2010 - 114

© A

SE

RT

2006-2

010

# language: en@newpostFeature: New Blog PostIn order to create a new blog entryBloggers should be able to select their name and category and enter text

Scenario Outline: New PostingGiven we are on the create blog entry pageWhen I have entered "<title>" as the titleAnd I have entered "<content>" as the contentAnd I have selected "<category>" from the "category" dropdownAnd I have selected "<author>" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: <title>.*"

Examples:| title | content | category | author || Title 1 | Content 1 | Home | Bart || Title 2 | Content 2 | Work | Homer || Title 3 | Content 3 | Food | Marge |

Page 115: Make Your Testing Groovy

...Cucumber Data Driven Example

QCON 2010 - 115

© A

SE

RT

2006-2

010

Page 116: Make Your Testing Groovy

JBehave

• Description– Behaviour-driven development in Java

• Also works out of the box for Groovy

– Behavior scenarios written in text

• Use the words Given, When, Then and And.

– Mapped using regular expressions and annotations

to step methods

– Web Runner available for non-technical users to

easily run tests

– Hooks to Selenium available in JBehave Web

• Other Java libraries (e.g. HtmlUnit) easy to use too

– Supports parameter converters

• Getting 'String' parameters into appropriate Object values

– Supports a 'StepDoc' function

• For listing available scenario clausesQCON 2010 - 116

© A

SE

RT

2006-2

010

Page 117: Make Your Testing Groovy

JBehave Example...

QCON 2010 - 117

© A

SE

RT

2006-2

010

Given we are on the create blog entry pageWhen I have entered "Bart was here" as the titleAnd I have entered "Cowabunga Dude!" as the contentAnd I have selected "Home" from the "category" dropdownAnd I have selected "Bart" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: Bart was here.*"

import org.jbehave.scenario.Scenarioimport org.jbehave.scenario.steps.Steps

class NewPostScenario extends Scenario {NewPostScenario() {

super([new CreateBlogSteps()] as Steps[])}

}Scenario:

Given we are on the create blog entry page

When I have entered "Bart was here" as the title

And I have entered "Cowabunga Dude!" as the content

And I have selected "Home" from the "category" dropdown

And I have selected "Bart" from the "author" dropdown

And I click the 'Create Post' button

Then I should see a heading message matching "Post.*: Bart was here.*"

Page 118: Make Your Testing Groovy

...JBehave Example...

QCON 2010 - 118

© A

SE

RT

2006-2

010

import org.jbehave.scenario.steps.Stepsimport org.jbehave.scenario.annotations.*import com.gargoylesoftware.htmlunit.WebClient

class CreateBlogSteps extends Steps {def page, form, result

@Given("we are on the create blog entry page")void gotoEntryPage() {

page = new WebClient().getPage('http://localhost:8080/postForm')}

@When('I have entered "$title" as the title')void enterTitle(String title) {

form = page.getFormByName('post')form.getInputByName('title').

setValueAttribute(title + ' (and so was JBehave)')}

@When('I have entered "$content" as the content')void enterContent(String content) {

form.getTextAreaByName('content').setText(content)}

...

Example uses HtmlUnit which must be added to CLASSPATH

Page 119: Make Your Testing Groovy

...JBehave Example

QCON 2010 - 119

© A

SE

RT

2006-2

010

...@When('I have selected "$option" from the "$name" dropdown')void selectOption(String option, String name) {

form.getSelectByName(name).getOptions().find {it.text == option }.setSelected(true)

}

@When("I click the 'Create Post' button")void clickPostButton() {

result = form.getInputByName('btnPost').click()}

@Then('I should see a heading message matching "$message"')void checkPost(String pattern) {

assert result.getElementsByTagName('h1').item(0).textContent.matches(pattern)

}}

...void checkPost(String pattern) {

ensureThat result.getElementsByTagName('h1').item(0).textContent.matches(pattern)

}...

Choose either traditional

style or BDD style

Page 120: Make Your Testing Groovy

JBehave Web Runner...

QCON 2010 - 120

© A

SE

RT

2006-2

010

Page 121: Make Your Testing Groovy

...JBehave Web Runner

QCON 2010 - 121

© A

SE

RT

2006-2

010

Page 122: Make Your Testing Groovy

Hacked* "JBehave Aware" GroovyConsole

QCON 2010 - 122

© A

SE

RT

2006-2

010

* Not currently publically available

Page 123: Make Your Testing Groovy

Robot Framework...

• Description– Keyword-driven test automation framework

for acceptance level testing and acceptance

test-driven development (ATDD)

– Easy to use tabular syntax for creating test

– Easy to use test libraries implemented either with

Python or Java

– Open source, Apache License 2.0

– Supports creating data-driven test cases.

– Provides tagging to categorize and select test cases

to be executed

– Provides easy-to-read reports and logs in HTML

format

– XML-RPC interface for remote testing

QCON 2010 - 123

© A

SE

RT

2006-2

010

Page 124: Make Your Testing Groovy

...Robot Framework...

• Tabular tests / Executable Specs– In HTML or Text

QCON 2010 - 124

© A

SE

RT

2006-2

010

***Settings***

Library OperatingSystem

***Variables***

${MESSAGE} Hello, world!

***Test Cases***

My Test [Documentation] Example test

Log ${MESSAGE}

My Keyword /tmp

Another Test

Should Be Equal ${MESSAGE} Hello, world!

***Keywords***

My Keyword [Arguments] ${path}

Directory Should Exist ${path}

Page 125: Make Your Testing Groovy

...Robot Framework...

• Reporting

QCON 2010 - 125

© A

SE

RT

2006-2

010

Page 126: Make Your Testing Groovy

...Robot Framework...

• Reporting

QCON 2010 - 126

© A

SE

RT

2006-2

010

Page 127: Make Your Testing Groovy

...Robot Framework...

• Tools

QCON 2010 - 127

© A

SE

RT

2006-2

010

Page 128: Make Your Testing Groovy

Robot Framework Example...

QCON 2010 - 128

© A

SE

RT

2006-2

010

***Test Cases***

Submit postGiven we are on the create blog entry pageWhen I have entered "Bart was here" as the titleAnd I have entered "Cowabunga Dude!" as the contentAnd I have selected "Home" from the "category" dropdownAnd I have selected "Bart" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: Bart was here.*"

***Keywords***

Given we are on the create blog entry pageOpen Browser http://localhost:8080/postForm ${BROWSER}# Set Selenium Speed ${DELAY}Title Should Be Welcome to SimpBlog

When I have entered "${text}" as the titleInput Text title ${text}

...

First look at the Python version using Selenium

Page 129: Make Your Testing Groovy

...Robot Framework Example

QCON 2010 - 129

© A

SE

RT

2006-2

010

...

And I have entered "${text}" as the contentInput Text content ${text}

And I have selected "${option}" from the "${list}" dropdownSelect From List ${list} ${option}

And I click the 'Create Post' buttonClick Button btnPost

Then I should see a heading message matching "${expected}"${text} = Get Text xpath=//h1Should Match Regexp ${text} ${expected}

***Settings***Library ${LIBRARY}Test Teardown Close Browser

***Variables***${BROWSER} *firefox3 C:/Program Files (x86)/Mozilla Firefox/firefox.exe${DELAY} 0${LIBRARY} SeleniumLibrary

Page 130: Make Your Testing Groovy

Robot Framework with Groovy...

QCON 2010 - 130

© A

SE

RT

2006-2

010

***Settings***Library Remote localhost:8270 WITH NAME Groovy

***Test Case***Multiple Blog PostsPost Bart Home Title 1 Content 1Post Homer Work Title 2 Content 2Post Marge Food Title 3 Content 3

Pybot

RunnerTestCase.txt

Groovy

XML RPC

Server

Groovy/Java

Driver

XML-RPC

Page 131: Make Your Testing Groovy

...Robot Framework with Groovy...

QCON 2010 - 131

© A

SE

RT

2006-2

010

import groovy.net.xmlrpc.*import java.net.ServerSocketimport com.gargoylesoftware.htmlunit.WebClient

def server = new XMLRPCServer()server.get_keyword_names = { ["Post"] }server.get_keyword_documentation = { "" }server.get_keyword_arguments = { ["*args"] }server.run_keyword = { name, args ->

assert name == 'Post'def (author, category, title, content) = argsdef client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextdef form = page.getFormByName('post')form.getInputByName('title').setValueAttribute("$title (entered with Robot Framework)")form.getSelectByName('author').getOptions().find{ it.text == author }.setSelected(true)form.getSelectByName('category').getOptions().find{ it.text == category }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()assert result.getElementsByTagName('h1').item(0).textContent.matches("Post.*: $title.*")def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == "Category: $category"assert h3headings.item(2).textContent == "Author: $author"def para = result.getByXPath('//TABLE//TR/TD/P')[0]assert para.textContent == contentreturn [status:'PASS', output:"$name $args", error:'bad arg']

}

def serverSocket = new ServerSocket(8270)server.startServer(serverSocket)

In the future, a generic version of this file may be possible

Page 132: Make Your Testing Groovy

...Robot Framework with Groovy...

QCON 2010 - 132

© A

SE

RT

2006-2

010

Page 133: Make Your Testing Groovy

...Robot Framework with Groovy

QCON 2010 - 133

© A

SE

RT

2006-2

010

Page 134: Make Your Testing Groovy

FitNesse/SLIM

• Description– Tool for enhancing collaboration

in software development

– Allows customers,

developers and

testers to easily

create and run

tests to compare

expected with

actual results

– Tests are captured

in wiki format and

mapped into code

using fixtures

QCON 2010 - 134

© A

SE

RT

2006-2

010

Source: http://fitnesse.org/

Page 135: Make Your Testing Groovy

SLIM with Groovy...

QCON 2010 - 135

© A

SE

RT

2006-2

010

Page 136: Make Your Testing Groovy

...SLIM with Groovy

QCON 2010 - 136

© A

SE

RT

2006-2

010

package simpblog

import com.gargoylesoftware.htmlunit.WebClient

class NewBlogPost {String author, title, content, categoryprivate resultdef execute() {

def client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextdef form = page.getFormByName('post')form.getInputByName('title').

setValueAttribute("$title (entered with Robot Framework)")form.getSelectByName('author').getOptions().find{

it.text == author }.setSelected(true)form.getSelectByName('category').getOptions().find{

it.text == category }.setSelected(true)form.getTextAreaByName('content').setText(content)result = form.getInputByName('btnPost').click()

}def mainHeading() {

def m = result.getElementsByTagName('h1').item(0).textContent =~/Post .*: (.*) \([^)]*\)/

m[0][1]}

}

Example uses HtmlUnit to call SimpBlog web site but those details aren't important here

Page 137: Make Your Testing Groovy

Topics

• Introduction

• Web Drivers

• Test Runners

Non-web Drivers

• Other Tools

• Going Beyond

• Further Information

QCON 2010 - 137

© A

SE

RT

2006-2

010

Page 138: Make Your Testing Groovy

QCON 2010 - 138

© A

SE

RT

2006-2

010

More Details: Working with Databases

• Using standard SQL statements

• Using DataSets

import groovy.sql.Sql

def foo = 'cheese'def db = Sql.newInstance("jdbc:mysql://localhost:3306/mydb",

"user", "pswd", "com.mysql.jdbc.Driver")

db.eachRow("select * from FOOD where type=${foo}") {println "Gromit likes ${it.name}"

}

import groovy.sql.Sql

def db = Sql.newInstance("jdbc:mysql://localhost:3306/mydb","user", "pswd", "com.mysql.jdbc.Driver")

def food = db.dataSet('FOOD')def cheese = food.findAll { it.type == 'cheese' }cheese.each { println "Gromit likes ${it.name}" }

Page 139: Make Your Testing Groovy

WebTest testing Emails

QCON 2010 - 139

© A

SE

RT

2006-2

010

def ant = new AntBuilder()

def webtest_home = System.properties.'webtest.home'

ant.taskdef(resource:'webtest.taskdef'){

classpath(){

pathelement(location:"$webtest_home/lib")

fileset(dir:"$webtest_home/lib", includes:"**/*.jar")

}

}

ant.testSpec(name:'Email Test'){

steps {

emailSetConfig(server:'localhost', password:'password',

username:'[email protected]', type:'pop3')

emailStoreMessageId(subject:'/Build notification/',

property:'msg')

emailStoreHeader(property:'subject',

messageId:'#{msg}', headerName:'Subject')

groovy('''def subject = step.webtestProperties.subject

assert subject.startsWith('Build notification')''')

emailMessageContentFilter(messageId:'#{msg}')

verifyText(text:'Failed build')

}

}

Page 140: Make Your Testing Groovy

SOAP Client and Server

QCON 2010 - 140

© A

SE

RT

2006-2

010

class MathService {double add(double a, double b) {

a + b}double square(double c) {

c * c}

}

import groovy.net.soap.SoapServer

def server = new SoapServer('localhost', 6789)server.setNode('MathService')server.start()

import groovy.net.soap.SoapClient

def math = new SoapClient('http://localhost:6789/MathServiceInterface?wsdl')

assert math.add(1.0, 2.0) == 3.0

assert math.square(3.0) == 9.0

Page 141: Make Your Testing Groovy

FEST• Description

– Framework for testing Swing GUIs (among other things)

– Simulation of user interaction with a GUI (e.g. mouse / keyboard input)

– Reliable GUI component lookup• by type, by name or custom search criteria

– Support for all Swing components included in the JDK

– Compact and powerful API for creation and maintenance of functional

GUI tests

– Regular expression matching

– Supports Applet testing

– Ability to embed screenshots of failed GUI tests in HTML test reports

– Can be used with either TestNG or JUnit

– Supports testing violations of Swing's threading rules

– Experimental Groovy Builder support (coming soon!)

QCON 2010 - 141

© A

SE

RT

2006-2

010

dialog.comboBox("domain").select("Users")dialog.textBox("username").enterText("leia.organa")dialog.button("login").click()dialog.optionPane().requireErrorMessage()

.requireMessage("Please enter your .*")

Page 142: Make Your Testing Groovy

WindowLicker• Description

– A framework for the test-driven development of Java systems through

the GUI

• Features– Provides a high-level API for controlling and making assertions about

graphical user interfaces including Swing & Dynamic HTML (aka

"AJAX") including GWT

– Deals with the asynchronous nature of GUI and AJAX programming so

the tests don't have to

– Controls the GUI by sending native mouse and keyboard events

– Handles different keyboard layouts

– Produces high quality error messages to help you easily diagnose test

failures

– Easily extensible to cope with new user interface components

QCON 2010 - 142

© A

SE

RT

2006-2

010

void hasColumnTitles() {def headers = new JTableHeaderDriver(this, JTableHeader)headers.hasHeaders(

matching(withLabelText("Item"), withLabelText("Last Price"),withLabelText("Last Bid"), withLabelText("State")))

}

Page 143: Make Your Testing Groovy

Topics

• Introduction

• Web Drivers

• Test Runners

• Non-web Drivers

Other Tools

• Going Beyond

• Further Information

QCON 2010 - 143

© A

SE

RT

2006-2

010

Page 144: Make Your Testing Groovy

QCON 2010 - 144

© A

SE

RT

2006-2

010

SoapUI...

Page 145: Make Your Testing Groovy

...SoapUI

• Tool for testing Web Services has a built-

in Groovy editor for custom steps

QCON 2010 - 145

© A

SE

RT

2006-2

010

Page 146: Make Your Testing Groovy

iTest2

• Tool for recording, refactoring and

running watir style tests

QCON 2010 - 146

© A

SE

RT

2006-2

010

We have had success cutting and pasting recorded test steps – using metaprogramming to map to Groovy testing DSL

Page 147: Make Your Testing Groovy

Sahi...• Description

– Tool for recording and running web tests

• Features– Supports in-browser controls (cross-platform/browser)

– Ability to add assertions (check points) for validation

– Text based programmable scripts with ability to

• parametrize variables

• re-factor into functions

• organize and include other scripts

– Can run scripts in batch mode for automated testing

– Command line and ant integration

– Automatic html reporting with error logs

– Easily extensible via simple javascript

– Support for data-driven testing

– Multi-threaded playback

– HTTP, HTTPS and AJAX support

QCON 2010 - 147

© A

SE

RT

2006-2

010

Page 148: Make Your Testing Groovy

...Sahi

QCON 2010 - 148

© A

SE

RT

2006-2

010

We have had success cutting and

pasting recorded test steps – using

metaprogramming to map to Groovy

testing DSL

Further Info:

http://sahi.co.in/w/

Page 149: Make Your Testing Groovy

QCON 2010 - 149

© A

SE

RT

2006-2

010

JMeter

• What is Apache JMeter?– Java desktop application designed to load test

functional behavior and measure performance

– Originally designed for testing Web Applications but

has since expanded to other test functions

Performance Testing

JMeter can call out to Groovy – reusing your functional tests

Page 150: Make Your Testing Groovy

JMeter Case Study...

QCON 2010 - 150

© A

SE

RT

2006-2

010

Part 1: Native JMeter Tests

* No JavaScript

* No reuse of functional tests

Page 151: Make Your Testing Groovy

...JMeter Case Study...

QCON 2010 - 151

© A

SE

RT

2006-2

010

Further details about Statistical Aggregate plugin: http://rubenlaguna.com/wp/better-jmeter-graphs/

Page 152: Make Your Testing Groovy

...JMeter Case Study...

QCON 2010 - 152

© A

SE

RT

2006-2

010

Part 2: JMeter Groovy JUnitTests

* Reuse functional tests

import junit.framework.TestCase

class TestSimpBlogJUnit extends TestCase {static lisaPostAndCheck = nullstatic bartPostAndCheck = nullstatic String script = '''@Grab('net.sourceforge.htmlunit:htmlunit:2.5')import com.gargoylesoftware.htmlunit.WebClientdef page = new WebClient().getPage('http://localhost:8080/postForm')def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title)form.getSelectByName('category').getOptions().find { it.text == category }.setSelected(true)form.getSelectByName('author').getOptions().find { it.text == author }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()def titleResult = result.getElementsByTagName('h1').item(0).textContentdef h3headings = result.getElementsByTagName('h3')def categoryResult = h3headings.item(1).textContentdef authorResult = h3headings.item(2).textContentdef para = result.getByXPath('//TABLE//TR/TD/P')[0]def contentResult = para.textContentreturn new ResultHolder(titleResult, contentResult, authorResult, categoryResult)'''void setUp() {

lisaPostAndCheck = setUpBlogger(lisaPostAndCheck, 'Lisa', "I'm Hungry", "Food")bartPostAndCheck = setUpBlogger(bartPostAndCheck, 'Bart', "Don't have a cow dude", "Home")

}...

Page 153: Make Your Testing Groovy

...JMeter Case Study...

QCON 2010 - 153

© A

SE

RT

2006-2

010

...private setUpBlogger(orig, blogger, content, category) {

if (orig) return origdef result = new GroovyShell().parse(script)def binding = new Binding()binding.setVariable('title', blogger + ' was here (and so was JMeter)')binding.setVariable('content', content)binding.setVariable('author', blogger)binding.setVariable('category', category)result.binding = bindingresult

}

void testBartWasHere() {def result = bartPostAndCheck.run()assert result.title.contains('Bart was here')

}

void testLisaWasHere() {def result = lisaPostAndCheck.run()assert result.title.contains('Lisa was here')

}}

class ResultHolder {String title, content, author, categoryResultHolder(String title, String content, String author, String category) {

this.title = titlethis.content = contentthis.author = authorthis.category = category

}}

Page 154: Make Your Testing Groovy

...JMeter Case Study...

QCON 2010 - 154

© A

SE

RT

2006-2

010

Page 155: Make Your Testing Groovy

...JMeter Case Study

QCON 2010 - 155

© A

SE

RT

2006-2

010

Page 156: Make Your Testing Groovy

Twist

QCON 2010 - 156

© A

SE

RT

2006-2

010

Future version will support Groovy

Page 157: Make Your Testing Groovy

Topics

• Introduction

• Web Drivers

• Test Runners

• Non-web Drivers

• Other Tools

Going Beyond

• Further Information

QCON 2010 - 157

© A

SE

RT

2006-2

010

Page 158: Make Your Testing Groovy

All Combinations

• Description– Don't have a bunch

of hard-coded, hard

to maintain manual

test data or even

manually generated

CSV file

– Much better to

generate test cases

from succinct

expressions of

what you are trying

to achieve

QCON 2010 - 158

© A

SE

RT

2006-2

010

[['MacOS', 'Linux', 'Vista'],['2G', '4G', '6G', '8G'],['250G', '350G', '500G']

].combinations().each{os, mem, disk ->test(os, mem, disk)

}

test('MacOS', '4G', '250G')test('Linux', '4G', '250G')test('Vista', '4G', '250G')test('MacOS', '8G', '500G')test('Linux', '8G', '500G')test('Vista', '8G', '500G')// 30 more rows

Page 159: Make Your Testing Groovy

All Combinations Case Study

QCON 2010 - 159

© A

SE

RT

2006-2

010

import com.gargoylesoftware.htmlunit.WebClient

def combos = [["Bart", "Homer", "Marge", "Lisa", "Maggie"],["Work", "School", "Home", "Travel", "Food"],["foo", "bar", "baz"]].combinations()

println "Found ${combos.size()} combos"combos.each { author, category, content ->

postAndCheck category, author, content}

def postAndCheck(String category, String author, String content) {// ...// details not shown (ran with HtmlUnit)// ...

}

Found 75 combos

Page 160: Make Your Testing Groovy

All Pairs

• Description– Sometimes

called

pairwise

testing or

orthogonal

array testing

– Technique

to limit the

explosion of test cases by identifying samples of

important classes of test cases (equivalence classes)

• providing maximum coverage with minimum testing

– Instead of all combinations, systematically use pair-

wise combinations of interactions between objects

• as most faults result from adverse two-way interactions

QCON 2010 - 160

© A

SE

RT

2006-2

010

Page 161: Make Your Testing Groovy

All Pairs Case Study...

QCON 2010 - 161

© A

SE

RT

2006-2

010

class AllPairs {private initialResults, results, rest

private buildPairs(Map partialCombinations, inputsLeft) {def first = getFirstEntry(inputsLeft)def partialResults = []first.value.each {

def next = [(first.key): it]getFirstEntry(next)next.putAll(partialCombinations)partialResults << next

}if (inputsLeft.size() == 1) {

initialResults.addAll(partialResults)} else {

partialResults.each {def rest = inputsLeft.clone()rest.remove(first.key)buildPairs(it, rest)

}}

}

private adjustPairs() {results = initialResults.clone()initialResults.each {

def restResults = results.clone()restResults.remove(it)if (allPairsCovered(it, restResults)) {

results.remove(it)}

}}

...

...private getFirstEntry(Map map) {

return map.entrySet().toList().get(0)}

private getAllPairsFromMap(map) {if (!map || map.size() <= 1) return nulldef allPairs = new HashSet()def first = getFirstEntry(map)def restMap = map.clone()restMap.remove(first.key)restMap.each {

def nextPair = new HashSet()nextPair << firstnextPair << itallPairs << nextPair

}def restPairs = getAllPairsFromMap(rest)if (restPairs != null) {

allPairs.addAll(restPairs)}return allPairs

}...

Details here not important:

Given here for completeness as this

library is not publically available yet

Page 162: Make Your Testing Groovy

...All Pairs Case Study...

QCON 2010 - 162

© A

SE

RT

2006-2

010

...private boolean allPairsCovered(candidate, remaining) {

def totalCount = 0def pairCombos = getAllPairsFromMap(candidate)pairCombos.each {candidatePair ->

def pairFound = falsedef pairs = candidatePair.toList()for (it in remaining) {

def entries = it.entrySet()if (!pairFound && entries.contains(pairs[0]) && entries.contains(pairs[1])) {

pairFound = truetotalCount++

}}

}return (totalCount == pairCombos.size())

}

private updateUsedPairs(map) {getAllPairsFromMap(map).each { usedPairs << it }

}

def generate(configurations) {initialResults = new HashSet()results = new HashSet()buildPairs([:], configurations)adjustPairs()results

}}

Details here not important:

Given here for completeness as this

library is not publically available yet

Page 163: Make Your Testing Groovy

...All Pairs Case Study...

QCON 2010 - 163

© A

SE

RT

2006-2

010

import com.gargoylesoftware.htmlunit.WebClient

def pairs = new AllPairs().generate(author: ["Bart", "Homer", "Marge", "Lisa", "Maggie"],category: ["Work", "School", "Home", "Travel", "Food"],content: ["foo", "bar", "baz"])

println "Found ${pairs.size()} pairs"pairs.each {

println it // just for debugging purposespostAndCheck it.category, it.author, it.content

}

def postAndCheck(String category, String author, String content) {// ...// details not shown (ran with HtmlUnit)// ...

}

Page 164: Make Your Testing Groovy

...All Pairs Case Study

QCON 2010 - 164

© A

SE

RT

2006-2

010

Found 18 pairs[content:bar, category:Food, author:Bart][content:bar, category:School, author:Homer][content:foo, category:Work, author:Bart][content:baz, category:School, author:Homer][content:bar, category:Home, author:Maggie][content:foo, category:School, author:Marge][content:bar, category:Work, author:Bart][content:baz, category:Travel, author:Bart][content:foo, category:Home, author:Homer][content:bar, category:Travel, author:Marge][content:baz, category:Work, author:Homer][content:bar, category:Travel, author:Lisa][content:baz, category:Travel, author:Maggie][content:baz, category:Home, author:Marge][content:baz, category:Food, author:Homer][content:baz, category:Travel, author:Lisa][content:foo, category:Food, author:Maggie][content:foo, category:Travel, author:Lisa]

Page 165: Make Your Testing Groovy

gpars...

• Description– Library classes and DSL sugar providing intuitive

ways for Groovy developers to handle tasks

concurrently. Four logical parts:

• Actors provide a Groovy implementation of Scala-like

actors, both thread-bound actors and thread pool-

bound (event-driven) ones

• Dataflow Concurrency allows for very natural shared-

memory concurrency model, based on single-

assignment variables

• Asynchronizer extends the Java 1.5 built-in support

for executor services to enable multi-threaded

collection and closure processing

• Parallelizer uses JSR-166y Parallel Arrays to enable

multi-threaded collection processingQCON 2010 - 165

© A

SE

RT

2006-2

010

Page 166: Make Your Testing Groovy

...gpars...

QCON 2010 - 166

© A

SE

RT

2006-2

010

// run multiple closures in parallelAsynchronizer.withAsynchronizer {

assert [10, 20] == AsyncInvokerUtil.doInParallel({calculateA()},{calculateB()}

)}

// multiply numbers asynchronouslyParallelizer.withParallelizer(5) {

final List result = [1, 2, 3, 4, 5].collectAsync {it * 2}assert ([2, 4, 6, 8, 10].equals(result))

}

Page 167: Make Your Testing Groovy

...gpars...

QCON 2010 - 167

© A

SE

RT

2006-2

010

// support for dataflow to avoid doing synchronisationimport static org.gparallelizer.dataflow.DataFlow.thread

final def x = new DataFlowVariable()final def y = new DataFlowVariable()final def z = new DataFlowVariable()

thread {z << x.val + y.valprintln "Result: ${z.val}"

}

thread { x << 10 }

thread { y << 5 }

Page 168: Make Your Testing Groovy

...gpars

QCON 2010 - 168

© A

SE

RT

2006-2

010

// actor supportimport static org.gparallelizer.actors.pooledActors.PooledActors.*

def me = actor {friend.send('Hi')react(10.seconds) {

// continue conversation}

}me.metaClass.onTimeout = {->friend.send('I see, busy as usual. Never mind.')}me.start()

Page 169: Make Your Testing Groovy

gpars and SimpBlog

QCON 2010 - 169

© A

SE

RT

2006-2

010

@Grab('net.sourceforge.htmlunit:htmlunit:2.6')import com.gargoylesoftware.htmlunit.WebClient@Grab('org.gparallelizer:GParallelizer:0.8.3')import static org.gparallelizer.Parallelizer.*

def testCases = [['Home', 'Bart', 'Content 1'],['Work', 'Homer', 'Content 2'],['Travel', 'Marge', 'Content 3'],['Food', 'Lisa', 'Content 4']

]

withParallelizer(3) {testCases.eachAsync{ category, author, content ->

postAndCheck category, author, content}

}

private postAndCheck(category, author, content) {...

Note:

Testing

DSL makes

tests more

readable

Page 170: Make Your Testing Groovy

Native Groovy Versions also possible

QCON 2010 - 170

© A

SE

RT

2006-2

010

@Grab('net.sourceforge.htmlunit:htmlunit:2.6')import com.gargoylesoftware.htmlunit.WebClient

Thread.start {postAndCheck 'Home', 'Bart', 'Content 1'

}Thread.start {

postAndCheck 'Work', 'Homer', 'Content 2'}Thread.start {

postAndCheck 'Travel', 'Marge', 'Content 3'}Thread.start {

postAndCheck 'Food', 'Lisa', 'Content 4'}

private postAndCheck(category, author, content) {...

Or use:

ant.parallel {

//...

}

Or use:

"command".execute()

Page 171: Make Your Testing Groovy

Constraint/Logic Programming...

• Description– Style of programming where relations between

variables are stated in the form of constraints

– First made popular by logic programming languages

such as Prolog but the style is now also used outside

logic programming specific languages

– Constraints differ from the common primitives of

other programming languages in that they do not

specify one or more steps to execute but rather the

properties of a solution to be found

– Popular libraries used with Groovy supporting

constraint programming include Gecode/J, Choco

and tuProlog

– We'll look at Choco as an example

QCON 2010 - 171

© A

SE

RT

2006-2

010

Page 172: Make Your Testing Groovy

...Constraint/Logic Programming...

QCON 2010 - 172

© A

SE

RT

2006-2

010

Source: http://xkcd.com/287/

Page 173: Make Your Testing Groovy

...Constraint/Logic Programming...

QCON 2010 - 173

© A

SE

RT

2006-2

010

// requires choco 2.1.0-basic.jar from http://choco.emn.fr/import static choco.Choco.*import choco.kernel.model.variables.integer.IntegerVariable

def m = new choco.cp.model.CPModel()def s = new choco.cp.solver.CPSolver()

def menu = ['Mixed fruit' : 215,'French fries' : 275,'Side salad' : 335,'Hot wings' : 355,'Mozzarella sticks' : 420,'Sampler plate' : 580

]def numOrdered = new IntegerVariable[menu.size()]def priceEach = new int[menu.size()]def sum = 1505...

Found a solution:7 * Mixed fruit

Found a solution:1 * Mixed fruit2 * Hot wings1 * Sampler plate

Page 174: Make Your Testing Groovy

...Constraint/Logic Programming

QCON 2010 - 174

© A

SE

RT

2006-2

010

...menu.eachWithIndex { name, price, i ->

// number ordered >= 0// number ordered * price <= sumnumOrdered[i] = makeIntVar(name, 0, sum.intdiv(price))priceEach[i] = price

}m.addConstraint(eq(scalar(numOrdered, priceEach), sum))s.read(m)

def more = s.solve()while (more) {

println "Found a solution:"numOrdered.each {

def v = s.getVar(it)if (v.val) println " $v.val * $v.name"

}more = s.nextSolution()

}

Page 175: Make Your Testing Groovy

SimpBlog Case Study...

• You have been asked to set up some test

cases representing the Simpsons weekly

blogging habits

• After some careful study you observe the

following strange behavior– They never blog on the same day

– Marge blogs only on a Saturday or Sunday

– Maggie blogs only on a Tuesday or Thursday

– Lisa blogs only on a Monday, Wednesday or Friday

– Bart blogs only on the day after Lisa

– Homer only blogs if noone else blogged the previous

day and doesn't allow anyone to blog the next day

QCON 2010 - 175

© A

SE

RT

2006-2

010

Page 176: Make Your Testing Groovy

...SimpBlog Case Study...

QCON 2010 - 176

© A

SE

RT

2006-2

010

// requires choco 2.1.0-basic.jar from http://choco.emn.fr/import static choco.Choco.*

def m = new choco.cp.model.CPModel()def s = new choco.cp.solver.CPSolver()

daysOfWeek = ["Sunday", "Monday", "Tuesday", "Wednesday","Thursday", "Friday", "Saturday"]

def bart = makeIntVar('Bart', 0, 6)def homer = makeIntVar('Homer', 0, 6)def marge = makeIntVar('Marge', 0, 6)def lisa = makeIntVar('Lisa', 0, 6)def maggie = makeIntVar('Maggie', 0, 6)def simpsons = [bart, homer, marge, lisa, maggie]...

Page 177: Make Your Testing Groovy

...SimpBlog Case Study...

QCON 2010 - 177

© A

SE

RT

2006-2

010

...

// They never blog on the same dayfor (i in 0..<simpsons.size())

for (j in 0..<i) m.addConstraint(neq(simpsons[i], simpsons[j]))

// Marge blogs only on a Saturday or Sundaym.addConstraint(or(eq(marge, 0), eq(marge, 6)))

// Maggie blogs only on a Tuesday or Thursdaym.addConstraint(or(eq(maggie, 2), eq(maggie, 4)))

// Lisa blogs only on a Monday, Wednesday or Fridaym.addConstraint(or(eq(lisa, 1), eq(lisa, 3), eq(lisa, 5)))

// Bart blogs only on the day after Lisam.addConstraint(eq(plus(lisa, 1), bart))

// Homer only blogs if noone else blogged the previous// day and doesn't allow anyone to blog the next daym.addConstraint(and(distanceNEQ(homer, marge, 1),

distanceNEQ(homer, bart, 1),distanceNEQ(homer, maggie, 1),distanceNEQ(homer, lisa, 1)))

...

Page 178: Make Your Testing Groovy

...SimpBlog Case Study

QCON 2010 - 178

© A

SE

RT

2006-2

010

...

s.read(m)def more = s.solve()if (!more) println "No Solutions Found"else println pad("Solutions:") +

simpsons.collect{ pad(it.name) }.join()while (more) {

print pad("")println simpsons.collect {

def v = s.getVar(it)pad(daysOfWeek[v.val])

}.join()more = s.nextSolution()

}

def pad(s) { s.padRight(12) }

Solutions: Bart Homer Marge Lisa Maggie Thursday Saturday Sunday Wednesday Tuesday Tuesday Saturday Sunday Monday Thursday Saturday Tuesday Sunday Friday Thursday Thursday Sunday Saturday Wednesday Tuesday

Page 179: Make Your Testing Groovy

Polyglot Programming...

QCON 2010 - 179

© A

SE

RT

2006-2

010

@Grab('org.clojure:clojure:1.0.0')import clojure.lang.Compilerimport clojure.lang.RT

def src = new File('temp.clj')src.text = '''(ns groovy)(defn factorial [n]

(if (< n 2)1(* n (factorial (- n 1))))

'''

src.withReader { reader ->Compiler.load reader

}

def fac = RT.var('groovy', 'factorial')println fac.invoke(4)

...def jy = getEngine("jython")jy?.eval('''def factorial(n):

i=fact=1while i <= n:

fact=fact*ii=i+1

return fact

result = factorial(4)''')println jy?.result

Page 180: Make Your Testing Groovy

...Polyglot Programming

• But so what?– I can use Groovy for Scripting my

environment and or leveraging its runners

and other testing capabilities

– I can call out to other languages when needed

• Cucumber via JRuby for more native control

• Watir instead of Watij

• ScalaCheck for test data generation

• Jython for Robot Framework for more native

control

• Rhino for JavaScript testing

• Rules engine integration

QCON 2010 - 180

© A

SE

RT

2006-2

010

Page 181: Make Your Testing Groovy

ModelJUnit...

• Description– Supports model-based testing

– Allows you to write simple finite

state machine (FSM) models or

extended finite state machine

(EFSM) models in Java or Groovy

– You can then generate tests from

those models and measure various

model coverage metrics

QCON 2010 - 181

© A

SE

RT

2006-2

010

Page 182: Make Your Testing Groovy

...ModelJUnit...

QCON 2010 - 182

© A

SE

RT

2006-2

010

// require modeljunit.jarimport nz.ac.waikato.modeljunit.coverage.*import nz.ac.waikato.modeljunit.*

class VendingMachineModel implements FsmModel {def state = 0 // 0,25,50,75,100void reset(boolean testing) {state = 0}

boolean vendGuard() {state == 100}@Action void vend() {state = 0}

boolean coin25Guard() {state <= 75}@Action void coin25() {state += 25}

boolean coin50Guard() {state <= 50}@Action void coin50() {state += 50}

}

def tester = new RandomTester(new VendingMachineModel())tester.buildGraph()def metrics = [new ActionCoverage(), new StateCoverage(),

new TransitionCoverage(), new TransitionPairCoverage()]metrics.each { tester.addCoverageMetric it }

tester.addListener "verbose"tester.generate 20

println '\nMetrics Summary:'tester.printCoverage()

Page 183: Make Your Testing Groovy

...ModelJUnit...

QCON 2010 - 183

© A

SE

RT

2006-2

010

...

done (0, coin50, 50)done (50, coin25, 75)done (75, coin25, 100)done Random reset(true)done (0, coin50, 50)done (50, coin25, 75)done (75, coin25, 100)done (100, vend, 0)done (0, coin50, 50)done (50, coin50, 100)done (100, vend, 0)done (0, coin25, 25)done (25, coin25, 50)done Random reset(true)done (0, coin50, 50)done (50, coin25, 75)done (75, coin25, 100)done (100, vend, 0)done (0, coin50, 50)done (50, coin25, 75)...

...

Metrics Summary:action coverage: 3/3state coverage: 5/5transition coverage: 7/8transition-pair coverage: 8/12...

Page 184: Make Your Testing Groovy

ModelJUnit: SimpBlog Case Study...

• Does the order in which form information

is entered affect the application?– Could AJAX effects be causing unexpected results?

QCON 2010 - 184

© A

SE

RT

2006-2

010

// require modeljunit.jar, htmlunit.jarimport nz.ac.waikato.modeljunit.coverage.*import nz.ac.waikato.modeljunit.*import com.gargoylesoftware.htmlunit.WebClient

class SimpBlogModel implements FsmModel {

boolean authorSelected = falseboolean categorySelected = falseboolean titleEntered = falseboolean contentEntered = falseint count = 0def client, page, form

// Special known method, allows equivalence class definition// example states: __ __ __ __, AU __ __ __, AU CA TI COdef getState() {"${authorSelected ? ' AU ' : ' __ '}${categorySelected ? ' CA ' : ' __ '}" +"${titleEntered ? ' TI ' : ' __ '}${contentEntered ? ' CO ' : ' __ '}"

}...

...void reset(boolean testing) {authorSelected = falsecategorySelected = falsetitleEntered = falsecontentEntered = falseclient = new WebClient()page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextform = page.getFormByName('post')

}...

Page 185: Make Your Testing Groovy

...ModelJUnit: SimpBlog Case Study...

QCON 2010 - 185

© A

SE

RT

2006-2

010

...boolean "enter title Guard"() { !titleEntered }

@Action void "enter title "() { titleEntered = trueform.getInputByName('title').setValueAttribute("Title ${count++}")

}

boolean enterContentGuard() { !contentEntered }

@Action void enterContent() { contentEntered = trueform.getTextAreaByName('content').setText("Content ${count++}")

}

boolean chooseAuthorGuard() { !authorSelected }

@Action void chooseAuthor() { authorSelected = true // simple version just Lisaform.getSelectByName('author').getOptions().find{ it.text == 'Lisa' }.setSelected(true)

}

boolean pickCategoryGuard() { !categorySelected }

@Action void pickCategory() { categorySelected = true // simple version just Homeform.getSelectByName('category').getOptions().find{ it.text == 'Home' }.setSelected(true)

}

boolean "submit post Guard"() { categorySelected && authorSelected &&titleEntered && contentEntered }

@Action void "submit post "() {def result = form.getInputByName('btnPost').click()assert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Title .*')// could do more asserts herereset(true)

}} // end of SimpBlogModel class definition...

Page 186: Make Your Testing Groovy

...ModelJUnit: SimpBlog Case Study...

QCON 2010 - 186

© A

SE

RT

2006-2

010

def tester = new RandomTester(new SimpBlogModel())tester.buildGraph()def metrics = [

new ActionCoverage(),new StateCoverage(),new TransitionCoverage(),new TransitionPairCoverage()

]metrics.each {

tester.addCoverageMetric it}

tester.addListener "verbose"tester.generate 50

println '\nMetrics Summary:'tester.printCoverage()

def graphListener = tester.model.getListener("graph")graphListener.printGraphDot "simpblog.dot"println "\nGraph contains " + graphListener.graph.numVertices() +

" states and " + graphListener.graph.numEdges() + " transitions."

Page 187: Make Your Testing Groovy

...ModelJUnit: SimpBlog Case Study...

QCON 2010 - 187

© A

SE

RT

2006-2

010

done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enterContent, __ CA __ CO )done ( __ CA __ CO , enter title , __ CA TI CO )done ( __ CA TI CO , chooseAuthor, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , chooseAuthor, AU CA __ __ )done ( AU CA __ __ , enter title , AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , pickCategory, AU CA __ __ )done ( AU CA __ __ , enter title , AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , enterContent, __ __ __ CO )done ( __ __ __ CO , pickCategory, __ CA __ CO )done ( __ CA __ CO , chooseAuthor, AU CA __ CO )done ( AU CA __ CO , enter title , AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enter title , __ CA TI __ )done ( __ CA TI __ , chooseAuthor, AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , pickCategory, AU CA __ __ )done ( AU CA __ __ , enter title , AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )...

...done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enterContent, __ CA __ CO )done ( __ CA __ CO , chooseAuthor, AU CA __ CO )done ( AU CA __ CO , enter title , AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , pickCategory, AU CA __ __ )done ( AU CA __ __ , enterContent, AU CA __ CO )done ( AU CA __ CO , enter title , AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , enter title , AU __ TI __ )done ( AU __ TI __ , pickCategory, AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done Random reset(true)done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enterContent, __ CA __ CO )done ( __ CA __ CO , enter title , __ CA TI CO )done ( __ CA TI CO , chooseAuthor, AU CA TI CO )

Metrics Summary:action coverage: 5/5state coverage: 12/16transition coverage: 19/33transition-pair coverage: 26/56

Graph contains 16 states and 33 transitions.

Page 188: Make Your Testing Groovy

...ModelJUnit: SimpBlog Case Study...

QCON 2010 - 188

© A

SE

RT

2006-2

010

Simplified

version

(just Lisa)

Advanced

version

Page 189: Make Your Testing Groovy

...ModelJUnit: SimpBlog Case Study

QCON 2010 - 189

© A

SE

RT

2006-2

010

Page 190: Make Your Testing Groovy

ScalaCheck

• Description– Tool for testing Scala, Java and Groovy programs

– Based on property specifications and automatic

random test data generation

QCON 2010 - 190

© A

SE

RT

2006-2

010

> scala -classpath ScalaCheck-1.5.jarWelcome to Scala version 2.7.6.final

scala> import org.scalacheck.Prop.forAll

scala> val propConcatLists = forAll { (l1: List[Int], l2: List[Int]) =>| l1.size + l2.size == (l1 ::: l2).size }

propConcatLists: org.scalacheck.Prop = Prop

scala> propConcatLists.check+ OK, passed 100 tests.

scala> val propSqrt = forAll { (n: Int) => scala.Math.sqrt(n*n) == n }propSqrt: org.scalacheck.Prop = Prop

scala> propSqrt.check! Falsified after 2 passed tests.> ARG_0: "-1" (1 shrinks, original arg: "-2")

Page 191: Make Your Testing Groovy

ScalaCheck: SimpBlog Case Study...

QCON 2010 - 191

© A

SE

RT

2006-2

010

class SimpBlogChecker {static postAndCheck = nullstatic clean(s) { s.replace('\\', '\\\\').replace('\n', '\\n') }static ResultHolder postAndReturn(String title, String content,

String author, String category) {if (!postAndCheck) postAndCheck = new GroovyShell().parse('''@Grab('net.sourceforge.htmlunit:htmlunit:2.5')import com.gargoylesoftware.htmlunit.WebClientdef page = new WebClient().getPage('http://localhost:8080/postForm')def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title)form.getSelectByName('category').getOptions().find {

it.text == category }.setSelected(true)form.getSelectByName('author').getOptions().find {

it.text == author }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()def titleResult = result.getElementsByTagName('h1').item(0).textContentdef h3headings = result.getElementsByTagName('h3')def categoryResult = h3headings.item(1).textContentdef authorResult = h3headings.item(2).textContentdef para = result.getByXPath('//TABLE//TR/TD/P')[0]def contentResult = para.textContentreturn new ResultHolder(titleResult, contentResult, authorResult, categoryResult)''')...

Page 192: Make Your Testing Groovy

...ScalaCheck: SimpBlog Case Study...

QCON 2010 - 192

© A

SE

RT

2006-2

010

...def binding = new Binding()binding.setVariable('title', title)binding.setVariable('content', clean(content))binding.setVariable('author', author)binding.setVariable('category', category)postAndCheck.binding = bindingpostAndCheck.run()

}}

class ResultHolder {String title, content, author, categoryResultHolder(String title, String content,

String author, String category) {this.title = titlethis.content = contentthis.author = authorthis.category = category

}}

> groovyc SimpBlogChecker.groovy

Page 193: Make Your Testing Groovy

...ScalaCheck: SimpBlog Case Study...

QCON 2010 - 193

© A

SE

RT

2006-2

010

//CheckSimpBlog.scala

import org.scalacheck.Prop._import org.scalacheck.ConsoleReporter.testStatsEximport org.scalacheck.Test.checkimport org.scalacheck.Arbitrary._import org.scalacheck.Gen._

object CheckSimpBlog {

val fieldsGen = for {title <- elements("Title 1", "Title 2")content <- arbitrary[String]author <- elements("Bart", "Homer", "Lisa", "Marge", "Maggie")category <- elements("Home", "Work", "Food", "Travel")

} yield (title, content, author, category)...

> scalac -classpath .;../lib/ScalaCheck-1.5.jar;../../groovy-1.7-beta-2-SNAPSHOT/embeddable/groovy-all-1.7-beta-2-SNAPSHOT.jar CheckSimpBlog.scala

Page 194: Make Your Testing Groovy

...ScalaCheck: SimpBlog Case Study...

QCON 2010 - 194

© A

SE

RT

2006-2

010

...val enterFieldsAcceptedAndEchoedProperty = forAll(fieldsGen)(

fields =>{

val (title, content, author, category) = fieldsval result = SimpBlogChecker.postAndReturn(title, content, author, category)result.getTitle contains titleresult.getContent contains contentresult.getAuthor contains authorresult.getCategory contains category

})

val tests = scala.List(("enterFieldsAcceptedAndEchoedProperty",

enterFieldsAcceptedAndEchoedProperty))

def main(args: scala.Array[String]) =tests foreach { case (name, p) => testStatsEx(name, check(p)) }

}

> scala -classpath .;../lib/ScalaCheck-1.5.jar;../../groovy-1.7-beta-2-SNAPSHOT/embeddable/groovy-all-1.7-beta-2-SNAPSHOT.jar;../../groovy-1.7-beta-2-SNAPSHOT/lib/ivy-2.1.0-rc2.jar CheckSimpBlog+ OK, passed 100 tests.

Page 195: Make Your Testing Groovy

...ScalaCheck: SimpBlog Case Study

QCON 2010 - 195

© A

SE

RT

2006-2

010

Page 196: Make Your Testing Groovy

> python ../src/ygTranslate.py simpblog.gr > simpblog.py> python simpblog.py

YouGen...

QCON 2010 - 196

© A

SE

RT

2006-2

010

{cov [([1,3,5],2),([7],1)]}

Test ::= 'postAndCheck(' Author ',' Title ',' Category ',' Content ')';

Author ::= '"Bart"';

Author ::= '"Homer"';

Author ::= '"Lisa"';

Author ::= '"Marge"';

Author ::= '"Maggie"';

Category ::= '"Home"';

Category ::= '"Work"';

Category ::= '"Food"';

Category ::= '"Travel"';

Title ::= '"Title 1"';

Title ::= '"Title 2"';

Title ::= '"Title 3"';

Content ::= '"Content 1"';

Content ::= '"Content 2"';

Content ::= '"Content 3"';

simpblog.gr

Page 197: Make Your Testing Groovy

...YouGen

QCON 2010 - 197

© A

SE

RT

2006-2

010

postAndCheck( "Bart" , "Title 1" , "Home" , "Content 1" )

postAndCheck( "Homer" , "Title 2" , "Home" , "Content 1" )

postAndCheck( "Bart" , "Title 2" , "Work" , "Content 1" )

postAndCheck( "Lisa" , "Title 3" , "Home" , "Content 1" )

postAndCheck( "Bart" , "Title 3" , "Food" , "Content 1" )

postAndCheck( "Homer" , "Title 1" , "Travel" , "Content 1" )

postAndCheck( "Lisa" , "Title 2" , "Travel" , "Content 1" )

postAndCheck( "Marge" , "Title 3" , "Travel" , "Content 1" )

postAndCheck( "Maggie" , "Title 1" , "Work" , "Content 1" )

postAndCheck( "Bart" , "Title 1" , "Travel" , "Content 1" )

postAndCheck( "Homer" , "Title 3" , "Work" , "Content 1" )

postAndCheck( "Lisa" , "Title 1" , "Food" , "Content 1" )

postAndCheck( "Marge" , "Title 2" , "Food" , "Content 1" )

postAndCheck( "Homer" , "Title 1" , "Food" , "Content 1" )

postAndCheck( "Marge" , "Title 1" , "Work" , "Content 1" )

postAndCheck( "Lisa" , "Title 1" , "Work" , "Content 1" )

postAndCheck( "Marge" , "Title 1" , "Home" , "Content 1" )

postAndCheck( "Maggie" , "Title 2" , "Home" , "Content 1" )

postAndCheck( "Maggie" , "Title 3" , "Food" , "Content 1" )

postAndCheck( "Maggie" , "Title 1" , "Travel" , "Content 1" )

postAndCheck( "Bart" , "Title 1" , "Home" , "Content 1" )

postAndCheck( "Bart" , "Title 1" , "Home" , "Content 2" )

postAndCheck( "Bart" , "Title 1" , "Home" , "Content 3" )

Page 198: Make Your Testing Groovy

Topics

• Introduction

• Web Drivers

• Test Runners

• Non-web Drivers

• Other Tools

• Going Beyond

Further Information

QCON 2010 - 198

© A

SE

RT

2006-2

010

Page 199: Make Your Testing Groovy

More Information...

199

© A

SE

RT

2006-2

010

• Yet more tools– http://grails.org/plugin/perf4j

– http://naleid.com/blog/2009/04/14/grails-build-test-data-01-plugin-

released/

– http://www.concordion.org/

– http://maxheapsize.com/2009/10/13/concordion-vs-cucumber-and-

java-based-acceptance-testing/

– http://www.pushtotest.com/

– http://maxq.tigris.org/, http://sonar.codehaus.org/

– http://celerity.rubyforge.org/, http://jcrawler.sourceforge.net/

– http://jameleon.sourceforge.net/

– http://databene.org/databene-benerator

– http://grinder.sourceforge.net/

Page 200: Make Your Testing Groovy

...More Information...

200

© A

SE

RT

2006-2

010

• Testing Web sites– http://www.testingreflections.com/

– http://testobsessed.com/

– http://www.agiletester.ca/

– http://www.testingspot.net/

– http://opensourcetesting.org/

Page 201: Make Your Testing Groovy

...More Information...

201

© A

SE

RT

2006-2

010

• Groovy Web sites– http://groovy.codehaus.org

– http://grails.codehaus.org

– http://pleac.sourceforge.net/pleac_groovy (many examples)

– http://www.asert.com.au/training/java/GV110.htm (workshop)

• Groovy User Mailing list– [email protected]

• Groovy Information portals– http://www.aboutgroovy.org

– http://www.groovyblogs.org

• Groovy Documentation (1000+ pages)– Getting Started Guide, User Guide, Developer Guide, Testing Guide,

Cookbook Examples, Advanced Usage Guide

Page 202: Make Your Testing Groovy

...More Information

202

© A

SE

RT

2006-2

010