Www.classfive.comDate Jim Moore, ClassFive LLC, CTO Unit Testing and Test Automation NOVATaig May 13, 2009.

www.classfive.com Date

Jim Moore, ClassFive LLC, CTO

Unit Testing and Test Automation

NOVATaig May 13, 2009

2

Topics

•Concepts and terminology of Unit Testing•Benefits and Limitations of Unit Test•See sample unit tests and test fixtures•Related technologies and methodologies•Discuss where Unit Test fits into overall testing methodology•Share knowledge

3

About Jim & ClassFive

Jim – Software Engineer – Multimedia, Office Automation, Point of Sale, Training, Consulting – 20 years exp

ClassFive - Focused on small to medium associations for IT consulting and providing Association Management systems and collabortaion tools

4

Why Unit Testing for ClassFive?

“Begin with the end in mind” -- Stephen Covey (The 7 Habits of Highly Effective People)

Helpful toolset/methodology to help creating a new product codebase that is highly stable, changeable and usable.

Something we can confidently sell and still sleep at night.

5

Challenges starting Unit Test

•We don’t have time to do this!•It won’t make meaningful difference.•My code doesn’t need tests!•Just make it work now!•How many unit tests do we need?•What toolset to use?•How far to go? (e.g. Mock objects, CI)•Deadlines vs process improvement•Can we reduce our testing staff if we unit test?

6

Challenges starting Unit Test – Part 2

Hey!

What is this Unit Testing thing – and what does it do?

7

This is a Unit

public class MyMath { public int Add(int i, int j) { return i + j; } }

8

Definition: Unit

The smallest piece of code software that can be tested in isolation.

9

This is a Unit Test

public void MyAddTest() { MyMath m = new MyMath(); Assert.AreEqual(m.Add(2, 3), 5); }

10

Definition: Unit Test

A unit test is a piece of code (usually a method) that invokes another piece of code and checks the correctness of some assumptions afterward. If the assumptions turn out to be untrue, the unit test has failed.

11

Unit Tests – Independence

•Units are tested independent of each other and their environment.•Tests should only test one thing.•Try to keep a traceable connection between potential bugs and test code.

12

Why Unit Test?

• Implicit documentation• Simplifies Integration• Improves code design – code becomes highly decoupled due to 2 users : application and test harness – code resists transition from 1 user to 2 users, the rest are easy• Facilitates Change• Allows programmer to refactor code at a later date and show individual parts are correct

• Continuous unit tests support sustained maintenance

13

Limitations of Unit Testing

•Don’t expect Unit Testing to solve all problems or catch all defects – not silver bullet•Will not catch integration errors – focuses on individual units•Only 1 pare of an effective testing methodology – cover all bases•Requires rigorous discipline•Requires sustainable process of construction and remediation

14

What is Quality?

15

Quality – definition

Value to some person (Jerry Weinberg).

16

More about Quality

•What is quality software?•Who decides?•Pretty front vs. termites in the framing.•Technical quality vs. Business value

17

Elements of a Good Unit Test Fixture

•Smell test (first reaction)•Unit test hierarchy should parallel units•Naming convention clear and consistent•Good coverage of units in module•Low to no % of unintended failures•Test intended failures, not just expected behavior

18

Unit Test Quality Types (Good, Bad, Ugly)

Good – effective, well designed testsBad – ineffective (wrong) tests (who cares about design?)Ugly – effective, poorly designed tests

19

Elements of a Good Unit Test

•Automatic • Invocation

• Checking

•Thorough• Depends on needs of project

•Repeatable•Independent•Professional

• Test code is “real code”(Source: Pragmatic Unit Testing with C#)

20

Software Development Testing Stages

Unit Testing – testing of individual componentsIntegration Testing – testing to expose problems from combination of componentsSystem Testing – test complete system prior to deliveryAcceptance Testing – by users to check system from satisfaction of requirements

21

Unit Test vs. Acceptance Test

•Unit Test – satisfy programmer that software does what programmer thinks it does

•Acceptance Test – satisfy customer that software provides business value and makes them willing to pay for software

22

What is a Unit Test Framework?

23

What is a Unit Test Framework

•Automatically run a set of Unit Tests

•Collects Unit Tests into cohesive groups (e.g. by module)

•Collects Unit Test results into a consolidated report

•Provides setup and teardown support to unit tests

24

What is a Test Harness?

Program that executes set of Test Fixtures, collects results and produces combined report.

25

What is a Test Fixture?

•Code that executes tests

•Phases:• Setup Environment

• Execute Test

• Teardown Environment

26

Sample Unit code

using System;namespace MyApp{ public class MyMath { public int Add(int i, int j) { return i + j; } }}

27

•Sample Test Fixture

using System;using NUnit.Framework;using MyApp; namespace MyAppTest{ [TestFixture] public class Class1 { [Test] public void MyAddTest( ) { MyMath m = new MyMath( ); Assert.AreEqual(m.Add(2, 3), 5); } }}

28

Unit Test Frameworks for All

•Java – JUnit, Cactus, EasyMock•Ruby – Test:Unit, RSpec•.Net - Nunit, TestDriven.Net, NMock•Javacript – JSUnit, JSpec, •Python – PyUnit

List of frameworks can be found at : http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

29

StringTemplate Unit Tests

StringTemplate is template library (e.g. merge data into template file)•Substantial set of unit tests

•Originally written in Java (and ANTLR)

•Ported to C# and Python

30

StringTemplate Unit Test Example – C#

C# version

[Test] public virtual void testSingleExprTemplateArgument() { string templates = "" + "group test;" + NL

+ "test(name) ::= \"<bold(name)>\"" + NL+ "bold(item) ::= \"*<item>*\"" + NL;

StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates), typeof(AngleBracketTemplateLexer)); StringTemplate e = group.GetInstanceOf("test"); e.SetAttribute("name", "Ter"); string expecting = "*Ter*"; string result = e.ToString();

Assert.AreEqual(expecting, result);}

31

StringTemplate Unit Test Example - Java

public void testSingleExprTemplateArgument() throws Exception {String templates =

"group test;" +newline+"test(name) ::= \"<bold(name)>\""+newline+"bold(item) ::= \"*<item>*\""+newline;

StringTemplateGroup group =new StringTemplateGroup(new StringReader(templates),

AngleBracketTemplateLexer.class);StringTemplate e = group.getInstanceOf("test");e.setAttribute("name", "Ter");String expecting = "*Ter*";String result = e.toString();assertEquals(expecting, result);

}

32

Common Unit Test Patterns

•Correct results•Boundary conditions correct•Inverse relationships checked (e.g. use result to check itself)•Cross check results by other means•Force error condition to occur•Was performance within bounds (e.g. impact of data set size, network latency)

33

Common Unit Test Patterns

Boundary Conditions:

•Format conformance•Result ordering•Input data range•External references required state•Resource existence (e.g database, file)•Correct number items (e.g. 0, 1, more than 1)•Time (e.g. timeout length, tolerance of delay)

34

What are Mock Objects?

35

Mock Objects

Simulated objects that mimic the behavior of real objects in controlled ways.

36

What is Refactoring?

37

Refactoring

The process of changing module’s internal structure without modifying its external functional behavior or existing functionality.

This is done to: • Improve internal non-functional properties of the software (e.g. readability)• Simplify code structure (e.g. adhere to a given programming paradigm), • Improve maintainability• Improve performance• Improve extensibility

38

What is Test Driven Development?

39

Test Driven Development (TDD)

A software development technique that uses short development iterations based on pre-written test cases that define desired improvements or new functions.

Each iteration produces code necessary to pass that iteration's tests. Finally, the programmer or team refactors the code to accommodate changes.

A key TDD concept is that preparing tests before coding facilitates rapid feedback changes.

40

TDD Steps

•Write tests that fail•Implement modules to make test pass.•Run all tests to ensure nothing broken•Refactor to keep code clean•Continue development

41

What is Continuous Integration?

42

Continuous Integration

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day.

Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.

43

Continuous Integration Practices

* Maintain a Single Source Repository. * Automate the Build * Make Your Build Self-Testing * Everyone Commits Every Day * Every Commit Should Build the Mainline on an Integration Machine * Keep the Build Fast * Test in a Clone of the Production Environment * Make it Easy for Anyone to Get the Latest Executable * Everyone can see what's happening * Automate Deployment

(Souce: http://martinfowler.com/articles/continuousIntegration.html)

44

Continuous Integration Tools

Version Control: Subversion, CVS, Git, SourceSafe

Build Tools: Ant, NAnt, MSBuild, Make

CI Server: CruiseControl, CruisControl.Net, TeamCity

Unit Test: NUnit, JUnit, NMock, EasyMock, etc

45

Unit Test As Software Test Automation

•Unit test execution best when part of an automated build system. •Unit Test practice can directly affect the quality of the code produced.•Good set of automated tests allow for incremental creation of code quality.•Unit Test starts with immature codebase while other automation requires mature codebase and design

46

Summary

•Unit Tests are independent tests of small chunks of code created during development.

•There are good Unit Test frameworks and harnesses available for many languages and environments.

•Unit Testing can directly affect the quality of code during construction.

•Unit Testing is only one part of an effective test automation methodology.

47

Q & A

www.classfive.com

Thanks for coming!

Jim Moore

CTO

ClassFive, LCC

[email protected]

http://www.classfive.com/