Pex

Ben [email protected]

PexThe future of unit testing?

Agenda

• What is Pex?• Problem Pex is trying to solve• Pex in the Real World• The future?

What is Pex?

Program Exploration

Microsoft Research

Research in Software Engineering (RiSE)

Nikolai Tillmann Peli de Halleux

Wolfram Schulte Nikolaj Bjørner

Nikolai Tillmann Peli de Halleux

Wolfram Schulte Nikolaj Bjørner

Pex generates inputs for your unit tests through automate exploratory testing

Identify Test Inputs

• Pex doesn’t make wild guesses• Pex gains an understanding of your code• Static and dynamic analysis• Every input should increase code coverage

Parameterised Unit Tests

[RowTest][Row(1, 2, 3)] [Row(-1, -2, -3)] public void TestForAddMethod(int a, int b, int expected){ int test = MyMethods.Add(a, b); Assert.AreEqual(test, expected);}

Parameterised Unit Tests

[RowTest][Row(1, 2, 3)] [Row(-1, -2, -3)] public void TestForAddMethod(int a, int b, int expected){ int test = MyMethods.Add(a, b); Assert.AreEqual(test, expected);}

Everything Ian said is still valid

Exploratory Testing

• Learns more over time• More it knows, more it knows how to break it • Starts with simplest input

HELLO WORLDDEMO

Under the covers

CLR Profiler API

ICorProfilerCallback2

Z3 Constraint Solver

The Problem

Some people aren’t unit testing

Ok... Most people

Lots of people think they are writing tests but in reality they

aren’t

[Test]public void Test1(){ Engine e = new Engine(); Customer c = e.GetCustomer(123); Order o = new Order(1, “Product X”) o.Customer = c; Assert.IsTrue(e.PlaceOrder(o));}

Those who are, 100% code coverage is rare

Especially with Legacy Code

Test Driven Development on complex code is difficult

Encryption and Decryption algorithms

Parsers and Compilers

Decision Tables

Aim of Pex is to solve these problems

The Real World

DECISION TABLEDEMO

IP ADDRESS GENERATORDEMO

PEX EXTENSIONSDEMO

The Future

Pex has limitations

Remember – it’s still research

Difficult to start using

Remember – it’s still research

Not available under a commercial license

VS2008 == ResearchVS2010 == Same as VS2010 CTP

However!

Very good with algorithms and complex code

Used internally to test core component at Microsoft

Code Contracts help guide Pex

Spec# \ C# 4.0

BDD and Pex

In summary...

Summary

• Pex is a project from Microsoft research• Available today for VS2008 and VS2010 • Aims to make unit testing more effective• BDD + Pex

Ben [email protected]

Thank you

Resources

• http://Blog.BenHall.me.uk• http://research.microsoft.com/pex/• http://www.codeplex.com/pex• http://blog.dotnetwiki.org/ (Peli’s Blog)• http://blogs.msdn.com/nikolait• PDC Session -

http://channel9.msdn.com/pdc2008/TL51

Flickr Images

• http://www.flickr.com/photos/djbrady/1460272108/• http://www.flickr.com/photos/yukonblizzard/2562986305/• http://www.flickr.com/photos/frielp/6829509/• http://www.flickr.com/photos/atomicjeep/274667176/• http://www.flickr.com/photos/wouterpostma/695850212/• http://www.flickr.com/photos/leon_homan/2856628778/• http://www.flickr.com/photos/sadsnaps/2741479621/