Peli de Halleux, Nikolai Tillmann Research in Software Engineering Microsoft Research, Redmond, USA.

Lightweight Test Stubs and Moles for .NET

Peli de Halleux, Nikolai TillmannResearch in Software EngineeringMicrosoft Research, Redmond, USA

1999… people packing groceries for the Y2k bug

How do you replace DateTime.Now?

The Y2K bug DEMO

if (DateTime.Now == new DateTime(2000,1,1)) throw Y2KBugException();

DateTime.Now = () => new DateTime(2000,1,1);

DateTime.Now

MDateTime.NowGet = () => new DateTime(2000,1,1);

Moles

Delegates naming convention

Lambda Expressions and Statements

(C# 3.0 Syntax)

delegate void Action<T>(T t); // void f(int i);delegate R Func<T, R>(T t); // int f(string i);// C# 2.0Func<string, int> f = delegate(string s) {return 0; }

// C# 3.0Func<string, int> f = (s) => 0Func<string, int> f = (s) => { return 0; }Func<string, int> f = _ => 0

Y2K BugDEMO

Isolation in Unit Testing

Motivation

A unit test is a small program with assertions

Tests a single (small) unit of code in isolation

Reality check: Real unit tests are not that simple!

Unit Testing

void ReadWrite() { var list = new List(); list.Add(3); Assert.AreEqual(1, list.Count);}

Unit Testing is not that easy

Components depend on other components

Hidden Integration Testsvoid FileExistsTest() { File.Write(“foo.txt”, “”); var result = IsFileEmpty(“foo.txt”) Assert.IsTrue(result);}

bool IsFileEmpty(string file) { var content = File.ReadAllText(file); return content.Length == 0;}

File.ReadAllText(file);

File.Write(“foo.txt”, “”);

Isolation is critical

Slow, complicated setup, non-deterministic tests

Solution: Replace by Simpler Environment (“mocking”)

Testable Design: Abstraction layer + Dependency Injection + Mocks for testing Simply uses virtual methods

Hard-coded Design: No abstraction layer, static methods, sealed types. Runtime rewriting needed

Stubs and Moles Framework

Replace Any .NET method with A Delegate

Method can be overridden? Use Stubs Interfaces, Abstract classes, Virtual methods in non-sealed types

Method cannot be overridden? Use Moles Static methods, Sealed types, Inline Constructor calls

Stubs for Unit Testing

Testable Design

Introduce abstraction for external components Replace them with something simpler, i.e. a Mock

bool IsFileEmpty(IFileSystem fs, string file) { var content = fs.ReadAllText(file); return content.Length == 0;}

void FileExistsTest() { IFileSystem fs = ???; fs.Write(“foo.txt”, “”); var result = IsFileEmpty(fs,“foo.txt”) Assert.IsTrue(result);}

IFileSystem fs

IFileSystem fs = ???;

Mock, Stub, Double, Fake, …

Stubs – Delegate Based Stubs

Replace Any .NET method with A Delegate

var fs = new SIFileSystem() { ReadAllTextString = file => “”;};

file => “”;

interface IFileSystem { string ReadAllText(string file);}

class SIFileSystem : IFileSystem {Func<string,string> ReadAllTextString; string IFileSystem.ReadAllText(string file) { return this.ReadAllTextString(file);}} // </auto-generated>

Func<string,string> ReadAllTextString;

this.ReadAllTextString(file);

string ReadAllText(string file);

IFileSystem DemoDEMO

Moles for Unit Testing

Hard-coded Design

Existing external components cannot be re-factored SharePoint, Asp.NET, VSTO

Need mechanism to stubnon-virtual methods Static methods, methods in sealed

types, constructors MSIL code rewriting required

Other Tools provide this functionality

Moles – Delegate Based Detours

Method redirected to user delegate, i.e. moled

Requires Code Instrumentation,e.g. via Profiler!

Pex provides [HostType(“Pex”)] NUnit, xUnit, etc… also supported

bool result = IsFileEmpty(“foo.txt”);Assert.IsTrue(result);

MFile.ReadAllTextString = file => “”;

Moles under the Hood

File.ReadAllText(string name) {

}

mscorlib.dll

File.ReadAllText(string name) { var d = GetDetour(); if (d != null) return d();

}

push ecxpush edxpush eax

.NET RuntimeJust In Time

Compiler

ExtendedReflection

File.ReadAllText DemoDEMO

Stubs and Moles

Lightweight Framework Type Safe Refactorable

Testable and “Hard-coded” Code Overridable methods -> Stubs Any other -> Moles

Delegate Based – use the language!

Stubs, Moles and PexIsolated Parameterized Unit Testing

Parameterized Unit Testing

var list = new List(capacity); list.Add(item); var count = list.Count;

Assert.AreEqual(1, count);}

A Unit Test has Three essential ingredients: Data Method Sequence Assertionsvoid Add() { int item = 3; int capacity = 4;// for all item, capacity...void Add(int item, int capacity) {

void List.Add(T item) { if (this.count >= this.Capacity) this.ResizeArray(); this.items[this.count++] = item;}

if (this.count >= this.Capacity)Capacity =

0 Test Case!

Isolated Parameterized Unit Testing

Automated White box Analysis does not ‘understand’ the environment

Isolate Code using Stubs and Moles

if (DateTime.Now == new DateTime(2000,1,1)) throw new Y2KException();

DateTime.Now

???

Void Y2k(DateTime dt) { MDateTime.NowGet = () => dt ...}

MDateTime.NowGet = () => dt DateTime.Now ==

dt

Stubs, Moles and PexIsolated Parameterized Unit Testing

DEMO

Future standalone download

Pex Components

ExtendedReflectionRuntime Code Instrumentation

Source Code Generation

MolesStubs

Z3Constraint

Solver

PexTest Generation

Automated White box Analysis

Stubs and Moles in Details

Stubs Naming Conventions

Types

Methods

Properties

Bar.IFoo -> Bar.Stubs.SIFoo

void Foo(string v) -> FooString

String Value {get;} -> ValueGet

Moles Naming Conventions

Types

Methods

Properties

Bar.Foo -> Bar.Stubs.MFoo

void Foo(string v) -> FooString

string Value {get;} -> ValueGet

Moles Type Structure

class Foo { static int StaticMethod() {…} int InstanceMethod() {…}}

class MFoo : MoleBase<Foo> { static Func<int> StaticMethod { set; } Func<int> InstanceMethod { set; }

implicit operator Foo (MFoo foo);}

Side Effects for free

Compiler generates closures for usvoid Test(string content) { var fs = new SIFileSystem(); bool called = false; fs.ReadAllText = file => { called = true; return content; }; ... Assert.IsTrue(called);}

bool called = false;

called = true;

called

Recursive Stubs

For free with Object Initializersinterface IBar { IFoo Foo {get;} } interface IFoo { string Value {get;} }

var bar = new SIBar { FooGet = () => new SIFoo { ValueGet = () => “hello” }};

IBar bar = …if(bar.Foo.Value == “hello”) ...

new SIBar().Foo.Value

Recursive Moles

For free with Object Initializers

class Bar { public Foo Foo {get;} }class Foo { public string Value {get;} }

MBar.Constructor = me => { new Mbar(me) => { FooGet = () => new MFoo { ValueGet = () => “hello”}}}

if(new Bar().Foo.Value == “hello”) ...new Bar()

.Foo.Value

Recursive Moles And Stubs

It just works!

class Bar { public Foo Foo {get;} }interface IFoo {string Value {get;} }

MBar.Constructor = (me) => { new Mbar(me) => { FooGet = () => new SIFoo { ValueGet = () => “hello”}}}

if(new Bar().Foo.Value == “hello”) ...new Bar()

.Foo.Value

Bind = Runtime Duck Typing

Bind all methods of an interface at onceclass Foo : IEnumerable<int> {...}

int[] values = {1,2,3};

var foo = new MFoo() .Bind(values); // bind all methods of // Ienumerable<int>

Trapping Environment

Set a trap to flag any call to a type

Iteratively build the mole sequence

MFoo.FallbackToNotImplemented();

Per Instance Moles

Dispatching moles per instanceclass Foo { public int Value {get;}}

var foo = new MFoo { ValueGet = () => 1 };var bar = new MFoo { ValueGet = () => 2 };

Assert.IsTrue(foo.Value != bar.Value);

Moles for New Objects

Attach Mole when Contructor is runclass Foo { public Foo() {} public int Bar() {…}}

MFoo.Constructor = me => { var foo = new MFoo(me) { Bar = () => 10 }; MFoo.Constructor = null; // only 1 instance};

MFoo.Constructor = _ => new MFoo(_) { Bar = () => 10 };

Partial Stubs

Stubs inherited from class may call base implementation

abstract class FooBase { public virtual string Value {get;}}

var foo = new SFooBase() { CallBase = true; }// call base class if no stub providedvar value = foo.Value;

CallBase = true;

Stubs Fallback Behavior

Defines behavior when stub not provided Default throws exceptioninterface IFoo { string Value {get;}}

StubFallbackBehavior.Current = StubFallbackBehavior.Default; var foo = new SIFoo();var value = foo.Value; // returns null

Moles Fallback Behavior

Defines behavior when mole not provided Default throws exceptionclass Foo { string Value {get;}}

var foo = new MFoo() { InstanceFallbackBehavior = MoleFallbackBehavior.Default }.Instance;

var value = foo.Value; //returns null

Pex Ready Stubs

Pex automatically detects and uses Stubs

Pex provides return valuesinterface IFoo { string Value {get;}}

[PexMethod]void Test(IFoo foo) { // pex uses SIFoo if (foo.Value == “foo”) throw ...; // pex chooses value

Give your feedback,Shape the future