Practical unit testing 2014

http://tinyurl.com/sf-rnt

http://spryfox.com/our-games/road-not-taken/ shut down wifi, skype. RECORD!

UnitTests

Practical

Andrew Fray, Spry Fox

Thank you for coming. This is Practical Unit Tests, and I’m Andrew Fray. My talk today is about how to structure your unit tests to aid iteration. We’re going to be looking in detail at my unit tests from a shipped AAA console project, the mistakes I made in writing them, and the concrete costs they caused. Then we’ll look at simple ways to avoid those costs in your own unit tests.!definitions Post mortem Anti-patterns

#pracunittests

Test Driven Development

move on!

#pracunittests

Backwards Is Forward: Making Better Games with Test-Driven Development

http://gdcvault.com/play/1013416/Backwards-Is-Forward-Making-Better

Sean Houghton, Noel Llopis

http://tinyurl.com/gddtdd

#pracunittests

2004

@tenpn

#pracunittests

2004

@tenpn

Definitions

#pracunittests

Unit TestSingle explicit assumption

IMPORTANT functional/integration interchangable

#pracunittests

Unit TestSingle explicit assumption

Integration TestMany implicit assumptions

IMPORTANT functional/integration interchangable

#pracunittests

Qualities of Good Unit Tests

Readable - why. up to speed. Maintainable - how. modify for iterating. Trustworthy - quick, deterministic. no external resources.

#pracunittests


Readable


#pracunittests


Readable

Maintainable


#pracunittests


Readable

MaintainableTrustworthy


PostMortem

#pracunittests

F1 2011 X360/PS3/PC

PRODUCTION CODE => SUBSYSTEM CODEHIGHLIGHT loc shipped 4+ previously

#pracunittests

F1 2011 X360/PS3/PC

• Isolated new subsystem


#pracunittests

F1 2011 X360/PS3/PC


• 502 tests, 6700 lines of test code


#pracunittests

F1 2011 X360/PS3/PC


• 502 tests, 6700 lines of test code

• 6200 lines of production code


#pracunittests

A Partial Succes


#pracunittests

• Clean, re-usable code

A Partial Succes


#pracunittests


• Fewer bugs

A Partial Succes


#pracunittests


• Fewer bugs

• Easy to optimise

A Partial Succes


#pracunittests


• Fewer bugs

• Easy to optimise

• At end, treacle-like progress

A Partial Succes


Unit TestAnti-Patterns

#pracunittests

1/4: The Opaque Anti-Pattern

types: light grey names: bold dark grey literals: red else: dark grey

#pracunittests

// in LinearDescriptionFixture… void testBackwardsToNormalLeftwardsGradient() { LinearDescription leftDesc = DefaultFlatLinearDescription() .withGradient(1f).withInitialValue(10.0f).withOffsetOrigin(0.0f) .withDirection(Direction.eLeft); TEST_GREATER(leftDesc.getValueAtOffset(-1.0f), 10.0f); }



#pracunittests

// in LinearDescriptionFixture… void testBackwardsToNormalLeftwardsGradient() { LinearDescription leftDesc = DefaultFlatLinearDescription() .withGradient(1f).withInitialValue(10.0f).withOffsetOrigin(0.0f) .withDirection(Direction.eLeft); TEST_GREATER(leftDesc.getValueAtOffset(-1.0f), 10.0f); } wat



#pracunittests

Opaque: Hard to see HOW

void testBackwardsToNormalLeftwardsGradient() { LinearDescription leftDesc = DefaultFlatLinearDescription() .withGradient(1f).withInitialValue(10.0f).withOffsetOrigin(0.0f) .withDirection(Direction.eLeft); TEST_GREATER(leftDesc.getValueAtOffset(-1.0f), 10.0f); }


#pracunittests




#pracunittests




#pracunittests

Opaque: No Magic Literals

break the rules when you need to!

#pracunittests

Opaque: No Magic Literalsvoid testBackwardsToNormalLeftwardsGradient() { float someValueAtOrigin = 10.0f; float someOrigin = 0.0f; float positiveGradient = 1.0f; float someLeftOfOrigin = someOrigin - 1.0f; !

LinearDescription leftDesc = DefaultFlatLinearDescription() .withGradient(positiveGradient).withInitialValue(someValueAtOrigin) .withOffsetOrigin(someOrigin).withDirection(Direction.eLeft); float valueToLeft = leftDesc.getValueAtOffset(someLeftOfOrigin); TEST_GREATER(valueToLeft, someValueAtOrigin); }


#pracunittests




#pracunittests




#pracunittests




#pracunittests

void testBackwardsToNormalLeftwardsGradient() { float someValueAtOrigin = 10.0f; float someOrigin = 0.0f; float positiveGradient = 1.0f; float someLeftOfOrigin = someOrigin - 1.0f; !


#pracunittests

void testBackwardsToNormalLeftwardsGradient() { float someValueAtOrigin = 10.0f; float someOrigin = 0.0f; float positiveGradient = 1.0f; float someLeftOfOrigin = someOrigin - 1.0f; !


#pracunittests

Opaque: Informative, Consistent Test Name

#pracunittests


void nameOfFunctionUnderTest_ContextOfTest_DesiredResultOfTest() {

#pracunittests



void testBackwardsToNormalLeftwardsGradient() {

#pracunittests



void

void withDirection_Left_InvertsGradient() {

#pracunittests

void withDirection_Left_InvertsGradient() { float someValueAtOrigin = 10.0f; float someOrigin = 0.0f; float positiveGradient = 1.0f; float someLeftOfOrigin = someOrigin - 1.0f; !


#pracunittests

void withDirection_Left_InvertsGradient() { float someValueAtOrigin = 10.0f; float someOrigin = 0.0f; float positiveGradient = 1.0f; float someLeftOfOrigin = someOrigin - 1.0f; !


#pracunittests

Opaque: Arrange-Act-Assert

#pracunittests

Opaque: Arrange-Act-Assertvoid withDirection_Left_InvertsGradient() { float someValueAtOrigin = 10.0f; float someOrigin = 0.0f; float positiveGradient = 1.0f; LinearDescription increasingDesc = DefaultFlatLinearDescription() .withGradient(positiveGradient).withInitialValue(someValueAtOrigin) .withOffsetOrigin(someOrigin); !

LinearDescription leftDesc = increasingDesc.withDirection(Direction.eLeft); !

float someLeftOfOrigin = someOrigin - 1.0f; float valueToLeft = leftDesc.getValueAtOffset(someLeftOfOrigin); TEST_GREATER(valueToLeft, someValueAtOrigin); }

#pracunittests




#pracunittests




#pracunittests




#pracunittests




#pracunittests

The Opaque Anti-Pattern

breathe at end!

#pracunittests

The Opaque Anti-Pattern• Hard to see "how"?

breathe at end!

#pracunittests


• Demystify magic literals

breathe at end!

#pracunittests



• Consistent informative test name

breathe at end!

#pracunittests



• Consistent informative test name

• Arrange-Act-Assert

breathe at end!

#pracunittests

2/4: The Wet Anti-Pattern

RacingLineOffsets.setSignedDistanceToRacingLine(RacingLine, float)

On refactoring

#pracunittests

RacingLineOffsets.setSignedDistanceToRacingLine(RacingLine, float, int)


RacingLineOffsets

On refactoring

#pracunittests

RacingLineOffsets.setSignedDistanceToRacingLine(RacingLine, float, int)


> Test library build failed with 235 error(s)

RacingLineOffsets

On refactoring

#pracunittests

void getIdealOffset_BeyondRacingLineEdge_ReturnsEdge() {

float someRacingLineRadius = 10.0f;

float someOffsetBeyondRacingLine = someRacingLineRadius + 1.0f;

! RacingLineOffsets beyondOffsets = new RacingLineOffsets();

float leftRacingLineEdge =

-someRacingLineRadius - someOffsetBeyondRacingLine;

beyondOffsets.setSignedDistanceToRacingLine(

RacingLine.eLeftEdge, leftRacingLineEdge);


RacingLine.eCenter, -someOffsetBeyondRacingLine);

float rightRacingLineEdge =

someRacingLineRadius - someOffsetBeyondRacingLine;


RacingLine.eRightEdge, rightRacingLineEdge);

! OffsetRequest idealRequest =

m_raceBehaviour.getIdealOffset(beyondOffsets);

! float idealRacingLineOffset = idealRequest.GetOffset();

TEST_EQUAL(idealRacingLineOffset, someRacingLineRadius);

}

void getIdealOffset_WithinRacingLineEdge_ReturnsOffset() { float someRacingLineRadius = 10.0f; float someOffsetWithinRadius = someRacingLineRadius * 0.8f; ! RacingLineOffsets withinOffsets = new RacingLineOffsets(); float leftRacingLineEdge = -someRacingLineRadius - someOffsetWithinRadius; withinOfffsets.setSignedDistanceToRacingLine( RacingLine.eLeftEdge, leftRacingLineEdge); withinOffsets.setSignedDistanceToRacingLine( RacingLine.eCenter, -someOffsetWithinRadius); float rightRacingLineEdge = someRacingLineRadius - someOffsetWithinRadius; withinOffsets.setSignedDistanceToRacingLine( RacingLine.eRightEdge, rightRacingLineEdge); ! OffsetRequest idealRequest = m_raceBehaviour.getIdealOffset(withinOffsets); ! float idealRacingLineOffset = idealRequest.GetOffset(); TEST_EQUAL(idealRacingLineOffset, someOffsetWithinRadius); }

#pracunittests



float someOffsetBeyondRacingLine = someRacingLineRadius + 1.0f;

! RacingLineOffsets beyondOffsets = new RacingLineOffsets();

float leftRacingLineEdge =

-someRacingLineRadius - someOffsetBeyondRacingLine;


RacingLine.eLeftEdge, leftRacingLineEdge);


RacingLine.eCenter, -someOffsetBeyondRacingLine);

float rightRacingLineEdge =

someRacingLineRadius - someOffsetBeyondRacingLine;


RacingLine.eRightEdge, rightRacingLineEdge);

! OffsetRequest idealRequest =


! float idealRacingLineOffset = idealRequest.GetOffset();

TEST_EQUAL(idealRacingLineOffset, someRacingLineRadius);

}

void getIdealOffset_WithinRacingLineEdge_ReturnsOffset() { float someRacingLineRadius = 10.0f; float someOffsetWithinRadius = someRacingLineRadius * 0.8f; ! RacingLineOffsets withinOffsets = new RacingLineOffsets(); float leftRacingLineEdge = -someRacingLineRadius - someOffsetWithinRadius; withinOfffsets.setSignedDistanceToRacingLine( RacingLine.eLeftEdge, leftRacingLineEdge); withinOffsets.setSignedDistanceToRacingLine( RacingLine.eCenter, -someOffsetWithinRadius); float rightRacingLineEdge = someRacingLineRadius - someOffsetWithinRadius; withinOffsets.setSignedDistanceToRacingLine( RacingLine.eRightEdge, rightRacingLineEdge); ! OffsetRequest idealRequest = m_raceBehaviour.getIdealOffset(withinOffsets); ! float idealRacingLineOffset = idealRequest.GetOffset(); TEST_EQUAL(idealRacingLineOffset, someOffsetWithinRadius); }

Not DRY

#pracunittests

Wet: Helper Functionsvoid getIdealOffset_WithinRacingLineEdge_ReturnsOffset() {


float someOffsetWithinRadius =

someRacingLineRadius * 0.8f;

!

RacingLineOffsets withinOffsets = CreateRacingLineOffsets(

someRacingLineRadius, someOffsetWithinRadius);

!

OffsetRequest idealRequest =

m_raceBehaviour.getIdealOffset(withinOffsets);

!

float idealRacingLineOffset = idealRequest.GetOffset();

TEST_EQUAL(

idealRacingLineOffset, someOffsetWithinRadius);

}



float someOffsetBeyondRacingLine =

someRacingLineRadius + 1.0f;

!

RacingLineOffsets beyondOffsets = CreateRacingLineOffsets(

someRacingLineRadius, someOffsetBeyondRacingLine);

!



!


TEST_EQUAL(

idealRacingLineOffset, withinOffsets.RightEdge);

}

#pracunittests





!



!



!


TEST_EQUAL(


}





!



!



!


TEST_EQUAL(


}

#pracunittests





!



!



!


TEST_EQUAL(


}





!



!



!


TEST_EQUAL(


}

#pracunittests





!



!



!

test_tauOffsetEqual(idealRequest, someOffsetWithinRadius);

}





!



!



!

test_racingLineOffsetEqual(

idealRequest, withinOffsets.RightEdge);

}

#pracunittests





!



!



!

test_tauOffsetEqual(idealRequest, someOffsetWithinRadius);

}





!



!



!

test_racingLineOffsetEqual(

idealRequest, withinOffsets.RightEdge);

}

#pracunittests

The Wet Anti-Pattern

breathe at end!

#pracunittests

The Wet Anti-Pattern• Hard-to-maintain hacky tests?

breathe at end!

#pracunittests


• Keep production sensibilities in unit test code

breathe at end!

#pracunittests



• Stay DRY with helper functions and custom asserts

breathe at end!

#pracunittests



• Stay DRY with helper functions and custom asserts

• Do not hide the call to the function under test

breathe at end!

#pracunittests

3/4: The Deep Anti-Pattern

> Test failed: > getOwnerAtOffset_WithOwner_ReferencesOwnerAtManyOffsets > With Assert: VehicleID 0 != 1

on introducing a bug chi == “kai”!

#pracunittests

void getOwnerAtOffset_WithOwner_ReferencesOwnerAtManyOffsets() { VehicleID someOwnerID = (VehicleID)1; LinearDescription ownedDescription = DefaultFlatLinearDescription().withOwner(someOwnerID); !

float someNegativeOffset = -1.0f; float ownerAtNegativeOffset = ownedDescription.getOwnerAtOffset(someNegativeOffset); float somePositiveOffset = 1.0f; float ownerAtPositiveOffset = ownedDescription.getOwnerAtOffset(somePositiveOffset); !

test_OwningVehicleIsEqualTo(ownerAtNegativeOffset, someOwnerID); test_OwningVehicleIsEqualTo(ownerAtPositiveOffset, someOwnerID); }

make it clear

#pracunittests




make it clear

#pracunittests




make it clear

#pracunittests




>1 explicit

assumption

make it clear

#pracunittests

Deep: One Assert Per Testvoid getOwnerAtOffset_WithOwnerAndNegativeOffset_ReturnsOwner() { float someNegativeOffset = -1.0f; !

float ownerAtNegativeOffset = m_ownedDescription.getOwnerAtOffset(someNegativeOffset); !

test_OwningVehicleIsEqualTo(ownerAtNegativeOffset, m_someOwnerID); } !

void getOwnerAtOffset_WithOwnerAndPositiveOffset_ReturnsOwner() { // *snip* }

#pracunittests

> Test failed: > getOwnerAtOffset_WithOwnerAndNegativeOffset_ReturnsOwner > With Assert: VehicleID 0 != 1 > Test failed: > getOwnerAtOffset_WithOwnerAndPositiveOffset_ReturnsOwner > With Assert: VehicleID 0 != 1

#pracunittests

The Deep Anti-Pattern

breathe at end!

#pracunittests


• Test failures not fully informative?

breathe at end!

#pracunittests



• Too many explicit assumptions per test

breathe at end!

#pracunittests



• Too many explicit assumptions per test

• Minimise assumptions per test

breathe at end!

#pracunittests

4/4: The Wide Anti-Pattern

On introducing a bug in behaviour system

#pracunittests

4/4: The Wide Anti-Pattern

> Executed 613 test(s), 599 test(s) passed, 14 test(s) failed.

On introducing a bug in behaviour system

#pracunittests

// in DraftBehaviourFixture… void updateImpl_WithCarToDraft_PushesAIToBehindCar() { WorldInfo worldInfo = new WorldInfo(); float someDraftTargetVehicleOffset = 2.0f; // *snip* blackboard setup !

DraftBehaviour draft = new DraftBehaviour(); !

BehaviourSystem behaviourSystem = new BehaviourSystem(); behaviourSystem.addBehaviour(draft); !

behaviourSystem.updateWithBlackboard(worldInfo); !

MovementRequest draftMovement = behaviourSystem.getMovementRequest(); TEST_EQUAL(draftMovement.desiredRacingLineOffset, someDraftTargetVehicleOffset); }

#pracunittests






#pracunittests






#pracunittests






>0 implicit

assumptions

#pracunittests






#pracunittests






#pracunittests






#pracunittests






#pracunittests

Wide: Seamsvoid DraftBehaviour.updateImpl(WorldInfo wi, HeatMap heatInOut);

Game Code

Inversion of Control Interfaces/virtual functions Mocking frameworks/helper classes

#pracunittests


class HeatMap { virtual void WriteHeat(float offset, float value) { … } }

Game Code


#pracunittests



class MockHeatMap : HeatMap { override void WriteHeat(float offset, float value) { … } }

Game Code

Test Library


#pracunittests



class MockHeatMap : HeatMap { override void WriteHeat(float offset, float value) { … } }

Game Code

Test Library


#pracunittests

Wide: MockHeatMapvoid updateImpl_WithCarToDraft_PushesAIToBehindCar() { WorldInfo worldInfo = new WorldInfo(); float someDraftTargetVehicleOffset = 2.0f; // *snip* blackboard setup !

DraftBehaviour draft = new DraftBehaviour(); MockHeatMap mockMap = new MockHeatMap(); !

draft.updateImpl(worldInfo, mockMap); !

float draftOffsetWithHighestHeat = mockMap.getHighestHeatOffset(); TEST_EQUAL(draftOffsetWithHighestHeat, someDraftTargetVehicleOffset); }

#pracunittests





#pracunittests





#pracunittests

The Wide Anti-Pattern

breathe at end!

#pracunittests

The Wide Anti-Pattern• False-negative test failures?

breathe at end!

#pracunittests


• Many implicit assumptions

breathe at end!

#pracunittests


• Many implicit assumptions

• Isolate code with seams, to enable simple fake impostors

breathe at end!

#pracunittests

Recap

Respect unit test quality as much as production code quality Write once, read many Only 1 explicit assumption As few as possible implicit assumptions

#pracunittests

Recap• Respect unit test source code as much as

production source code


#pracunittests



• Write once, read many


#pracunittests




• Only 1 explicit assumption


#pracunittests




• Only 1 explicit assumption

• Minimise implicit assumptions


#pracunittests

• [email protected]

• @tenpn

• andrewfray.wordpress.com

• Roy Osherove: Art of Unit Testing www.artofunittesting.com

• Michael Feathers: Working Effectively with Legacy Code

• Steve Freeman & Nat Pryce: Growing Object-Orientated Software, Guided By Tests

Colour scheme by Miaka www.colourlovers.com/palette/444487/Curiosity_Killed

Feedback! Thank the CAs!

Practical unit testing 2014

Documents

pracunittests opaque

pracunittests f1

pracunittests clean

subsystem code highlight

x360ps3pc production

partial succes production

bold dark grey literals

light grey names