Database Development: The Object-oriented and Test-driven Way

BT6 Concurrent Session 11/8/2012 2:15 PM

"Database Development: The Object-oriented and Test-driven

Way"

Presented by:

Max Guernsey Hexagon Software, LLC

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com

Max Guernsey Hexagon Software, LLC

As head of Hexagon Software, LLC and a software developer with fifteen years of experience, Max Guernsey helps organizations make the transition to agility with a focus on good technical practices such as test-driven development and modern object-oriented design. He develops his company’s DataClass product, a language that allows .NET developers to treat databases like “just another object.” Max is the author of Transition Testing, the foundation of his current TDD database work; Test Driven Database Development which focuses on adapting the discipline of TDD to forces present in the database world; and Goad Testingin which he discusses what unit tests really are and how to make them more resilient to design changes.

1

Database Development

The Test‐Driven and Object‐Oriented Way

© Copyright 2011 Hexagon Software LLC.All rights reserved. 19/12/2012

Max Guernsey, III

• Max Guernsey, III• 15 years as a developer• 15 years as a developer• Started w/DB early• Author:

– Test‐Driven Database Development

– Goad Testing


– Transition Testing

• [email protected]

2

Agenda

• Simplifying Assumptions• Establish the Problem• Roadmap to True TDD DB Development• Follow Roadmap

– Class of Databases– Safely Changing Design– Enforcing Interfaces


g– Defining Behaviors– Designing for Maintainability

• OO DB Development (Whirlwind Tour)

To fit in a 75 minute box


Simplifying Assumptions

3

Assumptions

• I assume that you knowI assume that you know...– ...know what TDD is– ...the difference between TDD and Test‐First– ...about mocking– ...a little about design patterns


• If not, there are great resources for all• Pick them up, read, & practice before talk

Know what’s in our way


Establish the Problem

4

Test Runner

[Test]public void AddEmployee() {

var company = Company GetInstance();

TDD in Middle Tier Code

Test Runner

Test1

Test2

Test3

ThisTest

Test5

var company = Company.GetInstance();var employee = Employee.GetInstance();

company.AddEmployee(employee);

Assert.That(company.Employees,Has.Some.SameAs(employee);

}

Setup

Trigger

Assertion


Test6

Test7

Test8

}

[Test]public void AddEmployee() {

var company = Company GetInstance();

TDD in Middle Tier Code

var company = Company.GetInstance();var employee = Employee.GetInstance();

company.AddEmployee(employee);

Assert.That(company.Employees,Has.Some.SameAs(employee);

}

Setup

Trigger

Assertion


}

5

[Test]public void CannotAddLineItemWithoutType() {

DBTestHelper ClearDatabase();

Typical DB Test

DBTestHelper.ClearDatabase();var cartId = DBTestHelper.CreateCart();

try {DBTestHelper.InsertLineItem(

cart: cartId, itemTypeId: DBNull.Value, count: 100);} catch (ConstraintViolationException) {

return;}

Incomplete setup

Trigger


}

Assert.Fail();}

Assertion

How Was the DB Created?

• You don’t know• No guarantee DB works like production• Potential for variation

• Leads to...– ...rigorous “controls” over change– ...general “slow down” culture


...general slow down culture– ...ultimately: bad database designs

• DB test suites must cover creation

6

How to solve the problem


Roadmap

Roadmap to TDD

• Creating a Class of Databases– So creation is covered by tests

• Testing Creation & Modification– So changes can be introduced w/confidence

• Enforcing Interfaces– So consumers are forced to recognize changes

• Defining “Behavior”


g– So your tests cover the right things

• Designing for Maintainability– So you don’t get bogged down by needless junk

7

There’s More!

• OODC ti t f l i l DB / d li & fl ibl d i– Creating systems of logical DBs w/good coupling & flexible designs

• Mocking– Real, useful mocking along interface boundaries

• Legacy Databases– Pulling legacy DBs into new process

• Non‐DB applications– Serialized objects, XML, file structures


• Won’t fit in 75 minutes• Shameless plug

– They’re in the book

You don’t want to run tests against production, do you?


Creating a Class of Databases

8

What is a Class of Databases?

• StrictlySt ct y– Provisions new DBs– Upgrades existing DBs– All DBs get same behavior

• Effecti el


• Effectively– Uniform create/upgrade path– All DBs “grew” same way

Problem: Different Build Paths

• Test databases need to be “cheap”est databases eed to be c eap– Easy to create & destroy

• Production databases need to last– Created w/scripts– Improved incrementally w/scripts

• Want all databases to ork the same


• Want all databases to work the same– Test & production

• Exactly the same

9

How to Make Test DBs?

• Test DBs are of low valueest s a e o o a ue• Especially when cheap to make• Break one?

– Pitch it• Got one dirty?


– Pitch it

• Make them however you like

How to Make Production DBs?

• Starts emptyStarts empty• Created w/series of instructions• Upgraded w/series of instructions to v2• Upgraded w/series of instructions to v3• Etc


• Etc.• No wiggle room

– What actually happens is what actually happens

10

Takeaway #1


The Sequence of Transitions

• v0 v1– Get a new v1 DB

• v1 v2– Upgrade from v1 to v2

• v0 v1 v2 v3 v4


– Get a new v4 DB• V2 v3 v4

– Upgrade from v2 to v4

11

Test Runner

[Test]public void HasAnAccountsTable() {

// you can do this once per suite if you want

Example: Writing a Test

Test Runner

Test1

Test2

Test3

ThisTest

Test5

// you can do this once per suite if you wantMyDatabase.Create(connection, 2);

connection.ExecuteSql("SELECT * FROM Accounts");

// exception thrown if no Accounts table}

DB created by test


Test6

Test7

Test8

<database><version number="1">

<script>

Example: Starting DB Script

<script>

</script></version>

</database>


12

Test Runner


<script>

Example: Updating DB Design

Test Runner

Test1

Test2

Test3

ThisTest

Test5

<script>

</script></version><version number="2">

<script><![CDATA[CREATE Accounts(INT ID, NAME CHAR(23));]]>


Test6

Test7

Test8

]]></version>

</database>

Reducing the risk of change rather than its frequency


Safely Changing Design

13

“Neat”

• Finding uniform build path a beginningd g u o bu d pat a beg g• Must drive error/variation from build

• Builders/installers/etc. are code• What drives error/variation from code?


• TESTS!!

Takeaway #2


14

A Transition Test

• SetupSetup– Bring database up to version X– Populate

• Trigger– Transition to subsequent version


• Assertion– Prove that content was preserved

Test Runner

[Test]public void ContactsFactoredFromAccountsInV9() {MyDatabase.Create(connection, 8);

Example: A Transition Test

Test Runner

Test1

Test2

Test3

ThisTest

Test5

var accountId = connection.ExecuteScalar(@"INSERT INTO Accounts(Name, PWHash, Email)VALUES('MaxGuernseyIII', 'pwhash', '[email protected]');

SELECT SCOPE_IDENTITY();");

MyDatabase.Create(connection, 9);

var emails = connection.ExecuteSelectList(@"SELECT Email FROM Contacts WHERE AccountID = ?",


Test6

Test7

Test8

accountId);Assert.That(emails,Is.EqualTo(new[] { "[email protected]" });

}

15

Test Runner


<script>

Example: Creating Upgrade Script

Test Runner

Test1

Test2

Test3

ThisTest

Test5

p<![CDATA[CREATE TABLE Contacts(

AccountId INT FOREIGN KEY REFERENCES Account(ID),Email NVARCHAR(2048));

INSERT INTO Contacts(AccountID, Email)SELECT ID, Email FROM Accounts;

ALTER TABLE A t DROP COLUMN E il


Test6

Test7

Test8

ALTER TABLE Accounts DROP COLUMN Email;]]>

</script></version>

</database>

Getting the best feedback you can


Enforcing Interfaces

16



Remember This?

// you can do this once per suite if you wantMyDatabase.Create(connection, 2);




Duplication Is Evil

• Two tests is annoyingTwo tests is annoying• Ten is a pain• Fifty is untenable • Any realistic number impossible to maintain w/duplication


w/duplication

17



Easy Solution

y p yMyDatabase.Create(connection, MyDatabase.CurrentVersion);



...


public class MyDatabase {public const int CurrentVersion = 2;...

}

It’s More Pervasive

• Version numbers just most obviousVersion numbers just most obvious• Symbols are the real evil...


18

[Test]public void CanInsertData() {connection.Execute(@"INSERT INTO Data(Time, Reading)VALUES('4:31:01 103 17 02 2009' 041 908);”);

Duplication Abounds

VALUES( 4:31:01.103 17.02.2009 , 041.908); );}...<script>CREATE TABLE Data(Time DATETIME, Reading NUMERIC(6,3));

</script>...private void EnterReading(double reading) {connection.Execute(@"INSERT INTO Data(Time, Reading) VALUES(?,?)",DateTime.Now, reading);


}...private IEnumerable<double> GetOrderedReadings() {return connection.Execute<double>(@"SELECT Reading FROM DataORDER BY Time ASC");

}

One Solution

• Minimize w/string constantsMinimize w/string constants• Couple to constants from tests & code

• What about interface changes?– Transition tests need old design


Transition tests need old design– Unit tests & production code always need latest

19

Another Solution

• Document design w/each transitionDocument design w/each transition• Couple transition tests to exact• Couple unit tests to latest• Couple production code to latest


• What about DB script?

A Really Good Solution

• Document design changes w/versionocu e t des g c a ges / e s o• Bind to design in scripts• Pre‐compile/package step:

– Inject bound symbols– Generate coupling codeC l i l d i


• Couple unit tests to latest design• Couple production code to latest design• Couple transition tests to specific design

20

<database><version number="2"><design>

Example Script

Generate code from this before compilation/packaging

<add id="NewTable"><add id="Col1" /><add id="Col2" />

</add></design><script><scope id="NewTable">

CREATE TABLE <id/>(<bind>Col1</bind> INT,

"Take the version 1 design and add thisstuff"

Replace w/ "NewTable" prior to packaging


<bind>Col2</bind> CHAR(1));</scope>

</script></version>

</database>

Replace w/ "Col2" prior to packaging

Takeaway #3


21

Replacing Symbol w/Itself?

• 1st‐ly: You can do morey ou ca do o e– E.g.: Use different DB and logical names

• 2nd‐ly: Gives advantages of compiler– across languages (SQL, C#, C++, etc.)

' b li il h d ?


• Don't believe compiler has advantages?– You're crazy– Keep it to yourself

If databases are objects, what are their behaviors?


Defining Behaviors

22

What Do Tests Test?

• Tests specify behaviorsTests specify behaviors• Objects contain behaviors• Databases are objects• DB tests should specify DB behaviors


Concepts

• Knowledge – persistent data of valueKnowledge persistent data of value• Information – useful part of signal• Data – superclass of knowledge & information• Behavior – Activity that produces a result

– Answer to a question


Answer to a question– Useful side effect

23

What Are Behaviors for Transitions?

• Transition behaviors simpleTransition behaviors simple• Change from old structure to new• Preserve knowledge

• Ignored for this segment


• Ignored for this segment

What Are DB Behaviors?

• DB behaviors slightly more subtleDB behaviors slightly more subtle• Absorb information & translate to knowledge• Translate knowledge into requested info.

• A table is no behavior


• A table is no behavior• A trigger is not behavior• Those are structure

24

Knowledge, Info, & Behavior

Client

DB

KnowledgeInfo.


TestBehavior

Takeaway #4


25

Don't and Do

Don't Do• Test a table• Test a trigger• Expose structure

– E.g.: tables etc.

• Test store & retrieval of info.• Test info causes side effect• Encapsulate structure

– w/Views & stored procs


Why Encapsulate Design?

• Weird question: Why not?Weird question: Why not?

• Hidden things cheap to change• Exposed ones expensive• Minimizing public interface maximizes


• Minimizing public interface maximizes flexibility

26

Doing less now enables doing more later


Designing for Maintainability

Don't Plan the Future

• Future is unpredictableFuture is unpredictable• You cannot plan it• Maybe your guess is right• Maybe we all die from asteroid strike• Usually: your guess "would have" been right


• Usually: your guess would have been right– If only universe had cooperated

27

Plan for the Future

• Plan for the future to happenPlan for the future to happen– You'll need to change stuff

• Design so that change is easiest• Largely: what's been shown so far• Additionally: don't overbuild


Additionally: don t overbuild

• "What does that mean?"

Takeaway #5


28

Planning for the Future

• Encapsulate design– So that structures can change

• Enforce interface strongly– So changes don't break downstream functionality

• Cover every behavior w/test– So changes don't introduce regression

• Cover every transition w/test


y /– So changes don't jeopardize knowledge

• Minimize features to those needed now– So needed features not impeded by useless ones

Time to watch this work


Live Example

29

A whirlwind tour


The Rest

OOD

• Once classes of DB used, OOD/OOP availableO ce c asses o used, OO /OO a a ab e• Use it• Divide & conquer

– Create composite DBs– Couple via public interface

b i & i i


• Even create abstractions & variations– Exploit division to introduce variation– Most important variation on next slide

30

Mocking

• Once you have OOD/OOP easy good mockingOnce you have OOD/OOP, easy, good mocking available

• Don't mock tables• Mock behaviors encapsulated in dependency DBs


Legacy DBs

• Mocking especially handy w/legacy DBsoc g espec a y a dy / egacy s• Create Façade DB, connects to legacy• Easy to test‐drive Façade behaviors:

– Just mock out legacy• Transfer behavior to Façade over time


• Alternatively:– "wrangle" legacy DB into tested state

31

Non‐DB Applications

• Much of this works for non‐DB– (not recommendation for views & sprocs, obviously)

• Have data? Want to evolve structure over time?– This process works

• E.g.:– File systems/directory structures


File systems/directory structures– Registry keys– XML documents– Serialized objects

Thanks for coming. Any other questions?


Q&A

Database Development: The Object-oriented and Test-driven Way

Technology

Database Development: The Object-oriented and Test-driven Way