Page 1
BT6 Concurrent Session 11/8/2012 2:15 PM
"Database Development: The Object-oriented and Test-driven
Way"
Presented by:
Max Guernsey Hexagon Software, LLC
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Page 2
Max Guernsey Hexagon Software, LLC
As head of Hexagon Software, LLC and a software developer with fifteen years of experience, Max Guernsey helps organizations make the transition to agility with a focus on good technical practices such as test-driven development and modern object-oriented design. He develops his company’s DataClass product, a language that allows .NET developers to treat databases like “just another object.” Max is the author of Transition Testing, the foundation of his current TDD database work; Test Driven Database Development which focuses on adapting the discipline of TDD to forces present in the database world; and Goad Testingin which he discusses what unit tests really are and how to make them more resilient to design changes.
Page 3
1
Database Development
The Test‐Driven and Object‐Oriented Way
© Copyright 2011 Hexagon Software LLC.All rights reserved. 19/12/2012
Max Guernsey, III
• Max Guernsey, III• 15 years as a developer• 15 years as a developer• Started w/DB early• Author:
– Test‐Driven Database Development
– Goad Testing
© Copyright 2011 Hexagon Software LLC.All rights reserved. 29/12/2012
– Transition Testing
• [email protected]
Page 4
2
Agenda
• Simplifying Assumptions• Establish the Problem• Roadmap to True TDD DB Development• Follow Roadmap
– Class of Databases– Safely Changing Design– Enforcing Interfaces
© Copyright 2011 Hexagon Software LLC.All rights reserved. 39/12/2012
g– Defining Behaviors– Designing for Maintainability
• OO DB Development (Whirlwind Tour)
To fit in a 75 minute box
© Copyright 2011 Hexagon Software LLC.All rights reserved. 49/12/2012
Simplifying Assumptions
Page 5
3
Assumptions
• I assume that you knowI assume that you know...– ...know what TDD is– ...the difference between TDD and Test‐First– ...about mocking– ...a little about design patterns
© Copyright 2011 Hexagon Software LLC.All rights reserved. 59/12/2012
• If not, there are great resources for all• Pick them up, read, & practice before talk
Know what’s in our way
© Copyright 2011 Hexagon Software LLC.All rights reserved. 69/12/2012
Establish the Problem
Page 6
4
Test Runner
[Test]public void AddEmployee() {
var company = Company GetInstance();
TDD in Middle Tier Code
Test Runner
Test1
Test2
Test3
ThisTest
Test5
var company = Company.GetInstance();var employee = Employee.GetInstance();
company.AddEmployee(employee);
Assert.That(company.Employees,Has.Some.SameAs(employee);
}
Setup
Trigger
Assertion
© Copyright 2011 Hexagon Software LLC.All rights reserved. 79/12/2012
Test6
Test7
Test8
}
[Test]public void AddEmployee() {
var company = Company GetInstance();
TDD in Middle Tier Code
var company = Company.GetInstance();var employee = Employee.GetInstance();
company.AddEmployee(employee);
Assert.That(company.Employees,Has.Some.SameAs(employee);
}
Setup
Trigger
Assertion
© Copyright 2011 Hexagon Software LLC.All rights reserved. 89/12/2012
}
Page 7
5
[Test]public void CannotAddLineItemWithoutType() {
DBTestHelper ClearDatabase();
Typical DB Test
DBTestHelper.ClearDatabase();var cartId = DBTestHelper.CreateCart();
try {DBTestHelper.InsertLineItem(
cart: cartId, itemTypeId: DBNull.Value, count: 100);} catch (ConstraintViolationException) {
return;}
Incomplete setup
Trigger
© Copyright 2011 Hexagon Software LLC.All rights reserved. 99/12/2012
}
Assert.Fail();}
Assertion
How Was the DB Created?
• You don’t know• No guarantee DB works like production• Potential for variation
• Leads to...– ...rigorous “controls” over change– ...general “slow down” culture
© Copyright 2011 Hexagon Software LLC.All rights reserved. 109/12/2012
...general slow down culture– ...ultimately: bad database designs
• DB test suites must cover creation
Page 8
6
How to solve the problem
© Copyright 2011 Hexagon Software LLC.All rights reserved. 119/12/2012
Roadmap
Roadmap to TDD
• Creating a Class of Databases– So creation is covered by tests
• Testing Creation & Modification– So changes can be introduced w/confidence
• Enforcing Interfaces– So consumers are forced to recognize changes
• Defining “Behavior”
© Copyright 2011 Hexagon Software LLC.All rights reserved. 129/12/2012
g– So your tests cover the right things
• Designing for Maintainability– So you don’t get bogged down by needless junk
Page 9
7
There’s More!
• OODC ti t f l i l DB / d li & fl ibl d i– Creating systems of logical DBs w/good coupling & flexible designs
• Mocking– Real, useful mocking along interface boundaries
• Legacy Databases– Pulling legacy DBs into new process
• Non‐DB applications– Serialized objects, XML, file structures
© Copyright 2011 Hexagon Software LLC.All rights reserved. 139/12/2012
• Won’t fit in 75 minutes• Shameless plug
– They’re in the book
You don’t want to run tests against production, do you?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 149/12/2012
Creating a Class of Databases
Page 10
8
What is a Class of Databases?
• StrictlySt ct y– Provisions new DBs– Upgrades existing DBs– All DBs get same behavior
• Effecti el
© Copyright 2011 Hexagon Software LLC.All rights reserved. 159/12/2012
• Effectively– Uniform create/upgrade path– All DBs “grew” same way
Problem: Different Build Paths
• Test databases need to be “cheap”est databases eed to be c eap– Easy to create & destroy
• Production databases need to last– Created w/scripts– Improved incrementally w/scripts
• Want all databases to ork the same
© Copyright 2011 Hexagon Software LLC.All rights reserved. 169/12/2012
• Want all databases to work the same– Test & production
• Exactly the same
Page 11
9
How to Make Test DBs?
• Test DBs are of low valueest s a e o o a ue• Especially when cheap to make• Break one?
– Pitch it• Got one dirty?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 179/12/2012
– Pitch it
• Make them however you like
How to Make Production DBs?
• Starts emptyStarts empty• Created w/series of instructions• Upgraded w/series of instructions to v2• Upgraded w/series of instructions to v3• Etc
© Copyright 2011 Hexagon Software LLC.All rights reserved. 189/12/2012
• Etc.• No wiggle room
– What actually happens is what actually happens
Page 12
10
Takeaway #1
© Copyright 2011 Hexagon Software LLC.All rights reserved. 199/12/2012
The Sequence of Transitions
• v0 v1– Get a new v1 DB
• v1 v2– Upgrade from v1 to v2
• v0 v1 v2 v3 v4
© Copyright 2011 Hexagon Software LLC.All rights reserved. 209/12/2012
– Get a new v4 DB• V2 v3 v4
– Upgrade from v2 to v4
Page 13
11
Test Runner
[Test]public void HasAnAccountsTable() {
// you can do this once per suite if you want
Example: Writing a Test
Test Runner
Test1
Test2
Test3
ThisTest
Test5
// you can do this once per suite if you wantMyDatabase.Create(connection, 2);
connection.ExecuteSql("SELECT * FROM Accounts");
// exception thrown if no Accounts table}
DB created by test
© Copyright 2011 Hexagon Software LLC.All rights reserved. 219/12/2012
Test6
Test7
Test8
<database><version number="1">
<script>
Example: Starting DB Script
<script><!-- Build the database -->
</script></version>
</database>
© Copyright 2011 Hexagon Software LLC.All rights reserved. 229/12/2012
Page 14
12
Test Runner
<database><version number="1">
<script>
Example: Updating DB Design
Test Runner
Test1
Test2
Test3
ThisTest
Test5
<script><!-- Build the database -->
</script></version><version number="2">
<script><![CDATA[CREATE Accounts(INT ID, NAME CHAR(23));]]>
© Copyright 2011 Hexagon Software LLC.All rights reserved. 239/12/2012
Test6
Test7
Test8
]]></version>
</database>
Reducing the risk of change rather than its frequency
© Copyright 2011 Hexagon Software LLC.All rights reserved. 249/12/2012
Safely Changing Design
Page 15
13
“Neat”
• Finding uniform build path a beginningd g u o bu d pat a beg g• Must drive error/variation from build
• Builders/installers/etc. are code• What drives error/variation from code?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 259/12/2012
• TESTS!!
Takeaway #2
© Copyright 2011 Hexagon Software LLC.All rights reserved. 269/12/2012
Page 16
14
A Transition Test
• SetupSetup– Bring database up to version X– Populate
• Trigger– Transition to subsequent version
© Copyright 2011 Hexagon Software LLC.All rights reserved. 279/12/2012
• Assertion– Prove that content was preserved
Test Runner
[Test]public void ContactsFactoredFromAccountsInV9() {MyDatabase.Create(connection, 8);
Example: A Transition Test
Test Runner
Test1
Test2
Test3
ThisTest
Test5
var accountId = connection.ExecuteScalar(@"INSERT INTO Accounts(Name, PWHash, Email)VALUES('MaxGuernseyIII', 'pwhash', '[email protected] ');
SELECT SCOPE_IDENTITY();");
MyDatabase.Create(connection, 9);
var emails = connection.ExecuteSelectList(@"SELECT Email FROM Contacts WHERE AccountID = ?",
© Copyright 2011 Hexagon Software LLC.All rights reserved. 289/12/2012
Test6
Test7
Test8
accountId);Assert.That(emails,Is.EqualTo(new[] { "[email protected] " });
}
Page 17
15
Test Runner
<database><version number="9">
<script>
Example: Creating Upgrade Script
Test Runner
Test1
Test2
Test3
ThisTest
Test5
p<![CDATA[CREATE TABLE Contacts(
AccountId INT FOREIGN KEY REFERENCES Account(ID),Email NVARCHAR(2048));
INSERT INTO Contacts(AccountID, Email)SELECT ID, Email FROM Accounts;
ALTER TABLE A t DROP COLUMN E il
© Copyright 2011 Hexagon Software LLC.All rights reserved. 299/12/2012
Test6
Test7
Test8
ALTER TABLE Accounts DROP COLUMN Email;]]>
</script></version>
</database>
Getting the best feedback you can
© Copyright 2011 Hexagon Software LLC.All rights reserved. 309/12/2012
Enforcing Interfaces
Page 18
16
[Test]public void HasAnAccountsTable() {
// you can do this once per suite if you want
Remember This?
// you can do this once per suite if you wantMyDatabase.Create(connection, 2);
connection.ExecuteSql("SELECT * FROM Accounts");
// exception thrown if no Accounts table}
© Copyright 2011 Hexagon Software LLC.All rights reserved. 319/12/2012
Duplication Is Evil
• Two tests is annoyingTwo tests is annoying• Ten is a pain• Fifty is untenable • Any realistic number impossible to maintain w/duplication
© Copyright 2011 Hexagon Software LLC.All rights reserved. 329/12/2012
w/duplication
Page 19
17
[Test]public void HasAnAccountsTable() {
// you can do this once per suite if you want
Easy Solution
y p yMyDatabase.Create(connection, MyDatabase.CurrentVersion);
connection.ExecuteSql("SELECT * FROM Accounts");
// exception thrown if no Accounts table}
...
© Copyright 2011 Hexagon Software LLC.All rights reserved. 339/12/2012
public class MyDatabase {public const int CurrentVersion = 2;...
}
It’s More Pervasive
• Version numbers just most obviousVersion numbers just most obvious• Symbols are the real evil...
© Copyright 2011 Hexagon Software LLC.All rights reserved. 349/12/2012
Page 20
18
[Test]public void CanInsertData() {connection.Execute(@"INSERT INTO Data(Time, Reading)VALUES('4:31:01 103 17 02 2009' 041 908);”);
Duplication Abounds
VALUES( 4:31:01.103 17.02.2009 , 041.908); );}...<script>CREATE TABLE Data(Time DATETIME, Reading NUMERIC(6,3));
</script>...private void EnterReading(double reading) {connection.Execute(@"INSERT INTO Data(Time, Reading) VALUES(?,?)",DateTime.Now, reading);
© Copyright 2011 Hexagon Software LLC.All rights reserved. 359/12/2012
}...private IEnumerable<double> GetOrderedReadings() {return connection.Execute<double>(@"SELECT Reading FROM DataORDER BY Time ASC");
}
One Solution
• Minimize w/string constantsMinimize w/string constants• Couple to constants from tests & code
• What about interface changes?– Transition tests need old design
© Copyright 2011 Hexagon Software LLC.All rights reserved. 369/12/2012
Transition tests need old design– Unit tests & production code always need latest
Page 21
19
Another Solution
• Document design w/each transitionDocument design w/each transition• Couple transition tests to exact• Couple unit tests to latest• Couple production code to latest
© Copyright 2011 Hexagon Software LLC.All rights reserved. 379/12/2012
• What about DB script?
A Really Good Solution
• Document design changes w/versionocu e t des g c a ges / e s o• Bind to design in scripts• Pre‐compile/package step:
– Inject bound symbols– Generate coupling codeC l i l d i
© Copyright 2011 Hexagon Software LLC.All rights reserved. 389/12/2012
• Couple unit tests to latest design• Couple production code to latest design• Couple transition tests to specific design
Page 22
20
<database><version number="2"><design>
Example Script
Generate code from this before compilation/packaging
<add id="NewTable"><add id="Col1" /><add id="Col2" />
</add></design><script><scope id="NewTable">
CREATE TABLE <id/>(<bind>Col1</bind> INT,
"Take the version 1 design and add thisstuff"
Replace w/ "NewTable" prior to packaging
© Copyright 2011 Hexagon Software LLC.All rights reserved. 399/12/2012
<bind>Col2</bind> CHAR(1));</scope>
</script></version>
</database>
Replace w/ "Col2" prior to packaging
Takeaway #3
© Copyright 2011 Hexagon Software LLC.All rights reserved. 409/12/2012
Page 23
21
Replacing Symbol w/Itself?
• 1st‐ly: You can do morey ou ca do o e– E.g.: Use different DB and logical names
• 2nd‐ly: Gives advantages of compiler– across languages (SQL, C#, C++, etc.)
' b li il h d ?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 419/12/2012
• Don't believe compiler has advantages?– You're crazy– Keep it to yourself
If databases are objects, what are their behaviors?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 429/12/2012
Defining Behaviors
Page 24
22
What Do Tests Test?
• Tests specify behaviorsTests specify behaviors• Objects contain behaviors• Databases are objects• DB tests should specify DB behaviors
© Copyright 2011 Hexagon Software LLC.All rights reserved. 439/12/2012
Concepts
• Knowledge – persistent data of valueKnowledge persistent data of value• Information – useful part of signal• Data – superclass of knowledge & information• Behavior – Activity that produces a result
– Answer to a question
© Copyright 2011 Hexagon Software LLC.All rights reserved. 449/12/2012
Answer to a question– Useful side effect
Page 25
23
What Are Behaviors for Transitions?
• Transition behaviors simpleTransition behaviors simple• Change from old structure to new• Preserve knowledge
• Ignored for this segment
© Copyright 2011 Hexagon Software LLC.All rights reserved. 459/12/2012
• Ignored for this segment
What Are DB Behaviors?
• DB behaviors slightly more subtleDB behaviors slightly more subtle• Absorb information & translate to knowledge• Translate knowledge into requested info.
• A table is no behavior
© Copyright 2011 Hexagon Software LLC.All rights reserved. 469/12/2012
• A table is no behavior• A trigger is not behavior• Those are structure
Page 26
24
Knowledge, Info, & Behavior
Client
DB
KnowledgeInfo.
© Copyright 2011 Hexagon Software LLC.All rights reserved. 479/12/2012
TestBehavior
Takeaway #4
© Copyright 2011 Hexagon Software LLC.All rights reserved. 489/12/2012
Page 27
25
Don't and Do
Don't Do• Test a table• Test a trigger• Expose structure
– E.g.: tables etc.
• Test store & retrieval of info.• Test info causes side effect• Encapsulate structure
– w/Views & stored procs
© Copyright 2011 Hexagon Software LLC.All rights reserved. 499/12/2012
Why Encapsulate Design?
• Weird question: Why not?Weird question: Why not?
• Hidden things cheap to change• Exposed ones expensive• Minimizing public interface maximizes
© Copyright 2011 Hexagon Software LLC.All rights reserved. 509/12/2012
• Minimizing public interface maximizes flexibility
Page 28
26
Doing less now enables doing more later
© Copyright 2011 Hexagon Software LLC.All rights reserved. 519/12/2012
Designing for Maintainability
Don't Plan the Future
• Future is unpredictableFuture is unpredictable• You cannot plan it• Maybe your guess is right• Maybe we all die from asteroid strike• Usually: your guess "would have" been right
© Copyright 2011 Hexagon Software LLC.All rights reserved. 529/12/2012
• Usually: your guess would have been right– If only universe had cooperated
Page 29
27
Plan for the Future
• Plan for the future to happenPlan for the future to happen– You'll need to change stuff
• Design so that change is easiest• Largely: what's been shown so far• Additionally: don't overbuild
© Copyright 2011 Hexagon Software LLC.All rights reserved. 539/12/2012
Additionally: don t overbuild
• "What does that mean?"
Takeaway #5
© Copyright 2011 Hexagon Software LLC.All rights reserved. 549/12/2012
Page 30
28
Planning for the Future
• Encapsulate design– So that structures can change
• Enforce interface strongly– So changes don't break downstream functionality
• Cover every behavior w/test– So changes don't introduce regression
• Cover every transition w/test
© Copyright 2011 Hexagon Software LLC.All rights reserved. 559/12/2012
y /– So changes don't jeopardize knowledge
• Minimize features to those needed now– So needed features not impeded by useless ones
Time to watch this work
© Copyright 2011 Hexagon Software LLC.All rights reserved. 569/12/2012
Live Example
Page 31
29
A whirlwind tour
© Copyright 2011 Hexagon Software LLC.All rights reserved. 579/12/2012
The Rest
OOD
• Once classes of DB used, OOD/OOP availableO ce c asses o used, OO /OO a a ab e• Use it• Divide & conquer
– Create composite DBs– Couple via public interface
b i & i i
© Copyright 2011 Hexagon Software LLC.All rights reserved. 589/12/2012
• Even create abstractions & variations– Exploit division to introduce variation– Most important variation on next slide
Page 32
30
Mocking
• Once you have OOD/OOP easy good mockingOnce you have OOD/OOP, easy, good mocking available
• Don't mock tables• Mock behaviors encapsulated in dependency DBs
© Copyright 2011 Hexagon Software LLC.All rights reserved. 599/12/2012
Legacy DBs
• Mocking especially handy w/legacy DBsoc g espec a y a dy / egacy s• Create Façade DB, connects to legacy• Easy to test‐drive Façade behaviors:
– Just mock out legacy• Transfer behavior to Façade over time
© Copyright 2011 Hexagon Software LLC.All rights reserved. 609/12/2012
• Alternatively:– "wrangle" legacy DB into tested state
Page 33
31
Non‐DB Applications
• Much of this works for non‐DB– (not recommendation for views & sprocs, obviously)
• Have data? Want to evolve structure over time?– This process works
• E.g.:– File systems/directory structures
© Copyright 2011 Hexagon Software LLC.All rights reserved. 619/12/2012
File systems/directory structures– Registry keys– XML documents– Serialized objects
Thanks for coming. Any other questions?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 629/12/2012
Q&A