How BDD style Unit Testing helps you write better test code (and some caveat) @ihower at RubyKaigi 2011 Ruby Taiwan & OptimisDev
Sep 12, 2014
How BDD style Unit Testing helps you write better
test code (and some caveat)
@ihower at RubyKaigi 2011 Ruby Taiwan & OptimisDev
About Me
• Chang, Wen-Tien 張文鈿
• a.k.a. ihower
• http://ihower.tw
• http://twitter.com/ihower
I’m ihower and here is my blog and twitterI have program ruby from 2006 for ruby on railsAnd I’m Yet another rubyist since rails.
From Taiwan
3.5hrs
I come from Taiwan which is far away about 4hours flightI’m sorry that I can’t speak Japanese, and English is not my native languageIf I speak Chinese, I guess you will escape now. So please forgive my english. I hope it’s simple to understand.
I work at Optimis International, which is an america company in Taipei
Ruby Taiwanhttp://ruby.tw
Beside daily job, I’m the founder and organizer of Ruby Taiwan community, I found the community since 2008.
RubyConf Taiwan 2011/8/26-27http://rubyconf.tw
We are organizing rubyconf taiwan now. The date is augest 26 and 27 and it open registration now.Welcome to Taiwan.
Last year photo. Nice small meeting room. pretty sweet.
Agenda
• Why Unit Testing?
• How to do unit test in BDD style?
• Some BDD caveat
Today’s agenda. There will be three part:First, I will talk about why, then how to do, finally talk about some caveat.
How many people do unit test?
Before get into BDD, how many people here do unit test all the way?Could you raise your hand?
How many people use BDD framework
(RSpec)?
And how many people use rspec?
VerificationAre we writing the code right?Unit Test
Customer Test(Acceptance Test)
ValidationAre we writing the right software?
Two kinds of test:
There are roughly two kinds of test:Unit test and Customer test(or called Acceptance test)Unit test is about verification, the question is are we writing the code right?Customer test is about validation, the question is are we writing the right software?
Test individual class and methodUnit Test
Customer Test(Acceptance Test)
Test functionality of the over-all system
Two kinds of test:
Unit test test individual class and methodAnd Customer Test test functionality of the over-all system
Today is all about Unit Testing
Today is all about unit testing
1. Why Unit Testing?
Let get started part one
1. VerificationDo we write the code right?
First, it does verification
How to verify this?There are three execution path
def correct_year_month(year, month)
if month > 12 if month % 12 == 0 year += (month - 12) / 12 month = 12 else year += month / 12 month = month % 12 end end return [year, month] end
For example, how to verify this simple if else code.There are three execution path which means if you manually testyou need play it 3 times in different way
via UI?
via irb console?
Manually test is inefficient
Unit testing give us instant coding feedback
it "should return correct year and month" do correct_year_month(2011, 3).should == [2011, 3] correct_year_month(2011, 14).should == [2012,2] correct_year_month(2011, 24).should == [2012,12]end
automatic unit testing can give us instant feedbackyou can test many times you want.
2.Check regressionSafety Net: when you add new feature and refactoring
test also check the regressionit’s a safety net when you add new feature or refactoringIt can give you confidence that you won’t break up any existed behavior
3.API designguide you write better API, especially test first.
third, Since test code is the client for your production code,so if the test code is hard to write, it may means your API is not good.Writing unit testing can guide you write better API, especially when you write test first.
4.DocumentationWhat the result should be?
Communication is the key for software development.
finally, uniting test can be your code documentation.It can answer you what’s the class and method result should be?Communication the key for successful software development.
Brief Summary
• Your code is not trivial: Automatic test can save your verify/debug time
• Your code is not throwing way:Regression test
• Test first: Guide you write better API
• Tests as Documentation
2. How to do unit test in BDD style?
xUnit Framework
• Each test has four-phase:
• Setup
• Exercise
• Verify (assertion)
• TeardownFirst, We use xUnit framework to test code. Each test should be isolation and has four phase:1. Setup Data2. Exercise production code3. Verify the result 4. Cleanup the data
Ruby Test::Unitclass OrderTest < Test::Unit::TestCase
def setup @order = Order.new end def test_order_amount assert_equal 0, @order.amount end end
1
23
The is the classical ruby test::unit codeRed 1 is setup Red 2 is exerciseRed 3 is verify. We assert the result equal to zero
Minitest::Specdescribe Order do
before do @order = Order.new end it "should have default amount is 0" do @order.amount.must_equal 0 end
end
1
2 3
Here is the BDD style using minitest::specred 1 is setupred 2 is exercisered 3 is verify. the result must equal to zero
First impression about BDD
• Syntax is different
• Like specification
• More readable
So, what’s the first impression about bdd style?Ok, the syntax is different, it looks like spec and may more readable Otherwise the structure is not much different
What’s BDD?
• An improved xUnit Framework
• Focus on clearly describe the expected behavior
• The emphasis is Tests as Documentation rather than merely using tests for verification.
Let’s back to see what’s BDD means.It’s not totally new test framework, it just an improved xUnit frameworkIt focus on describe the expected behaviorIt emphasize tests as documentation, not just for verification.
Terminology changedNew paradigm: Executable Specification
• “Test” becomes “Spec”
• “Assertion” becomes “Expectation”
• “test method” becomes “example” (RSpec)
• “test case” becomes “example group” (RSpec)
For correspond to the new paradigm: the test is the executable specificationThe “test” becomes “spec”“Assertion” becomes “Expectation”
Ruby BDD tools
• RSpec
• Minitest/Spec
In Ruby, there are some choices:RSpec is the most powerful, has many features and many awesome syntax.Minitest/spec is the Ruby 1.9 build-in xunit library. It’s simple, fast, and just works!
Learn BDD style
• It’s about syntax
• syntax
• syntax
• syntax
Let’s learn how to write BDD style test code.Basically, it’s just learn about syntax.
BDD style element (1)Context
one test case
a class or string
describe Order do # ...end
# or
describe "A Order" do # ...end
Instead of test case class, BDD style use “describe” block for one test caseIt accept parameter which means the stuff you want to describe, usually a class name or just description text.
Can be nested
#method means instance method
.method means class method
describe Order do
describe "#amount" do # ... end
describe ".size" do # ... end
end
“describe” block can be nested.And inner “describe” usually describe method.Pound means instance methoddot means class method
In RSpec,alias_method :context, :describe
describe Order do
context "when initialized" do describe "#amount" do # ... end
describe ".size" do # ... end
end
end
1
2
3
3
Is RSpec, it also alias describe to context.it used to organize test code into different context.
BDD style element (2)Declarative test
one test method: ityou can write test description in plain text
describe Order do
describe "#amount" do describe "when user is vip" do it "should discount five percent if total > 1000" do # ... end it "should discount ten percent if total > 10000" do # ... end end end
Each test method is “it” blockAnd it accept one parameter which is description text
Setup method: beforedescribe Order do
before do @user = User.new( :is_vip => true ) @order = Order.new( :user => @user ) end end
the setup method is “before” block
describe Order do
before do @user = User.new( :is_vip => true ) @order = Order.new( :user => @user ) end describe "#amount" do before do @order2 = Order.new( :user => @user ) end end end
nested beforebtw, the outer data will be shared, make the test faster
1
2
“before” block also support nestedMore important, the outer setup data will be shard for all inside test methods.Make the test faster since we need not create again every time.
https://github.com/citrusbyte/contest
class SomeTest < Test::Unit::TestCase setup do @value = 1 end
test "sample test" do assert_equal 1, @value end
context "a context" do setup do @value += 1 end
test "more tests" do assert_equal 2, @value end
endendThere are library which make test/unit support context and declarative test ,
like contest
https://github.com/jm/context
class UserTest < Test::Unit::TestCase
context "A new User" do before do @user = User.first end
test "should have the right full_name" do assert_equal "Dude Man", @user.full_name end end
Or context
Rails support declarative test(but no nested context)
class PostsControllerTest < ActionController::TestCase
setup @post = posts(:one) end test "should show post" do get :show, :id => @post.id assert_response :success end
end
Even rails, It supports declarative test syntax. It use “test” block But no nested context.
BDD style element (3)Expectation
Expectation(Assertion)Minitest/Spec
@order.amount.must_equal [email protected]_equal 2000lambda{ order.ship! }.must_raise NotPaidError
BDD style assertion is called expectation.In menitest/spec, the methods are must_equal, wont_equal, must_raiseIn unit/test, some people may confuse assert method that first parameter and second parameter, which one is actual and which one is expect. But BDD style, it’s impossible to confuse it.
Expectation(Assertion)RSpec
@order.amount.should == [email protected]_not == 2000lambda{ order.ship! }.should_raise NotPaidError# orexpect { order.ship! }.to raise_error(NotPaidError)
Here is rspec version. should equal and should_not equal
Matchyhttps://github.com/jm/matchy
x.should == 42y.length.should_not be(4)lambda { raise "FAIL" }.should raise_error
There is also library can make test/unit support bdd style expectation only.
BDD cares aboutthe output
Output (rspec)
Really nice code documentation
RSpec output :)
RSpec does the output very well, you can see really nice code documentation
HTML format
Minitest Output :(
But I can’t not find how to generate code documentation in minitest/spec
require 'minitest/pride'# Show your testing pride!
I found there is a minitest/pride you can required! It may help format the output, so I try it!
Very funny @_@
And here is the output....... very funny. Good job!
Use turn gemgem 'minitest' # use gem version instead of stdlibrequire 'minitest/spec'require 'turn'
MiniTest::Unit.use_natural_language_case_names = trueMiniTest::Unit.autorun
Anyway, There is a gem called trun which used in Rails 3.1It can generate nice test output for unit/testSo here I require it.Btw, I suggest you use the latest minitest gem version, not standard library. Since the gem version is much new. I totally agree tenderlove suggestion yesterday that we should move some standard library into gem.
Minitest::Spec output :)
Here is the output, very nice.
Brief Summary
• Three BDD style elements:
• context
• declarative test
• expectation
• test output can be code documentation
So...
Why Syntax Matters?
http://www.nytimes.com/2010/08/29/magazine/29language-t.html
Your language shape how you think!
Language will not limit you, but it does influence how you think.
For example:
test_datetime_format
# ruby/test/logger/test_logger.rbclass TestLogger < Test::Unit::TestCase
def test_datetime_format dummy = STDERR logger = Logger.new(dummy) log = log_add(logger, INFO, "foo") assert_match(/^\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d.\s*\d+ $/, log.datetime) logger.datetime_format = "%d%b%Y@%H:%M:%S" log = log_add(logger, INFO, "foo") assert_match(/^\d\d\w\w\w\d\d\d\d@\d\d:\d\d:\d\d$/, log.datetime) logger.datetime_format = "" log = log_add(logger, INFO, "foo") assert_match(/^$/, log.datetime) end
end
This is a test code from ruby core for Loggerit test datetime_format method
the same test in rubyspec# rubyspec/logger/logger/datetime_format_spec.rbdescribe "Logger#datetime_format" do # ... it "returns the date format used for the logs" it "returns nil logger is using the default date format"end
describe "Logger#datetime_format=" do # ...
it "sets the date format for the logs" do format = "%Y" @logger.datetime_format = "%Y" @logger.datetime_format.should == "%Y" @logger.add(Logger::WARN, "Test message") @log_file.rewind
regex = /2[0-9]{3}.*Test message/ @log_file.readlines.first.should =~ regex end
it "follows the Time#strftime format" endThis is the same test in rubyspec
you can see it divide into many describe and it block.
In this case:
• Test::Unit version just do verify
• Spec version not only verify but also describe its behavior:
• It help us understand what’s the method should do
• It increases the test coverage
The BDD syntax guides you focus on the
expected behavior
Emphasis on Tests as Documentation
Rather than merely using tests for
verification
3. BDD style caveat
1. Too magic?
Metaprogramming?
• The BDD framework using metaprograming should not be the problem.
• The problem is some (RSpec) advanced awesome syntax may make the test code hard to understand or not intuitive.
Metaprogramming implementation should be the problem. If it is, then many ruby library is bad including rails.
The problem is some advanced awesome syntax may hard to understand.
describe Order do its(:status) { should == "New" } end
implicit subject(RSpec)
For example. RSpec has a feature callsd implict subject.You can omit the receiver of expectation method.It also has a syntax call its, its...... its.... well, hard to explain... let see another equal version
describe Order do before do @order = Order.new end it "should have default status is New" do @order.status.should == "New" end end
Equal to this version:
oh.... I see...its will covert Order instance and call its method
well, seems magic!
describe PostsController do describe "PUT #update" do before do @post = Factory(:post) end it "allows an author to edit a post" do sign_in @post.author put :update, :id => post.id, :body => "new content"
response.should be_successful end
it "does not allow a non-author to edit a post" do sign_in Factory(:user) put :update, :id => post.id, :body => "new content"
response.should be_forbidden end endend
would like DRY?
1
2
3
2
3
Another example to test rails controller:red 1 is setup partred 2 is exercisered 3 is verify
The red 2 exercise are almost the same, so some people may want to DRY.
But readable?describe PostsController do describe "PUT #update" do let(:post) { Factory(:post) }
before do sign_in user put :update, :id => post.id, :body => "new content" end
context "when logged in as post's author" do let(:user) { post.author }
it "allows the post to be updated" do response.should be_success end end
context "when logged in as another user" do let(:user) { Factory(:user) }
it "does not allow the post to be updated" do response.should be_forbidden end end endend
3
3
2
1
So they move the code into before blockand rspec has a feature called “let”, it’s a lazy and memorized method for setup data.
Finally in the test method, Only left assertion: response.should be_success
Four-phase Test form is key to understand the
test code• Setup
• Exercise
• Verify (assertion)
• Teardown
Why? I think for test code:four-phase test form is the key to understand.
When I see one test method, I must need to figure out what’s the setup, exercise and verify.
So, it should be easy to trace.
2.English-like DSL?http://pragdave.blogs.pragprog.com/pragdave/2008/03/
the-language-in.html
Second caveat is english-like DSL.Dave Thomas has a nice article explain this problem.
“The language in a DSL should be the language of the domain, not the natural
language of the developer. “ by DaveThomas
He said...
so... To be english-like is not DSL’s purpose. The easy to use and readable DSL API is the purpose
# Baconit 'should have an object identity' do @ary.should.not.be.same_as Array.newend
the be is just “self”
Let’s see one obvious example. This is a bacon test code. There is a be method inside method chain, but it does nothing and just return self.
It existed only because we want to make it like english. Is it right?
RSpec matcher (1)
target.should be_true # targer.should == truetarget.should be_false # targer.should == falsetarget.should be_nil # targer.should == nil
target.should be_a_kind_of(Array) # target.class.should == Arraytarget.should be_an_instance_of(Array) # target.class.should == Array
On the other hand, RSpec has core feature called matcher for expectation.It also make the test code readable. In some way, It’s also very easy to use.
like * be_true, * be_false * be_a_kind_of
RSpec matcher (2)target.should respond_to(:foo) # target.repond_to?(:foo).should == true
target.should have_key(:foo) # target[:foo].should_not == nil
target.should include(4) # target.include?(4).should == true
target.should have(3).items # target.items.length == 3
There are more example like
* have_key* include
RSpec matcher (3)
target.should be_empty # target.empty?.should == true
target.should be_blank # target.blank?.should == true
target.should be_admin # target.admin?.should == true
any prefix be_ matcher will covert it to that method with question mark.
RSpec matcher is very powerful
# Custom Matcherdescribe 9 do it "should be a multiple of 3" do 9.should be_a_multiple_of(3) # (9 % 3).should == 0 endend
# RSpec supports custom matchers DSLRSpec::Matchers.define :be_a_multiple_of do |expected| match do |actual| actual % expected == 0 endend
RSpec matcher is very powerful since it even provide DSL to define your custom matcher which make the your test readable.
Some matchers I like, but some I dislike.
It becomes personal taste :|
3. Deep nested context?describe Order do before do @order = Order.new end describe "#amount" do before do @order2 = Order.new( :total => 2000) end describe "when user is vip" do before do @user = User.new(:is_vip => true) end it "should discount five percent if total >= 1000" do # ... end end endend
1
1
1
There are three nested context here. For the most inner test method, it has three before block outside.
Deep nested is hatefulIt’s hard to understand what’s setup going on in sub-sub-sub-sub context
Deep nested context is hateful, It hard to trace the setup part when there is sub-sub-sub-sub context
Four-phase Test form is key to understand the
test code• Setup
• Exercise
• Verify (assertion)
• Teardown
Again. Four-phase Test form:
Deep context make setup part hard to trace.
That’s why Rails does not support nested context.
Rails team like flat test structure.
4. Too mockist?
• rspec-rails gem divides tests into model,controller,view,helper and encourages you use mocks.
• should_receive easily make people test implementation rather than result by default. Make your test fragile.
RSpec ship with very nice mock feature,And RSpec-rails divides tests into four layer which encourage you use mocks.
But mocks has many caveats too, It make the test fragile easily.
5.Performance penalty?
• minitest/spec performance is almost equal to test/unit. It’s really fast.
• RSpec performance is slower comparing to test/unit.
minitest/spec performance is pretty well, you need not to worry about it.
RSpec is slower, But when running test, I think the bottleneck is your production code and the testing framework factor is not big problem.
Conclusion
• BDD testing framework is unlike ActiveRecord for SQL, Rails for web development. It does not “simplify” test. It’s not an abstract tool for testing.
• So, yes, It’s not required and some people think it’s unnecessary.
yes, it’s not required...
But it’s very valuable for your team!
• The BDD syntax guides you focus on the expected behavior, and increasing the test coverage.
• Emphasis on Tests as Documentation rather than merely using tests for verification.
It focus on describe the expected behavior and increase the test coverageIt emphasize tests as documentation, not just for verification.
Thanks for listening!Please tweet me if you have questions:
@ihower
Reference
• RSpec 讓你愛上寫測試 slides, ihower OSDC.TW 2011
• xUnit Test Patterns
• http://avdi.org/devblog/2011/04/07/rspec-is-for-the-literate/
• http://objectreload.com/articles/2010/09/thoughts-on-testing-part-1.html