Behavior-driven testing with RSpec

A comprehensive approach to test-driven development

Testing fever has infected the Ruby programming community, and the infection is spreading. One of the most promising innovations in testing in the past year is the introduction and rapid growth of RSpec, a behavior-driven testing tool. Learn how RSpec can change the way you think about testing.

Share:

Bruce Tate (bruce@rapidred.com), CTO, WellGood LLC

Bruce TateBruce Tate is a father, mountain biker, and kayaker in Austin, Texas. The CTO of WellGood, LLC and the chief architect behind ChangingThePresent.org, he's also the author of nine books, including Beyond Java, From Java to Ruby, and Ruby on Rails: Up and Running. He spent 13 years at IBM and later formed the RapidRed consultancy, where he specialized in lightweight development strategies and architectures based on Ruby, and in the Ruby on Rails framework. He now works with a team of Rails developers to build and maintain the charity portal, ChangingThePresent.org.



28 August 2007

Also available in Chinese

In the past ten years, software developers have begun to embrace testing at increasingly lower levels. The simultaneous explosion of dynamic languages, which do not offer a compiler to catch the most basic errors, makes testing even more important. As the testing community grows, developers are beginning to notice benefits above and beyond the most basic benefits of catching bugs:

  • Testing improves your designs. Each target object that you test must have at least two clients: your production code, and your test case. These clients force you to decouple your code. Testing also encourages and rewards smaller, simpler methods.
  • Testing reduces unnecessary code. When you write your test cases first, you get in the habit of writing only enough code to make the test case pass. You reduce the temptation to code features because you might need them later.
  • Test-first development is motivating. Each test case that you write establishes a small problem. Solving that problem with code is rewarding and motivating. When I do test-driven development, the hours fly by.
  • Testing allows more freedom. If you have test cases that will catch likely errors, you'll find that you're more willing to make improvements to your code.

Test-driven development and RSpec

Rather than preach on about the virtues of testing, I'm going to walk you through a simple example of test-driven development (TDD) using RSpec. The RSpec tool is a Ruby package that lets you build a specification alongside your software. This specification is actually a test that describes the behavior of your system. Here's the flow for development with RSpec:

  • You write a test. This test describes the behavior of a small element of your system.
  • You run the test. The test fails because you have not yet built the code for that part of your system. This important step tests your test case, verifying that your test case fails when it should.
  • You write enough code to make the test pass.
  • You run the tests and verify that they pass.

In essence, an RSpec developer turns test cases from red (failing) to green (passing) all day. It's a motivating process. In this article, I walk you through working with the basics in RSpec.

To get started, I'll assume you've installed Ruby and gems. You'll also need to install RSpec. Type:

gem install rspec


Starting an example

Next, I build a state machine, step by step. I follow the rules of TDD. I write my test cases first, and I won't write any code until my test cases make me do so. The author of Rake, Jim Weirich, says it helps to role play. When you're writing the actual production code, you want to play the part of a jerk developer who will only do the minimum amount of work possible to make the test pass. When you're writing tests, you're playing the part of the poor tester who is trying to make the developer do something useful.

The following example shows how to build a state machine. If you've not seen a state machine before, check resources. A state machine has several states. Each of these states supports events that will move the state machine from one state to another. The key to starting with test-driven development is to start at the top, and make as few assumptions as possible. Let the tests drive your design.

Create a file called machine_spec.rb with the contents of Listing 1. This file is your specification. You don't know what the file called machine.rb will do. Just create the empty file for now.

Listing 1. The initial machine_spec.rb
  require 'machine'

Next, you want to run the test. You always run the test by typing spec machine_spec.rb. As expected, Listing 2 shows the damage:

Listing 2. Running the empty spec
~/rspec batate$ spec machine_spec.rb 
/opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require':
 no such file to load -- machine (LoadError)
        from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
        from ./state_machine_spec.rb:1
        from ...

In test-driven development, you want to move in small increments, so you should fix this problem before moving on to the next. Now, I'll play the part of the jerk role player who will do just enough to make the application work. I'll create an empty file called machine.rb, making the tests pass. I can now sit back and laugh. The tests pass, and I've done practically nothing.

The role play goes on. I switch hats back to the exasperated tester, and make the jerk actually do something. I'll code the following spec, one that will require the existence of the Machine class, as in Listing 3:

Listing 3. Initial spec
require 'machine'

describe Machine do
  before :each do
    @machine = Machine
  end
end

This spec describes a nonexistent class called Machine. The RSpec descriptions go in a describe method, and you pass in the name of the class under test and a code block that has the actual spec. Normally, test cases have a certain amount of set up work. In RSpec, you put the set up work in a before section. You pass the before method an optional symbol and a code block. The code block contains the set up work. The symbol determines how often RSpec should execute the code block. The default symbol is :each, meaning RSpec will call the set up block before each test. You can also specify :all, meaning RSpec will call the before block once before all of the tests. You should almost always use :each, to keep each test isolated and independent of the others.

Run the test by typing spec, as in Listing 4:

Listing 4. Flunking the existence test
~/rspec batate$ spec machine_spec.rb 


./machine_spec.rb:3: uninitialized constant Machine (NameError)

Now the exasperated tester has forced the jerk's hand—he'll have to wake up and create the class. The jerk in me responds. It's time to fix the error. In machine.rb, I type the bare minimum, as in Listing 5:

Listing 5. Creating the initial Machine class
class Machine
end

I save the file, and run the test. Sure enough, Listing 6 shows the test reports no errors:

Listing 6. Testing the existence of Machine
~/rspec batate$ spec machine_spec.rb 


Finished in 5.0e-06 seconds

0 examples, 0 failures

Establishing behavior

Now, I can start to force some more behavior. I know that all state machines must start in some initial state. I don't know how I'm going to design this yet, so I'll write a very simple test, saying initially, the state method should return the symbol :initial. I modify machine_spec.rb and run the test, as in Listing 7:

Listing 7. Require an initial state and run the test
require 'machine'

describe Machine do
  before :each do
    @machine = Machine.new
  end
  
  it "should initially have a state of :initial" do
    @machine.state.should == :initial
  end
  
end


~/rspec batate$ spec machine_spec.rb 


F

1)
NoMethodError in 'Machine should initially have a state of :initial'
undefined method `state' for #<Machine:0x10c7f8c>
./machine_spec.rb:9:

Finished in 0.005577 seconds

1 example, 1 failure

Notice the form of a rule: it "should initially have a state of :initial" do @machine.state.should == :initial end. The first thing you'll notice is that the rule reads like an English sentence. Remove the punctuation, and you get, it should initially have a state of initial. The next thing you notice is that the rule does not look like typical object-oriented code. It isn't. You have a single method, called it. The method takes a string parameter, enclosed in quotes, and a code block. The string should describe the requirement you're working to test. Finally, the code block between do and end includes the code for the test case.

You can see that I'm working in very small increments. These small steps have a variety of benefits. They let me improve the density of tests, and give me time to think about the expected behavior, and the accompanying API. The small steps also let me keep up with my code coverage as I develop, and build a richer specification.

This style of testing has dual purposes: testing your implementation, and building requirements design documentation as you test. You will later see me create a list of requirements from the test cases.

I fix the test in the simplest way possible, returning :initial, as in Listing 8:

Listing 8. Assigning an initial state
class Machine
  
  def state
    :initial
  end
end

As you read the implementation, you probably either laughed out loud or felt a little cheated. For test-driven development, you have to twist your thinking a little bit. Your goal is not to write your eventual production code, at least, not right away. Your goal is to make your test cases pass. When you learn to work this way, you may find that new implementations occur to you, and you will write far less code than you did before adopting TDD.

The next step is to run the code, and watch it pass:

Listing 9. Run the initial state test.
~/rspec batate$ spec machine_spec.rb 


.

Finished in 0.005364 seconds

1 example, 0 failures

Take a moment to reflect on this past iteration. If you look at the code, you might be disheartened. You haven't made much progress. If you look at the whole iteration, you should see more: You captured an important requirement and wrote a test case to enforce that requirement. As a programmer, my first behavioral test lets me hit my stride. The implementation details come into clearer focus.

Now, I can force a more robust implementation of a state. Specifically, I want to handle more than one state within the machine. I'll create a new rule to ask for a list of valid states. As always, I'll run the test and watch it fail.

Listing 10. Allow specification of valid states.
 it "should remember a list of valid states" do
    @machine.states = [:shopping, :checking_out]
    @machine.states.should == [:shopping, :checking_out]
  end
  

run test(note: failing first verifies test)

~/rspec batate$ spec machine_spec.rb 


.F

1)
NoMethodError in 'Machine should remember a list of valid states'
undefined method `states=' for #<Machine:0x10c7154>
./machine_spec.rb:13:

Finished in 0.005923 seconds

2 examples, 1 failure

In Listing 10, you see the RSpec form of an assertion. The assertion starts with the method should, and then adds some kind of comparison. The should method makes some kind of observation about the application. A working application should behave in a certain way. The should method captures this requirement nicely. In this case, my state machine should remember two different states.

This time, you need to add an instance variable to actually remember the state. As always, I'll run the test case after making the coding change and watch the test case succeed.

Listing 11. Create an attribute to remember states.
class Machine
  attr_accessor :states

  def state
    :initial
  end
end




~/rspec batate$ spec machine_spec.rb 


..

Finished in 0.00606 seconds

2 examples, 0 failures

Driving refactoring

At this point, I do not like my decision to force the first state in the state machine to be called :initial. Instead, I'd rather require first state to be the first element in the state array. My understanding of the state machine is evolving. This kind of revelation is not uncommon. Test-driven development will often force me to revisit early assumptions. Since I've captured early requirements in the form of test cases, I can easily refactor my code. In this context, think of refactoring as turning code that works into better code that works.

Change the first test to look like Listing 12, and run the test:

Listing 12. The initial state should be the first state specified
it "should initially have a state of the first state" do
  @machine.states = [:shopping, :checking_out]
  @machine.state.should == :shopping
end



~/rspec batate$ spec machine_spec.rb 


F.

1)
'Machine should initially have a state of the first state' FAILED
expected :shopping, got :initial (using ==)
./machine_spec.rb:10:

Finished in 0.005846 seconds

2 examples, 1 failure

I can tell that the test case works because it fails, so I'm free to make the code work. My task is clear. I need to make the test case pass. I like where things are going, because my test case is driving my design. I'm going to pass the initial states in to the new method. I'll change the implementation slightly to conform to my revised specifications, as in Listing 13.

Listing 13. Assign the initial state.
start to fix it
class Machine
  attr_accessor :states
  attr_reader :state

  def initialize(states)
    @states = states
    @state = @states[0]
  end
end





~/rspec batate$ spec machine_spec.rb 


1)
ArgumentError in 'Machine should initially have a state of the first state'
wrong number of arguments (0 for 1)
./machine_spec.rb:5:in `initialize'
./machine_spec.rb:5:in `new'
./machine_spec.rb:5:

2)
ArgumentError in 'Machine should remember a list of valid states'
wrong number of arguments (0 for 1)
./machine_spec.rb:5:in `initialize'
./machine_spec.rb:5:in `new'
./machine_spec.rb:5:

Finished in 0.006391 seconds

2 examples, 2 failures

Now, that was unexpected. I caught some bugs in my implementation. The test cases don't use the right interface any more, because I did not pass the initial states into the state machine. I can see that the test cases are already protecting me. I made a sweeping change, and the test cases caught the bug. We need to refactor the tests to match the new interface, passing an initial list of states into new. Rather than repeat this initialization code, I'll just set it up in the before method, as in listing 14:

Listing 14. Initialize the state machine in "before."
require 'machine'

describe Machine do
  before :each do
    @machine = Machine.new([:shopping, :checking_out])
  end
  
  it "should initially have a state of the first state" do
    @machine.state.should == :shopping
  end
  
  it "should remember a list of valid states" do
    @machine.states.should == [:shopping, :checking_out]
  end
  
end




~/rspec batate$ spec machine_spec.rb 


..

Finished in 0.005542 seconds

2 examples, 0 failures

The state machine is starting to take shape. The code still has some problems, but is starting to evolve nicely. I'll start to work some transitions into the state machine. Those transitions will force the code to actually remember the current state.

My test case forces me to think through the design of the API. I need to understand how I will represent events and transitions. Rather than starting with a full-blown object-oriented implementation, I'm going to represent transitions in a hash table. Later, requirements may make me change my assumption, but for now, I will stay simple. Listing 15 shows the revised code for the tests:

Listing 15. Adding events and transitions
remember events... change before conditions


require 'machine'

describe Machine do
  before :each do
    @machine = Machine.new([:shopping, :checking_out])
    @machine.events = {:checkout => 
                               {:from => :shopping, :to => :checking_out}}
  end
  
  it "should initially have a state of the first state" do
    @machine.state.should == :shopping
  end
  
  it "should remember a list of valid states" do
    @machine.states.should == [:shopping, :checking_out]
  end
  
  it "should remember a list of events with transitions" do
    @machine.events.should == {:checkout => 
                               {:from => :shopping, :to => :checking_out}}
  end
  
  
end




~/rspec batate$ spec machine_spec.rb 


FFF

1)
NoMethodError in 'Machine should initially have a state of the first state'
undefined method `events=' for #<Machine:0x10c6f38>
./machine_spec.rb:6:

2)
NoMethodError in 'Machine should remember a list of valid states'
undefined method `events=' for #z7lt;Machine:0x10c5afc>
./machine_spec.rb:6:

3)
NoMethodError in 'Machine should remember a list of events with transitions'
undefined method `events=' for #<Machine:0x10c4a58>
./machine_spec.rb:6:

Finished in 0.006597 seconds

3 examples, 3 failures

Since the new test code was in before, the new code broke all three of my tests. Listing 16 shows the tests are easy to fix, though. I'll just add another accessor:

Listing 16. Remembering events
class Machine
  attr_accessor :states, :events
  attr_reader :state

  def initialize(states)
    @states = states
    @state = @states[0]
  end
end



~/rspec batate$ spec machine_spec.rb 


...

Finished in 0.00652 seconds

3 examples, 0 failures

test

The tests all pass. I'm closing in on a working state machine. The next few tests bring it all together.


Getting real

So far, I've done nothing to actually trigger a state transition, but all of the ground work is done. I've accumulated a set of requirements. I've built a set of tests. I have working code that sets up much of the data that my state machine will use. At this point, managing a single state machine transition represents a single tiny transition, so I'll simply add the test in Listing 17:

Listing 17. Building a state transition into the state machine
it "should transition to :checking_out upon #trigger(:checkout) event " do
  @machine.trigger(:checkout)
  @machine.state.should == :checking_out
end




~/rspec batate$ spec machine_spec.rb 


...F

1)
NoMethodError in 'Machine should transition to :checking_out upon
#trigger(:checkout) event '
undefined method `trigger' for #<Machine:0x10c4d00>
./machine_spec.rb:24:

Finished in 0.006153 seconds

4 examples, 1 failure

I need to resist the temptation to do too much too soon. I should write only enough code to make the test case pass. The iteration in Listing 18 will let me express the API and a requirement. That's enough for now:

Listing 18. Defining the trigger method
def trigger(event)
  @state = :checking_out
end



~/rspec batate$ spec machine_spec.rb 


....

Finished in 0.005959 seconds

4 examples, 0 failures

Here's an interesting side note. When I wrote this code, I messed this trivial method up twice. The first time I returned :checkout; the second time I set state to :checkout instead of :checking_out. The tiny steps saved me time, because the test cases caught the errors that might be much harder to catch later. The final step in this article is to actually do a state machine transition. In the first example, I don't care what state the machine is actually in. I'll simply force a blind transition according to the event, regardless of state.

A two-node state machine will not do the trick. I will need to build in a third node. I will leave the existing before method alone, and just add additional states in the new state. I will do two transitions in the test case, to make sure the state machine is making the transitions appropriately, as shown in Listing 19:

Listing 19. Forcing the first transaction
it "should transition to :success upon #trigger(:accept_card)" do
    @machine.events = {
       :checkout => {:from => :shopping, :to => :checking_out},
       :accept_card => {:from => :checking_out, :to => :success}
    }
  
    @machine.trigger(:checkout)
    @machine.state.should == :checking_out
    @machine.trigger(:accept_card)
    @machine.state.should == :success
  end




~/rspec batate$ spec machine_spec.rb 
....F

1)
'Machine should transition to :success upon #trigger(:accept_card)' FAILED
expected :success, got :checking_out (using ==)
./machine_spec.rb:37:

Finished in 0.007564 seconds

5 examples, 1 failure

This test sets up a new state machine with :checkout and :accept_card events. I might choose to use two events instead of one while processing a checkout to prevent double orders. The checkout code can make sure the state machine is in the shopping state before checking out. The first checkout will first make the transition from shopping to checking_out. The test case makes both transitions by triggering the checkout and accept_card events, and verifying the states correctly after calling the events. As expected, the test case fails—I have not written a trigger method that will handle more than one transition. The fix involves one line of code, a very important one. Listing 20 shows the heart of the state machine:

Listing 20. The heart of the state machine
def trigger(event)
    @state = events[event][:to]
  end


~/rspec batate$ spec machine_spec.rb 
.....

Finished in 0.006511 seconds

5 examples, 0 failures

And the test case runs. For the first time, this crude piece of code has evolved into something that you could actually call a state machine. It is far from complete. Right now, the state machine is too permissive. The state machine will trigger an event regardless of the state of the machine. For example, triggering :accept_card when you're in the shopping state should not take you to the :success state. You should only be able to trigger :accept_card from the :checking_out state. In programming terms, the trigger method should be scoped to event. I'll remedy that problem by writing a test and then fixing the bug. I'll write a negative test, meaning one that asserts a behavior that should not occur, as in Listing 21:

Listing 21: A negative test
it "should not transition from :shopping to :success upon :accept_card" do
    @machine.events = {
       :checkout => {:from => :shopping, :to => :checking_out},
       :accept_card => {:from => :checking_out, :to => :success}
    }
  
    @machine.trigger(:accept_card)
    @machine.state.should_not == :success
  end
  
rspec batate$ spec machine_spec.rb 
.....F

1)
'Machine should not transition from :shopping to :success upon :accept_card' FAILED
expected not == :success, got :success
./machine_spec.rb:47:

Finished in 0.006582 seconds

6 examples, 1 failure

I can now run the tests again, and one of the tests breaks as expected. The fix, once again, is a one-liner, shown in Listing 22:

Listing 22. Fixing the scoping problem in trigger
def trigger(event)
    @state = events[event][:to] if state == events[event][:from]
  end


rspec batate$ spec machine_spec.rb 
......

Finished in 0.006873 seconds

6 examples, 0 failures

Putting things together

At this point, I have a simple working state machine. It's not perfect by any stretch of the imagination. You can probably see a few of the problems:

  • The hash of states doesn't really do anything. I should either validate the events and their transitions against the states, or scratch the array of states altogether. Future requirements will likely dictate the outcome.
  • A given event can only exist on one state. This is probably not an acceptable restriction. For example, submit and cancel states might need to be on many states.
  • The code is not particularly object oriented. To keep configuration easy, I've put most of the data in a hash. Future iterations could well drive the design toward a more object-oriented design.

But you can also see that this state machine already satisfies a bunch of requirements. I also have documentation, describing the behavior of the system, and a good start at a test suite. Each test case backs up a fundamental requirement in the system. In fact, you can see a basic report consisting of the specifications of the system by running spec machine_spec.rb --format specdoc, as in Listing 23:

Listing 23. Seeing the specification
spec machine_spec.rb --format specdoc

Machine
- should initially have a state of the first state
- should remember a list of valid states
- should remember a list of events with transitions
- should transition to :checking_out upon #trigger(:checkout) event 
- should transition to :success upon #trigger(:accept_card)
- should not transition from :shopping to :success upon :accept_card

Finished in 0.006868 seconds

The test-driven approach is not right for everyone, but a growing number of people use the technique to build quality code that's flexible, adaptable, and built from the ground up with testing in mind. You can definitely get many of the same benefits from other frameworks such as test_unit. RSpec also provides an excellent way to get you there. The new testing framework shines in the expressiveness of the code that you build with it. Beginners especially can benefit from the behavior-driven approach to testing. Try it out and let me know what you think.

Resources

Learn

  • RSpec is the behavior-driven framework that enhances test-driven development in Ruby.
  • Martin Fowler's interview about Test Driven Development lends insight into why the techniques are so powerful and productive.
  • The Ruby home page provides outstanding resources to get you started with the Ruby programming language.
  • From Java to Ruby is the author's book. Bruce Tate builds a manager's guide describing why Ruby makes sense for some business problems.
  • The finite state machine is a software concept that divides problems into finite states with transitions between them, often triggered by events. The state machine forms the basis of this article.
  • See all of developerWorks' featured trial downloads.
  • Subscribe to the developerWorks Web development newsletter.
  • Get your hands on more howto articles from the Web development zone's technical library.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Web development on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development
ArticleID=252160
ArticleTitle=Behavior-driven testing with RSpec
publish-date=08282007