Extending beyond unit testing

In Part 1 of this two-part miniseries, you saw how dynamic languages facilitate unit testing. This article shows how some of the advantages of integrated environments come into play in functional and integration testing. Unit testing involves testing small pieces of code, such as methods, and often seeks to isolate them from surrounding elements. Functional tests and integration tests exercise progressively larger pieces of an application. A functional test tests a single feature, usually involving an interface, the business code that does the task, and the code that interfaces with middleware services, such as databases. Integration tests exercise several different features of an application. (Functional testing is also often loosely referred to as integration testing.)

Java developers have done a very good job of addressing unit testing, but integration testing doesn't generate quite as much excitement. Most Java testing frameworks, such as JUnit or TestNG, focus primarily on unit testing. One reason for the lack of integration-testing frameworks in Java programming is the lack of a centralized architecture or development philosophy. In the sections that follow, I'll continue to walk you through a Ruby on Rails example, focusing this time on functional testing and the new Rails integration testing framework. You'll see how much easier it is to test when you're working with an integrated framework.

Running your tests

If you haven't already done so, read through Part 1 first. Then, if you want to code along with this article, make sure you've got a working Rails application. In Part 1, you implemented a simple unit test and a few fixtures. If you coded along then but can't remember if you left the application in a working condition, take advantage of your test cases by changing to your project directory and running rake. Listing 1 shows my results:

> bruce-tates-computer:~/rails/trails batate$ rake
(in /Users/batate/rails/trails)
/usr/local/ror/bin/ruby -Ilib:test 
   "/usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb" 
   "test/functional/trails_controller_test.rb" 
Loaded suite /usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader
Started
EEEEEEEEEEEEEEEE
Finished in 0.070797 seconds.

  1) Error:
test_create(TrailsControllerTest):
Errno::ENOENT: No such file or directory - /tmp/mysql.sock
    /usr/local/ror/lib/ruby/gems/1.8/gems/activerecord-1.14.0/
      lib/active_record/vendor/mysql.rb:104:in 'initialize'
    /usr/local/ror/lib/ruby/gems/1.8/gems/activerecord-1.14.0/
      lib/active_record/vendor/mysql.rb:104:in 'real_connect'
    /usr/local/ror/lib/ruby/gems/1.8/gems/activerecord-1.14.0/
      lib/  active_record/connection_adapters/mysql_adapter.rb:331:in 'connect'
    
    
...results deleted...


8 tests, 0 assertions, 0 failures, 16 errors
/usr/local/ror/bin/ruby -Ilib:test "/usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/
   lib/rake/rake_test_loader.rb"  
rake aborted!
Test failures

(See full trace by running task with --trace)

I can already see that there are problems: rake generated sixteen errors. The trace shows that Rails could not establish a connection. I simply forgot to start my database engine. I'll start it and run rake again. This time, I get the results shown in Listing 2:

rake 
(in /Users/batate/rails/trails)
/usr/local/ror/bin/ruby -Ilib:test 
   "/usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb" 
   "test/unit/trail_test.rb" 
Loaded suite /usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader
Started
...
Finished in 0.09541 seconds.

3 tests, 5 assertions, 0 failures, 0 errors
/usr/local/ror/bin/ruby -Ilib:test 
   "/usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb" 
   "test/functional/trails_controller_test.rb" 
Loaded suite /usr/local/ror/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader
Started
........
Finished in 0.169756 seconds.

8 tests, 28 assertions, 0 failures, 0 errors

That's much better. The tests are running, and we're ready to build some more test cases. If you look closely at Listing 2, you'll find that rake generated two sets of results. The first set -- the unit test from Part 1 -- should look familiar. The next set is a functional test, generated automatically from the scaffolding.

Back to top

A quick controller and view primer

Before looking at the test code, you need a better understanding of the Rails user interface layer. When you generated scaffolding code in Part 1 with script/generate scaffold Trail Trails, Rails created a controller and a series of views for the application, based on the contents of the database. The controller code is in app/controller/trails_controller.rb, and the views are all in various directories within app/views/trails. The application has:

• Default Web page implementations for a list of trails (called list)
• A show page for a trail's details
• A generic form for a trail
• Pages to create or edit trails

To see how it all hangs together, look at the list method within trails_controller.rb, shown in Listing 3:

def list
  @trail_pages, @trails = paginate :trails, :per_page => 10
end

Incoming Hypertext Transfer Protocol (HTTP) requests come into the controller. (HTTP is the underlying protocol that powers browsers, Rails, and all browser-based applications.) Later in this article, you'll see that functional tests invoke functional test cases by using HTTP commands. The code in Listing 3 sets up the instance variables Rails needs to show a paginated list of trails. The view needs a paginator object that Rails assigns to @trail_pages and a list of trails in @trails. By default, Rails renders the view with the same name as the controller method. To see the view, look at the table definition within app/views/trails/list.rhtml, shown in Listing 4:

<table>
   <tr>
   <% for column in Trail.content_columns %>
      <th><%= column.human_name %></th>
   <% end %>
   </tr>

<% for trail in @trails %>
   <tr>
   <% for column in Trail.content_columns %>
      <td><%=h trail.send(column.name) %></td>
   <% end %>
      <td><%= link_to 'Show', :action => 'show', :id => trail %></td>
      <td><%= link_to 'Edit', :action => 'edit', :id => trail %></td>
      <td><%= link_to 'Destroy', { :action => 'destroy', :id => trail }, 
         :confirm => 'Are you sure?', :post => true %></td>
   </tr>
<% end %>
</table>  

The view strategy in Rails is to create a simple string and then make some substitutions. This strategy, called templating, forms the foundation of most modern Web frameworks, including Java frameworks such as Tapestry, JavaServer Faces (JSF), JavaServer Pages (JSP), and WebWork. In this case, Rails does the following:

1. Executes the code fragment between <% and %> (called a statement) and replaces that fragment with the fragment's execution output. A statement might not exist.
   
   
2. Executes the code fragment between <%= and %> (called an expression) and replaces that fragment with the value returned by the fragment.
   
   
3. Handles layouts, partials, helpers, and other types of fragments. These features let you build complex Web pages from different composite parts. I won't go into such details here.

With the template strategy in mind, look again at Listing 4. You see the list.rhtml view accessing the Active Record Trail model and looping through each trail in @trails with the <% for trail in @trails %> command. (You populated the @trails instance variable in the controller.) For each trail, the view gets Trail.content_columns, which is a list of columns in the trails table in the trails_development database. Then, the view loops through each of these to provide the value of each column in the database. The trail.send(column_name) command sends the name, difficulty, and description methods to trail.

It's time to see the results on the screen. If you recall, you already have some test data if you typed in the examples from Part 1 in the form of fixtures. To load them into the development environment (fixtures are loaded into the test environment by default), type rake load_fixtures. You can see the result by starting a Rails server (script/server on Unix or ruby script/server on Windows) and pointing your browser to localhost:3000/trails/list. In this URL, trails is the name of the controller and list is the name of the action, implemented with the list controller method. Figure 1 shows the results:

As you'd expect, you see a table with each trail's name, description, and difficulty. Next, I'll show you how the Rails functional testing framework accesses a Web page by simply doing an HTTP put.

Back to top

Breaking down the functional tests

Recall that Rails unit tests handle only models. A functional test in Rails exercises a feature, from top to bottom, including model, view, and controller, by invoking the Web page and examining the result. This level of integration testing is important because you ensure that the interaction between the major elements of your system works as you expect it to for each feature you provide.

Each functional test case for Rails does HTTP puts and gets. These invoke controller actions; the controllers access both models and views and render a Web page and a result. For a detailed working example, look at the test case that Rails generated for you with your scaffolding:

def test_list
  get :list

  assert_response :success
  assert_template 'list'

  assert_not_nil assigns(:trails)
end

The test case in Listing 5 does a simple HTTP get with the get :list command. Then, the test case runs three assertions:

• assert_response :success: The HTTP command completed successfully.
• assert_template 'list': The controller action rendered the list template.
• assert_not_nil assigns(:trails): The controller assigned the @trails instance variable to some non-nil value.

As with the unit-testing framework, if all assertions are true and no errors occur, the test case passes; otherwise, the test case fails.

The test_list test case asserts a response of :success, but it could assert :redirect (for an HTTP redirect), :missing (for not_found), or any integer for individual HTTP return codes. See Resources for a link to an exhaustive list of HTTP return codes. Now look at test_create, which uses an HTTP put. Change test_create to look like Listing 6:

def test_create
   num_trails = Trail.count

   post :create, :trail => {:name => "Hermosa Creek", :description => 
      "Lots of altitude, all down", :difficulty => "Medium"}

   assert_response :redirect
   assert_redirected_to :action => 'list'

   assert_equal num_trails + 1, Trail.count
end

The automatically generated version of this test case in trails_controller_test.rb included post :create, :trail => {}, which invokes the create method with an empty hash map for the new trail. That code would have created a new trail with a Trail object with nulls for all attributes. Listing 6 changes the code to pass a hash map representing the attributes for a trail. This hash map interface is quite useful for specifying objects within a testing framework. Then, the test case uses the Trail model to make sure a new trail was created.

The test cases in Listings 5 and 6 do not handle every detail, as the unit tests in Part 1 did. But they make sure that the business logic is invoked, that the controller logic doesn't detect any errors, and that the right HTTP response is achieved.

Rails provides still one more kind of test case: the integration test.

Back to top

Integration tests

Whereas functional tests exercise a single feature, integration tests work through scenarios that can hit many different pages. For example, a shopping cart unit test might test that you can add an item to a cart through the model API. A functional test for a shopping cart would ensure that you could add an item to a cart through a post to a Web page. An integration test case would ensure you could log in, add a few items, and check out.

In "Running Your Rails App Headless" (see Resources), Mike Clark, one of the Rails community's leading testing experts, walks through the integration testing framework in detail. He starts the discussion by showing how to run an application without Web pages (that is, headless). This capability makes it much easier to collect enough information to write integration test cases. Beginning with Rails 1.1, you can invoke your controllers directly from the console. You access your application's Web pages, without a browser, by calling put and get methods on the object called app.

Start the console and issue a list action through an HTTP get, by typing the commands in Listing 7:

> script/console Loading development environment.
>> app.class
=> ActionController::Integration::Session
>> app.get('trails', 'list')
=> 200
>> app.get("trails/list")
=> 200
>> app.response =~ /Barton Creek/
=> false
>> app.response =~ /Emma Long/
=> false
>> app.response.body =~ /Emma Long/
=> 331
>> 

In Listing 7, you send a request from console, in two forms, to invoke the list action on the trails controller. Then, you can see that the resulting HTML page contains Emma Long, one of the trails, by matching the regular expression /Emma Long/. You can continue to run posts and gets:

>> app.post("trails/destroy/1")
=> 302
>> Trail.find_all
=> [#<Trail:0x25a8e34 @attributes={"name"=>"Bear Creek", "id"=>"2", 
   "description"=>"Too many downed trees.", "difficulty"=>"easy"}>]
>> Trail.find_all.size
=> 1
>> app.response.redirect_url
=> "http://www.example.com/trails/list"
>> 

Through the console integration-testing API, you now have enough information to build an integration test. Generate an integration test case with script/generate integration_test DestroyAndShow and edit it to look like Listing 9:

require "#{File.dirname(__FILE__)}/../test_helper"

class DestroyAndShowTest < ActionController::IntegrationTest
  fixtures :trails

  def test_multiple_actions
    get "trails/list"
    assert_response :success
    
    post "trails/destroy/1"
    assert_response :redirect
    assert_nil(response.body =~ /Emma Long/)
    assert_equal(2, Trail.find_all.size)
    
    follow_redirect!    
    assert_response :success
    
    
    get "trails/show/2"
    assert_response :success
    
    
  end
end

This example uses the same integration framework that you used through the Rails console, and it uses the same assertions model as the functional and unit testing frameworks. You can run the test cases with rake, or you can run the test cases individually. By using the console and the integration framework in concert, you can try the various aspects of your application, get results within the console, and use those results to supply your assertions in your automated test cases.

Back to top

Testing in Ruby versus the Java language

You can begin to see how integration tests in an integrated framework are different. With this example, you were able to use fixtures, and they worked within the integration-testing framework. The assertions, and ways of expressing ideas such as requests and responses, have a uniform feel.

Some of the capabilities in the base Ruby language make Rails testing even more powerful. You can take advantage of Ruby to do things such as mocks and stubs. As I write this article, I'm working on some automated integration tests in Rails. I have a class that is dependent on the current date. I simply open up the existing Ruby class for Date and redefine the today method to return Date.civil(2, 2, 2006), as in Listing 10:

require "#{File.dirname(__FILE__)}/../test_helper"

 class Date
   def self.today
     return Date.civil(2006, 2, 2) 
   end
 end

class NameOfTest ...continue test case here...

I don't need to do anything else to my test case. Now, whenever the test case is run, today will be the American holiday Groundhog Day. With a mere five lines of code, I have a workable testing stub. In this case, I can use this mock object for only a single test case. If I need to run the mock for more than one test case, I can add the code for the mock to test/mocks and reuse it.

All in all, I would rate the Ruby testing experience as both much more necessary -- because of a dynamic language's error-prone nature -- and more powerful. Part of the power comes from the integrated experience that makes code generation, assertions, database backing, and diagnostic tools work seamlessly through Rails.

But Java technology does have its advantages. It integrates testing into development environments better, and it has better continuous-integration tools. You can also find more frameworks to mimic the most common enterprise features. Java developers have another philosophical advantage: they can run applications more easily without database backing. Testing Rails applications without database backing would not be nearly as meaningful, given that much of Rails's value derives from weaving SQL features together through metaprogramming. The result is that a Java test suite can often run much more quickly because the test cases in the suite don't need to access a database.

If you use Java code generation, Rails can give you some good ideas about how to augment your code generation with test generation. If you're supplementing your own testing frameworks, the Rails testing API is simple and clean. And if you're interested in stepping out beyond the Java programming language, Rails can provide some real value for the right lightweight, database-backed applications.

In the next article in the series, I'll step away from Rails and look at Web-based templating strategies. You'll see how code generation can work for dynamic languages.

Resources

Learn

• Beyond Java (O'Reilly, 2005): The author's book about the Java language's rise and plateau and the technologies that could challenge the Java platform in some niches.
  
  
• HTTP 1.1 Status Code Definitions: Provides more information about HTTP standard status codes.
  
  
• "Running your Rails App Headless": One of the best discussions about the Rails integration testing framework.
  
  
• "Book review: Agile Web Development with Rails" (Darren Torpey, developerWorks, May 2005): Get the scoop on a book that deepens readers' understanding of Rails and the rationale behind agile development approaches.
  
  
• "Test-first Ruby programming" (Pat Eyler, developerWorks, May 2005): Build a simple Ruby application following Test First principles.
  
  
• Java To Ruby: Things Your Manager Should Know (Pragmatic Bookshelf, 2006): The author's book about when and where it makes sense to make a switch from Java programming to Ruby on Rails, and how to make it.
  
  
• Programming Ruby (Pragmatic Bookshelf, 2005): A popular book on Ruby programming.
  
  
• The Java technology zone: Hundreds of articles about every aspect of Java programming.
  
  

Get products and technologies

• Ruby on Rails: Download the open source Ruby on Rails Web framework.
  
  
• Ruby: Get Ruby from the project Web site.
  
  

Discuss

• Improve Your Code Quality discussion forum: For questions related to testing and code quality, this discussion forum, hosted by Andrew Glover, is a must.
  
  

About the author

Bruce Tate is a father, mountain biker, and kayaker in Austin, Texas. He's the author of three best-selling Java books, including the Jolt winner Better, Faster, Lighter Java. He recently released Beyond Java.. He spent 13 years at IBM and is now the founder of the RapidRed consultancy, where he specializes in lightweight development strategies and architectures based on Java technology and Ruby on Rails. 

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Extending beyond unit testing
• Running your tests
• A quick controller and view primer
• Breaking down the functional tests
• Integration tests
• Testing in Ruby versus the Java language
• Resources
• About the author
• Comments

Next steps from IBM

Spring, JRuby, and Ajax development is easier with WebSphere Application Server - a smart Java 5 and J2EE Web services-based application server.

---

• Try: The no-charge WebSphere Application Server Community Edition is a pre-integrated, lightweight Java 5 application server built on Apache Tomcat and other best-of-breed open source software such as OpenEJB, Apache Axis, and Apache Derby.
• Article: The article leverage the Spring Framework and the WebSphere Application Server to improve your J2EE project productivity.
• Tutorial: See how the free WebSphere Application Server and XML can improve the efficiency of your JRuby on Rails and Ajax development.
• Buy: WebSphere Application Server - Express

My developerWorks community

Dig deeper into Java on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers