The case for thorough unit testing

A unit test is usually written to test a piece of source code. Theoretically, this piece, or unit, should be the smallest testable portion of source code. Normally, a unit test is automated, but it does not have to be, and the result of the unit test indicates whether the code is behaving as designed.

It's common knowledge that the software developer is often crunched for time. There is extreme pressure to release products to the market as quickly as possible, so why spend even more time coding unit tests? The answer is that an adequate unit test suite not only produces higher quality code, but it also saves time in the end because you're likely to spend less time fixing bugs. And if you follow the agile development method, writing unit tests before attempting to write the source code will likely cause you to write less code. You will have thought through the design before simply coding away, which reduces the amount of code written to achieve the goal of the unit test.

Back to top

What is the Dojo Objective Harness?

There are a lot of supporters of unit testing, as can been seen in Extreme Programming, Agile, and so on. However, the widespread use of Asynchronous JavaScript + XML (Ajax) and Web 2.0 user interfaces has produced a need for client-side unit testing. The Dojo Objective Harness is the Web 2.0 UI developer's answer to JUnit. Unlike existing JavaScript unit test frameworks, such as JSUnit, the DOH not only provides a framework for automating JavaScript functions with or without the use of Dojo, it can also unit test the actual visualization of the user interface. This is because the DOH (what a great acronym) offers both command-line and browser-based interfaces to the testing framework.

Back to top

Browser and non-browser environment

As previously mentioned, the DOH provides a command-line interface as well as a browser-based interface. If the unit tests need to be fully automated and no visualization component is required, the command-line interface is a good choice because it can be kicked off by a build script and the results can be logged. Also, this interface provides a unit test environment very similar to JUnit. The DOH uses Rhino, an open-source JavaScript engine written in Java™ code, for its command-line interface. Because of this, references to the document, window, DOMParser, and XMLHttpRequest objects cannot be resolved. Another issue with Rhino is that it uses a different JavaScript interpreter than the popular browsers, so it is possible for a test to pass in one runtime and not another.

If the visual component of the unit tests and access to various JavaScript objects are required, the browser-based interface is your best bet. The caveat here is that unit tests using the browser are not 100% automated; you must launch the unit test in the desired browser and inspect the results. This is not totally surprising. Ensuring that a UI looks "good" is normally a subjective decision by a human. The browser test runner provides two ways to view the unit test results: visual results and unit test statistics. Figure 1 shows the test cases run on the left, with the visualization of the code execution on the right under the Test Page tab. (Click here to see a larger version of Figure 1.)

Figure 2 shows the unit test stats under the Log tab. (Click here to see a larger version of Figure 2.)

Back to top

Browser compatibility

As anyone developing client-side code for multiple browsers and versions knows, the ability to quickly detect differences in browser behavior through unit testing is important. Because the DOH test runner is just HTML and JavaScript, unit tests can be executed in any browser. This means that you can run unit tests in FireFox, Internet Explorer, and Safari (and different versions of each) and compare the results with one another. Not only can you ensure that basic JavaScript methods behave the same way across multiple platforms, you can also ensure that the visualization is the same, or at least acceptable, in the various platforms as well. We all know that a widget may look perfect on one browser, but is barely recognizable in another. Cross-browser bugs are often nasty and difficult to fix. Testing browser compatibility early, and in an automated way, ensures that cross-browser support problems are surfaced and fixed before the software gets to the market.

Back to top

Available test functions

Every test framework needs to supply you with a method to check the outcome of a unit test and the DOH is no exception. The DOH offers three assertion APIs for use in test verification, as shown in Listing 1.

     doh.assertEqual(expectedResult, actualResult)
     doh.assertFalse(testCondition)
     doh.assertTrue(testCondition)

Additionally, the shorthand versions of these functions can be used. Listing 2 shows these versions.

     doh.is(expectedResult, actualResult)
     doh.f(testCondition)
     doh.t(testCondition)

When an assertion fails, an exception is thrown. If any type of exception is thrown in a unit test, the DOH declares the entire test as failed. This is important to know if you are expecting your test to throw an exception. In this case, you'll need to surround your code with a try catch block. When the unit tests are invoked, the DOH reports any errors that occurred and the specific test that has failed. The DOH also reports the total number of tests run, errors that occurred, and tests that failed.

When composing unit tests, it's best to keep the number of assertions to a minimum, as the DOH error reporting design can make it difficult to determine which assertion caused the failure. Although it's generally easier to determine which equals assertion caused the failure, true and false assertions are harder to find.

Sometimes errors occur in unit tests that aren't thrown by assertions. In these cases, either the unit test or the code to be tested is most likely incorrect. Luckily, the Firebug add-on for Firefox can also be used to debug fundamental code problems with unit tests.

Back to top

Asynchronous function testing

Wouldn't it be great to unit test the behavior of the asynchronous calls that a client-side application makes? The DOH can help. Testing the behavior of Ajax requests is one of the most valuable features of the DOH. Because its browser-based interface has access to the XMLHttpRequest object, the DOH can support asynchronous unit tests. To indicate that a test case is asynchronous, the test case alerts the harness by returning a doh.Deferred object. If the DOH is unaware that the test is asynchronous, after the code of the test is executed the DOH believes that the test is complete and no errors have occurred. Obviously, this leads to false positives and portions of your code left untested.

The test case itself must also be written with the understanding of an asynchronous context. When a doh.Deferred object is returned from a unit test, you must catch all errors from the asynch call and pass them to the object's errback method. If no exceptions occur, the object's callback method should be called with a parameter of true. This enables the DOH to report failed tests accurately.

To make writing asynchronous tests easier, the doh.Deferred object provides a getTestCallback function to implicitly handle exceptions that occur in the callback function of an asynchronous call. You just need to pass your test function to getTestCallback, which, in turn, contains the assertions that you want to execute. This relieves you of manually handling exceptions that occur during an asynchronous call. See Writing your own test suite for more details.

The DOH also allows you to specify a custom timeout in milliseconds that will fail the test if a response is not returned within the specified time. The default timeout value for asynchronous tests is 500 ms, or half a second, so many times it's a good idea to explicitly specify a longer timeout value so your test does not fail.

Back to top

Writing your own test suite

Writing your own test suite with the DOH can appear daunting at first, but it is actually not very difficult. The DOH framework is extremely flexible in how tests can be defined and loaded, so often the load flow can be modified to suit your particular structure. That said, the unit tests of Dojo almost all follow a common structure to make it simple for new module owners to pick up and run with it. It is recommended that you follow existing conventions until you are comfortable with how the DOH functions.

Back to top

The basic DOH test case structure

The test case structure is illustrated by using a demonstration module, demo.doh, that is placed as a peer module to the Dojo directory structure. The reason for the peer structure is that the DOH framework uses Dojo's module loader structure, and without doing a dojo.registerModulePath() to tell dojo where your source code is located, it assumes that your module directory is a peer directory to Dojo. While you can work with this by editing util/doh/runner.html to register your module paths, along with the import of doh.runner, ahead of time, it is simpler for beginning users to conform to the expectations of Dojo. Figure 3 shows the general directory structure that will be referred to throughout this section.

As Figure 3 shows, it is a good practice to have each dojo module contain unit tests for just that module. This lets the module developer run the unit tests separate from the overall project. That said, it does not mean that there cannot be a test suite module file that loads all the unit tests for all modules. Doing this will be covered in a later section of this article, after the basics of the structure are explained in detail.

Back to top

A demonstration set of test cases for the DOH

Before we get into the tests and how they work, it helps to understand what is being tested. In the case of demo.doh, it is a module that has a file containing helper functions and a simple DemoWidget. The reason for both is that they effectively illustrate how to test non-visual (JavaScript functions), as well as widgets used directly in html, just as how they are used in an application. These files implement trivial behaviors to make them easy to understand. Listing 3 shows the contents of the demoFunctions.js and Listing 4 shows the contents of the DemoWidget.js.

dojo.provide("demo.doh.demoFunctions");

//This file contains a collection of helper functions that are not
//part of any defined dojo class.

demo.doh.demoFunctions.alwaysTrue = function() {
  //  summary:
  //    A simple demo helper function that always returns the boolean true when 
  //    called.
  //  description:
  //    A simple demo helper function that always returns the boolean true when 
  //    called.
  return true; // boolean.
};

demo.doh.demoFunctions.alwaysFalse = function() {
  //  summary:
  //    A simple demo helper function that always returns the boolean false when 
  //    called.
  //  description:
  //    A simple demo helper function that always returns the boolean false when 
  //    called.
  return false; // boolean.
};

demo.doh.demoFunctions.isTrue = function(/* anything */ thing) {
  //  summary:
  //    A simple demo helper function that returns true if the thing passed in is
  //     logically true.
  //  description:
  //    A simple demo helper function that returns true if he thing passed in is 
  //    logically true.
  //    This means that for any defined objects, or Boolean  values of true, it 
  //    should return true,
  //    For undefined, null, 0, or false, it returns false.
  //  thing:
  //    Anything.  Optional argument.
  var type = typeof thing;
  if (type === "undefined" || thing === null || thing === 0 || thing === false) {
    return false; //boolean
  }
  return true; // Boolean
};

demo.doh.demoFunctions.asyncEcho = function(/* function */ callback,
                                                    /* string */ message){ 
  //  summary:
  //    A simple demo helper function that does an asynchronous echo 
//     of a message.
  //  description:  
  //    A simple demo helper function that does an asynchronous echo 
//      of a message.
  //    The callback function is called and passed parameter 'message' 
//       two seconds 
  //    after this helper is called.
  //  callback:
  //    The function to call after waiting two seconds.  Takes one
//       parameter, 
  //    a string message.
  //  message:
  //    The message to pass to the callback function.
  if (dojo.isFunction(callback)) {
    var handle;
    var caller = function() {
      callback(message);
      clearTimeout(handle);
      handle = null;
    };
    handle = setTimeout(caller, 2000);
  }
};

dojo.provide("demo.doh.DemoWidget");
dojo.require("dijit._Widget");
dojo.require("dijit._Templated");

dojo.declare("demo.doh.DemoWidget", [dijit._Widget, dijit._Templated], 

     //The template used to define the widget default HTML structure.
     templateString: '<div dojoAttachPoint="textNode" style="width: 150px; ' +
          ' margin: auto; background-color: #98AFC7; font-weight: bold; color: ' + 
          'white; text-align: center;"></div>',

     textNode: null,          //Attach point to assign the content to.

     value: 'Not Set',     //Current text content.

     startup: function() {
          //     summary:
          //          Overridden startup function to set the default value.
          //     description:
          //          Overridden startup function to set the default value.
          this.setValue(this.value);
     },

     getValue: function() {
          //     summary:
          //          Simple function to get the text content under the textNode
          //     description:
          //          Simple function to get the text content under the textNode
          return this.textNode.innerHTML;
     },

     setValue: function(value) {
          //     summary:
          //          Simple function to set the text content under the textNode
          //     description:
          //          Simple function to set the text content under the textNode
          this.textNode.innerHTML = value;
          this.value = value;
     }
});

Back to top

Testing stand-alone functions, both synchronous and asynchronous, in the DOH

As you can see in Listings 3 and 4, we have implemented a simple widget and small set of stand-alone functions. Now that they have been defined, it would be great to implement unit tests that exercise the functions and the widgets to confirm they behave as expected. With other JavaScript Unit Test frameworks, the synchronous functions would be easily testable, but the asynchronous function demo.doh.demoFunctions.asyncEcho and the widget would not. So, enter the DOH and its facility for handling widget testing in browsers as well as synchronous function testing.

The simplest place to begin is to test stand-alone functions. Writing stand-alone function test cases is as simple as defining a JavaScript array. The array should contain test functions, test fixtures, or a mix of both. The complexity of what you're testing determines which one you should use. In most cases, simple test functions are more than adequate for testing code. It is only when you need to alter timeouts, perform setup operations, or tear down data after a test that you would need to construct a test fixture. After the array of functions have been defined, to register them with the DOH is a matter of calling the tests.register with two parameters, the name you want to assign to the collection of tests, and the array of the tests. Listing 5 is the code listing for a small set of tests for the demoFunctions.js stand-alone functions.

dojo.provide("demo.doh.tests.functions.demoFunctions");

//Import in the code being tested.
dojo.require("demo.doh.demoFunctions");

doh.register("demo.doh.tests.functions.demoFunctions", [
     function test_alwaysTrue(){
          //     summary:
          //          A simple test of the alwaysTrue function
          //     description:
          //          A simple test of the alwaysTrue function
          doh.assertTrue(demo.doh.demoFunctions.alwaysTrue());
     },
     function test_alwaysFalse(){
          //     summary:
          //          A simple test of the alwaysFalse function
          //     description:
          //          A simple test of the alwaysFalse function
          doh.assertTrue(!demo.doh.demoFunctions.alwaysFalse());
     },
     function test_isTrue(){
          //     summary:
          //          A simple test of the isTrue function
          //     description:
          //          A simple test of the isTrue function with multiple permutations of 
           //          calling it.
          doh.assertTrue(demo.doh.demoFunctions.isTrue(true));
          doh.assertTrue(!demo.doh.demoFunctions.isTrue(false));
          doh.assertTrue(demo.doh.demoFunctions.isTrue({}));
          doh.assertTrue(!demo.doh.demoFunctions.isTrue());
          doh.assertTrue(!demo.doh.demoFunctions.isTrue(null));
          doh.assertTrue(!demo.doh.demoFunctions.isTrue(0));
     },
     {
          //This is a full test fixture instead of a stand-alone test function.  
          //Therefore, it allows over-riding of the timeout period for a deferred test.  
          //You can also define setup and teardown function
          //for complex tests, but they are unnecessary here.
          name: "test_asyncEcho",
          timeout: 5000, // 5 seconds.
          runTest: function() {
               //     summary:
               //          A simple async test of the asyncEcho function.
               //     description:
               //          A simple async test of the asyncEcho function.
               var deferred = new doh.Deferred();
               var message  = "Success";
               function callback(string){
                    try {
                         doh.assertEqual(message, string);
                         deferred.callback(true);
                    } catch (e) {
                         deferred.errback(e);
                    }
               }
               demo.doh.demoFunctions.asyncEcho(callback, message);
               return deferred;      //Return the deferred.  DOH will 
                                     //wait on this object for one of the callbacks to 
                                     //be called, or for the timeout to expire.
          }
     }
]);

As Listing 5 shows, defining a basic set of tests does not require a lot of code per test, even if it requires a test fixture to execute it due to needing to alter the default timeout. The tests also show one of the other best practices for writing unit tests; keep the tests as simple and small as possible. The reason to only have a few asserts per test is that it makes it quicker to isolate the failure in the test from the error the DOH reports. Too many asserts can make it difficult to determine exactly which assert caused the error.

The other point of interest to note with the tests is how the async tests usually need to be written. Because the callback runs later, it cannot be easily try/catch caught by the DOH when there is a failure, like it does for a synchronous test. Instead, the unit test must take this into account. With the asyncEcho test, it wraps the asserts in a try/catch, and any errors are passed back to the DOH through the deferred.errback(error) call. If wrapping was not done, the test would still fail on an error, but all the DOH would report is the test timed out. This is because the error thrown from the failed assert prevented the deferred.callback() from being executed. So, the test, according to the DOH, never reported completion, and therefore, gets timed out. In other words, the only way the DOH knows an async test passed or failed is if an operation is invoked on the Deferred.

Back to top

Testing widgets in the DOH

As the previous section shows, testing simple stand-alone functions is easy to do. You create an array of functions or test fixtures, register them, and the DOH will execute them when loaded. That's great, but stand-alone functions and non-visual code isn't what JavaScript is all about; it's about manipulating the browser DOM to provide a more interactive look and feel. So, the next question to explore is how do you test widgets?

The great news is that the DOH provides a good framework and method for registering tests that require a Web browser to load an HTML file that instantiates the widgets to be tested. Effectively, what the DOH does is create a bridge between an instance of the DOH running in an HTML file in an iframe and the instance of the DOH running its UI and stand-alone tests. Something to remember here is that unlike the stand-alone function tests, widget tests cannot generally be run headless through a JavaScript interpreter like Rhino.

So, how do you define widget tests? Well, you define an HTML file that instantiates the DOH, instantiates widgets, then defines the test functions to execute. Listing 6 shows a code listing of an HTML file that makes use of the DOH to test demo.doh.DemoWidget.

<html>
    <head>
        <title>DemoWidget Browser Tests</title>
        <script type="text/javascript" src="../../../../dojo/dojo.js" 
                djConfig="isDebug: true, parseOnLoad: true"></script>
        <script type="text/javascript">
        dojo.provide("demo.doh.tests.widgets.DemoWidgetHTML");
        dojo.require("dojo.parser");
        dojo.require("doh.runner");
        dojo.require("demo.doh.DemoWidget");

        dojo.addOnLoad(function(){
             doh.register("demo.doh.tests.widgets.DemoWidget", [
                  function test_DemoWidget_getValue(){
                         //     summary:
                         //          Simple test of the Widget getValue() call.
                     doh.assertEqual("default", dijit.byId("demoWidget").getValue()); 
                  },
                  function test_DemoWidget_setValue(){
                         //     summary:
                         //          Simple test of the Widget setValue() call.
                    var demoWidget = dijit.byId("demoWidget");
                    demoWidget.setValue("Changed Value");
                   doh.assertEqual("Changed Value", demoWidget.getValue());
                  }
             ]);
          //Execute D.O.H. in this remote file.
             doh.run();
        });
        </script>
    </head>
    <body>
        <!-- Define an instance of the widget to test. -->
        <div id="demoWidget" dojoType="demo.doh.DemoWidget" value="default"></div>
    </body>
</html>

So, as Listing 6 shows, it is a stand-alone file that runs the DOH. That's great, but it doesn't display the DOH's UI, so it's difficult to tell if tests pass or not. It would be great if the DOH provided a mechanism that could still run this HTML file and still display the UI. Well, good news, it can. The DOH has a another test registration function called doh.registerUrl(). This function lets you point the DOH runner.html UI at a separate HTML file. What it will then do is load that HTML file into a frame, connect the DOH instance created by that HTML file with the UI's DOH instance, and then the UI can also display failures and successes from the HTML page! Listing 7 shows the code for the module file that registers a URL as a source of tests and results.

dojo.provide("demo.doh.tests.widgets.DemoWidget");

if(dojo.isBrowser){
     //Define the HTML file/module URL to import as a 'remote' test.
     doh.registerUrl("demo.doh.tests.widgets.DemoWidget", 
                         dojo.moduleUrl("demo", 
“doh/tests/widgets/DemoWidget.html"));
}

Back to top

Bringing it all together: Combining the test definitions into a single DOH test suite

You have now seen how to write individual test files. As demonstrated, writing single tests is not complicated. So, the question that remains is how do you take these test definitions, load them into the DOH's UI, and execute them. This is also not difficult. You write an HTML file that redirects to the runner.html of the DOH. As part of the redirect, you pass a query parameter that defines what JavaScript module file to load. This single module file, usually called module.js, uses dojo.require() to load each of your test files. When the dojo.require() brings the files in, they register the tests. When all test files have been loaded by the DOH the framework automatically executes the tests. Listing 8 is the redirection file. Listing 9 is the module.js file that brings in all your test files.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
  <head>
    <title>demo.doh Unit Test Runner</title>
    <meta http-equiv="REFRESH"         
content="0;url=../../../util/doh/runner.html?testModule=demo.doh.tests.module">
  </head>
  <body>
      Redirecting to D.O.H runner.
  </body>
</html>

dojo.provide("demo.doh.tests.module");
//This file loads in all the test definitions.  

try{
     //Load in the demoFunctions module test.
     dojo.require("demo.doh.tests.functions.demoFunctions");
     //Load in the widget tests.
     dojo.require("demo.doh.tests.widgets.DemoWidget");
}catch(e){
     doh.debug(e);
}

Back to top

In conclusion

While the DOH can be daunting for the novice user, it is a flexible and powerful unit testing framework. It modularizes tests into separately loadable files, provides functions to associate tests as groups, provides a series of test APIs to assert conditions about the code being executed, and even provides a framework for handling asynchronous tests and browser tests for widgets through URL registration and iframe page loading.

By looking at the DOH piece by piece, the complexity disappears. Writing simple test cases is quick and easy, and combining those test cases into a suite is nothing more than writing a single JavaScript file that dojo.require()'s in each separate set of tests. This module file becomes your test suite entry point. The DOH also provides a powerful UI that shows success, failures, and even what errors were thrown. To make use of it, all that has to occur is that the runner.html is loaded with a query parameter defining which file to load that will register tests.

Lastly, the DOH is not limited to browser environments. The basic DOH loader and framework can be used in headless JavaScript environments such as SpiderMonkey and Rhino. The DOH is truly one of the most complete and effective frameworks for testing JavaScript code.

Back to top

Download

Information about download methods

Resources

• The Dojo toolkit Web site provides documentation on Dojo and the Dojo Objective Harness.
  
  
• Dustin Machi wrote an insightful blog entry on unit testing with the DOH.
  
  
• Learn more about unit testing.
  
  
• Rhino is the Java-based JavaScript interpreter used by the DOH.
  
  
• Agile software development and extreme programming both promote writing unit test cases before developing the source code.
  
  
• More information on other Ajax technologies (including Dojo) can be found in the developerWorks Ajax resource center.
  
  
• You can also get a complete reference of the Dojo API.
  
  

About the authors

Jared Jurkiewicz is an advisory software engineer in the WebSphere® family of products. He has held many roles in the WebSphere organization, from being a UNIX® operating systems expert, to being the lead on handling initial support of new operating systems and hardware platforms. His current assignment is as the release architect for WebSphere FeaturePack for Web 2.0, and he is also a contributor and committer to the Dojo Toolkit.

Stephanie Walter is an advisory software engineer on the Tivoli Service Availability and Performance Management architecture team. She previously led the development of WebSphere Business Monitor Dashboards and has worked extensively with Dojo and Web 2.0 technologies.

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• The case for thorough unit testing
• What is the Dojo Objective Harness?
• Browser and non-browser environment
• Browser compatibility
• Available test functions
• Asynchronous function testing
• Writing your own test suite
• The basic DOH test case structure
• A demonstration set of test cases for the DOH
• Testing stand-alone functions, both synchronous and asynchronous, in the DOH
• Testing widgets in the DOH
• Bringing it all together: Combining the test definitions into a single DOH test suite
• In conclusion
• Download
• Resources
• About the authors
• Comments

Next steps from IBM

Easily build dynamic Web page capabilities with the powerful Open Source Dojo programming and WebSphere Portal.

---

• Try: Save valuable time with easily tailored WebSphere Portal out-of-the-box Internet and intranet example sites.
• Article: Use these advanced Ajax techniques to create dynamic, cooperative portlets with the Dojo toolkit and WebSphere Portal.
• Tutorial: This tutorial takes you through the steps to enable drag-and-drop functionality using WebSphere Portal.
• Buy: WebSphere Portal express

My developerWorks community

Dig deeper into Web development on developerWorks

Try IBM Cognos 8 Business Intelligence

Use reports, analysis, dashboards and scorecards to monitor business performance, analyze trends and measure results.

Special offers