Unit Testing COBOL and PL/I Applications with zUnit, Part 1 (IBM Rational community)

Unit Testing COBOL and PL/I Applications with zUnit, Part 1

sjpeich Comments (3) Visits (17590)

The IBM z/OS Automated Unit Testing Framework (zUnit) feature of IBM Rational Developer for System z (RDz) provides a code-driven unit testing framework for Enterprise COBOL and PL/I developers. zUnit is an automated solution for running and verifying test cases that are implemented in Enterprise COBOL or PL/I. Just like JUnit, zUnit adopts the xUnit framework. In Part I of this topic, we look at what xUnit and zUnit are and how they relate to each other. In a later post, we’ll look at creating and running zUnit test cases.

What is xUnit?

xUnit is the name given to a category of frameworks that assist developers in writing code to perform repeatable, self-checking unit tests [1]. xUnit defines a set of concepts that together provide a lightweight architecture for implementing unit testing frameworks. JUnit, for example, is a very popular instance of the xUnit architecture [2]. Not unlike other instances of the xUnit architecture, JUnit is language-specific. Several attempts at implementing unit test frameworks for various programming languages are tracked at [3]. Figure 1 illustrates the components of the xUnit architecture.

Figure 1. Components of the xUnit architecture.

The following descriptions apply to the components of the xUnit architecture:

Configuration: an object or structured file that provides the test runner with a list of test cases and test suites that should be run.
Test Case: a program (or class) that implements one or more tests against some functional unit. Any number of individual tests can be included in a test case. It is highly encouraged that tests are not state or order-dependent to allow the test runner to randomize the order in which they are run. The logical flow of an xUnit test case is:
- initialize(“descriptive test case name”);
- setup(“test001”); // allocate test fixture for test001
- test001(); // test some function
- teardown(“test001”); // release test fixture for test001
- . . .
- setup(“testnnn”); // allocate test fixture for testnnn
- testnnn(); // test some function
- teardown(“testnnn”); // release test fixture for testnnn
Test Suite: a program (or class) that acts as a shallow container for test cases. Test suites are used to define subsets of test cases; these subsets often reflect some composite functionality of the application being tested.
Assertion: assertions are used to fail tests within a test case. For example if the functional unit being tested does not return the expected output for a particular input, an assertion exception is generated to end the test with a failing result.
Test Runner: An application that takes a configuration as input and runs the test cases and/or test suites specified therein. Test case and test suite results should be made available in some programmatically consumable way.
Results: an object or structured file that provides detailed run information for each test case and test suite that was executed. Information for failed test cases or test suites should include traceback information for assertions and unexpected exceptions.

JUnit (and consequently xUnit) was invented by Kent Beck, however his passion for programmer-centric unit testing date back to his Smalltalk days [4]. Kent's main concerns at the time he published the article “Simple Smalltalk Testing” were the lack of a testing culture in the Smalltalk community and the brittle nature of user-interface based tests. The philosophy, concepts, and examples provided in the article are reflected today in JUnit and other xUnit instances to encourage developers to adopt the practice of Test-Driven Development (TDD) [5] [6]. Probably the best way to describe TDD is through its motto of "Red, Green, Refactor." [6]. Figure 2 describes the TDD process for developers.

Figure 2. Test-Driven Development process for developers.

The long term value of adopting TDD and xUnit can be described using the following points:

As software complexity grows, confidence in making changes to a code base wanes.
Suites of xUnit tests defend against regressions, thereby maintaining confidence.
xUnit tests are runnable by other developers and the build process.
Automated xUnit tests allow for continuous testing [6] of the code base.
New/enhanced xUnit tests for every code change (maintain confidence).

Implementing the TDD process using an xUnit framework can be challenging because it requires that an application be implemented from the ground up with two principles in mind: functional decomposition (FD) and separation of concerns (SoC). These two principles require vigilance throughout the lifecycle of an application.

Functional decomposition (FD) describes a process by which the high-level tasks performed by an application program are iteratively divided into composite functions which are then further divided into elementary functions. Each high-level task maps to a composite function that executes one or more elementary functions to implement the program flow. When a program is written using FD, the following can be learned by reading the source code:

The high-level tasks that the program performs.
The composite function for each high-level task.
The program flow for each composite function.
The elementary functions that each composite function relies on.

Listing 1. Pseudocode for a composite function.

000: void depositFunds(DepositDetail dd) throws
000:                   VerificationException, DepositException {
001:     verifyCustomer(dd.getCustomer());
002:     verifyTargetAccount(dd.getTargetAccount());
003:     verifyFundingSource(dd.getFundingSource());
004:     doDeposit(dd);
005: }

Listing 1 provides pseudocode for a composite function depositFunds. From the listing we can see that the composite function is composed of four elementary functions: verifyCustomer, verifyTargetAccount, verifyFundingSource, and doDeposit. Each of these four functions are good candidates for an automated xUnit test case. For example, a test case for the verifyCustomer function would contain several tests that call the function passing valid and invalid input to confirm that the function behaves as expected. Many other composite functions likely use, or will use, the verifyCustomer function, therefore if this elementary function can be tested independently, a developer is saved from having to run the composite functions to periodically check their work while fixing or enhancing the elementary function; this enables the notion of continuous testing [7].

Separation of concerns (SoC) goes a step further than FD and prescribes that all functions be completely isolated from another--no function should be dependent on the internal state of another. Note that this implies that an elementary function should not depend on the internal state of a composite function. Listing 2 provides pseudocode for an elementary function updateTotalCost that is dependent on the state of the encompassing composite function computeTotalCost.

Listing 2. Pseudocode for a state-dependent elementary function.

001: void computeTotalCost(Item items[]) {
002:     GLOBAL Decimal totalCost = 0.00;
003:     foreach Item item in items do;
004:         call updateTotalCost(item);
005:     end;
006:     print “Total cost: ” . totalCost;
007: }
008:
009: void updateTotalCost(Item item) {
010:     EXTERNAL Decimal totalCost;
011:     totalCost += (item.quantity * item.cost);
012: }

The code in Listing 2 demonstrates why FD alone is not enough to ensure that elementary functions can be unit tested. If we want to be able to test the elementary function updateTotalCost without having to invoke the composite function computeTotalCost, we will need to implement a SoC between the two functions. Listing 3 provides the refactored code.

Listing 3. Result of applying SoC to Listing 2.

001: void computeTotalCost(Item items[]) {
002:     private Decimal totalCost = 0.00;
003:     foreach Item item in items do;
004:         totalCost = updateTotalCost(totalCost, item);
005:     end;
006:     print “Total cost: ” . totalCost;
007: }
008:
009: Decimal updateTotalCost(Decimal oldTotalCost, Item item) {
010:     Decimal newTotalCost =
011:             oldTotalCost + (item.quantity * item.cost);
012:     return newTotalCost;
013: }

After seeing Listing 2 and Listing 3, most would agree that SoC is completely obvious and everyone is or should be doing it. Unfortunately, a large percentage of existing programs written in COBOL, and to a lesser degree PL/I, do not implement SoC. Furthermore, FD is often limited to the use of paragraphs (labels in PL/I) or nested programs (nested procedures in PL/I). These paragraphs and nested programs usually operate on variables with program scope that are in a last-used state, violating SoC. Since paragraphs (labels) do not have interfaces, and nested programs are not externalized, it is difficult to isolate and unit test the logic contained within these paragraphs and nested programs from an external test case. Therefore, in order for COBOL and PL/I developers to apply TDD with xUnit to existing programs, refactoring using FD and SoC principles is likely required. Of course, for new COBOL and PL/I programs, FD and SoC can be applied from the start. There is still value in writing automated unit tests of composite functions, so it’s not an all-or-nothing proposition.

What is zUnit?

zUnit is an adaptation of the xUnit framework for use by Enterprise COBOL and PL/I developers on z/OS. Since xUnit is defined in object-oriented (OO) terms, it’s a natural/easy fit for OO languages like Java, but not COBOL or PL/I. Certain OO concepts such as type introspection, reflection, and inheritance significantly reduce, if not eliminate completely, the visible instrumentation in a typical JUnit test case and make life easier for the test runner. For example, the JUnit test runner can identify a class as being a legitimate test case through type introspection as well as execute those class methods whose names begin with “test”, or annotated with @Test, using reflection.

In the absence of object-oriented concepts, it is still feasible to implement a unit test framework like JUnit for COBOL and PL/I. However, certain aspects of said framework will be "closer to the metal"--as one would expect in a non-OO environment. For example, zUnit test cases templates generated by IBM Rational Developer for System z (RDz) contain code for handshaking with the zUnit test runner to identify a program as a test case and to dynamically add subprograms or procedures as individual tests. zUnit implements all of the recommended components of the xUnit architecture shown in Figure 1, except for Test Suites which have yet to be implemented. Note that this doesn’t prevent the zUnit test runner from running collections of test cases; this can easily be done by providing a list of test cases in the zUnit Configuration XML. Figure 3 illustrates the components of the zUnit architecture.

Figure 3. Components of the zUnit architecture.

The following descriptions apply to the components of the zUnit architecture:

zUnit Configuration XML: an XML file that provides the zUnit test runner with a list of zUnit test cases to run. Continuation options, such as whether to continue executing tests or test cases after an assertion exception (see AZUASTF[M|A]) or unexpected exception, may be specified in this file.
zUnit Test Runner: A z/OS batch mode application that takes as input a zUnit configuration XML file and generates a zUnit results XML file. The test runner operates in a try-catch mode so that it may continue to execute tests and test cases in the event of assertion exceptions or unexpected exceptions.
zUnit Test Case: A COBOL or PL/I program that is loaded and executed by the zUnit test runner. This type of program has a prescribed interface and internal structure where the location of the ADDTESTS, SETUP, TEARDOWN subprograms or procedures are supplied to the zUnit test runner on an initial call. After the initial call, the test runner calls ADDTESTS to allow adding of subprograms or procedures as individual tests {TEST001,...,TESTnnn}() . The runner then executes all of the tests in the order in which they were added, surrounding each with calls to SETUP and TEARDOWN. The logical flow of an zUnit test case is:
- TESTCASE(“user-supplied test case description”);
- ADDTESTS(); // add tests TEST001 through TESTnnn
- SETUP(“TEST001”); // allocate test fixture for TEST001
- TEST001(); // test some functional unit
- TEARDOWN(“TEST001”); // release test fixture for TEST001
- . . .
- SETUP(“TESTnnn”); // allocate test fixture for TESTnnn
- TESTnnn(); // test some functional unit
- TEARDOWN(“TESTnnn”); // release test fixture for TESTnnn
zUnit Results XML: an XML file generated by the zUnit test runner that includes information about the number of test cases and tests run, pass-fail statistics, assertions and unexpected errors, and runtime statistics.
zUnit Test Runner API: provides APIs for use by zUnit test cases:
- AZUTCINI: Initialize a COBOL or PL/I program as a zUnit test case. This API can only be called during execution of TESTCASE.
- AZUTCADD: Add an unbounded number of test subprograms or procedures to the test case. This API can only be called during execution of ADDTESTS.
- AZUASTF[M|A]: Fail the current test in a test case with or without a user-supplied message by generating an assertion exception. Calling this API from within SETUP or TEARDOWN will also fail the current test.

One key thing to note about the zUnit test runner is that it is not language-specific like the JUnit test runner. While zUnit currently claims support for COBOL and PL/I only, the truth of the matter is that zUnit is ignorant of the programming language that any given test case is implemented in. This leaves the door open for other programming languages such as C. The language-agnostic nature of zUnit is one of its most valuable characteristics. One could imagine a scenario where an enterprise has developers using multiple programming languages (COBOL, PL/I, C, HLASM) but wants to harmonize on a single unit testing framework to simplify build verification and QA reporting. zUnit is designed to support this type of scenario.

zUnit test cases must have certain structural elements, but otherwise the contents of test cases are not constrained. Test cases must be compiled and linked as DLLs and stored in PDSEs. The DLL linkage requirement does not extend beyond the test case itself. Therefore, load modules that will be linked with the test case need not be linked as DLLs. Figure 4 illustrates the required structure of zUnit test cases. Note that the names “TEST001” and “TESTnnn” are arbitrary, they can be whatever you prefer.

Figure 4. zUnit test case structure and interaction with test runner APIs.

Conclusion

In Part 1 of this topic we covered what xUnit and zUnit are and how they are related. We also covered Test-Driven Development and how it can help developers confidently write quality code. In Part 2 of this topic we’ll actually write and run a zUnit test case. Stay tuned.

References

[1] xUnit, available at http://en.wikipedia.org/wiki/XUnit
[2] JUnit, available at http://junit.sourceforge.net/
[3] Unit Testing, available at http://www.xprogramming.com/software.htm
[4] Simple Smalltalk Testing: With Patterns, available at http://www.xprogramming.com/testfram.htm
[5] Test-driven Development, available at http://en.wikipedia.org/wiki/Test-driven_development
[6] Guidelines for Test-driven Development, available at http://msdn.microsoft.com/en-us/library/aa730844.aspx
[7] Automation for the people: Continuous testing, available at http://www.ibm.com/developerworks/java/library/j-ap03137/index.html

About the author

Teodoro Cipresso is a lead developer on the Enterprise Service Tools and IBM z/OS Automated Unit Testing Framework (zUnit) features of IBM Rational Developer for System z. His day-to-day activities involve the design, implementation, and automated testing of these features as well as working directly with end-users to provide support and education. Ted has significant experience and continuous exposure to XML and WS-* standards and is proficient in multiple object-oriented and non object-oriented programming languages.