 | Level: Introductory Michael Kelly, Software Engineer, QA, Liberty Mutual
24 Nov 2003 This article is a step-by-step introduction to using TestManager and RequisitePro to store your artifacts for future use, build traceablity into your test cases, and ultimately reduce the time it takes for test reviews.
The process of getting test plans reviewed can either be a simple and straightforward task or a time-wasting, hair-pulling nightmare, depending on how your automated-testing team sets it up. Trust me -- I learned this from bitter experience. And I'm here to tell you that your life will be 100% easier if you build test plans around the people who'll be reviewing them. Rational gives you the tools to do this, and it's well worth your time to learn how to use them. This article is a step-by-step introduction to using TestManager and RequisitePro to store your artifacts for future use, build traceablity into your test cases, and ultimately reduce the time it takes for test reviews.
How Not to Handle Test Plan Reviews: A Cautionary Tale
As a shining example of how not to handle test plan reviews, let me tell you about an iteration of a project I was working on once that involved some minor user-requested code changes and optimizations to an existing system. The time to gather requirements was minimal as most of them were documented in change requests, and the development time was minimal as it was mostly GUI changes that were called for, with a few new rules being applied to the database. In the greater scheme of things this iteration should have been a breath of fresh air to everyone involved with the project. It was short, simple, and very uncomplicated.
Following a customized version of the Rational Unified Process (something that we were growing into at the time), the project team was using ClearQuest for defect tracking, RequisitePro for requirements, and the TestStudio tools to varying degrees. I was the lead for the automated-testing team at the time, and as such I attended the project meetings, assigned the automated-testing resources, and assisted in the requirements, design, and test plan reviews. I had a front-row seat to witness what should have been an inexpensive one-month iteration stretch out into a grossly over-budget three-month nightmare.
Now as always, there were other things happening within the company that contributed to some of the project delay (staff turnover, company politics, and such). But the main reason for the poor use of time and the costly waste of human resources was the way in which we were performing our test reviews. While the requirements staff and the development staff for the project were leveraging their tools correctly, the test team hadn't allocated time to effectively learn the Rational test tools, and consequently we were still using a Word numbering scheme to trace the test cases back to the requirements in RequisitePro. If you've ever done this, you know how tedious this can get when there are a lot of test cases.
Normally, for a project this size there would only be a handful of test cases to create as well as some modifications to make to some existing test cases. But due to poor test management and project management on previous iterations, there were no existing test cases to work from, so our team created robust test cases from scratch and started to document the system. We created about four or five test plans, each with about fifty associated test cases. This was overkill for the iteration, but we all knew it had to be done at some point in the project and we figured it might as well be now.
As our test plans and test cases started to come closer to completion, we started to share them with the business unit. They were the ones who would check them against their requirements and verify the test coverage. Due to the large number of requirements referenced (as we were including requirements that should have been tested in previous iterations) and the number of test cases, the business people -- not surprisingly -- got lost. They passed the test plans back to us and asked us to change the format to make things "more readable." We reengineered the test plans and cases to a new format and sent them back for review. Again the business unit found them hard to follow and asked for another format.
At this point the project manager stepped in and took control of the situation. He informed the business unit that there would be no more format changes, and he informed us that there would be no more additional test coverage. The rest of the time was devoted to us walking the business people through the test plans and cases and making necessary corrections and changes. This whole process took several iterations and some number of weeks.
TestManager to the Rescue
Maybe you've experienced something similar to what I just described, or maybe you saw the writing on the wall and never took this route. But I'm young and prone to learning from mistakes rather than figuring things out ahead of time. Here's what I learned from my unfortunate experience:
- Don't expect your business staff to think like the IT staff.
- Communicate with your business staff early in the process.
- Develop a consistent method of documenting and delivering artifacts.
- Store artifacts in a common location where they can be found for future iterations of the current project or future maintenance of a finished project.
Little did I know that the RUP and the Rational tools address all of these issues (like I mentioned above, we were growing into the process). After the fact, we started to look more deeply into some of the features of the tools, as well as the practices outlined in the RUP. The RUP addresses the issues we had with communication and time management, and TestManager provides a way to document, store, and deliver artifacts, and to build traceability into your test cases.
I'm going to take you step-by-step through the process of using Test Manager to expedite test plan reviews. First, though, I would advise you to get your whole IT staff involved. This can't be done unless everyone is behind it, especially your requirements team and your project management. You need good requirements and good inputs from RequisitePro for successful traceablity. Your project manager should understand how the review process is done and should work with you in getting your reviewers to try it.
Here's the process in a nutshell, then:
- Set up your requirements in RequisitePro.
- Set up the test inputs for the project.
- Create a test plan.
- Create folders to organize your test cases.
- Create your test cases.
- Establish traceability from your test inputs to your test cases.
- Set up your test cases so they actually test something.
Now I'll take you through each step in detail.
Set up your requirements in RequisitePro.
It's important to correctly set up the requirements for the project in RequisitePro. For the purpose of this article, I used the requirements in the Rational Learning Project (included in versions 2001A and 2002). If you have questions about how to set up your requirements in RequisitePro, you should speak with the person responsible for the requirements for your project, look over the Rational documentation, or go through the Rational Suite Tutorial. Another good place to look for answers to questions about using RequisitePro is on the Rational Developers Network; see Development Resources > Rational Tools > RequisitePro and Discussions > Public Forums > Requirements Management and RequisitePro.
Set up the test inputs for the project.
Test inputs are nothing more than the things that you'll use to plan your testing for the project. They can include but aren't limited to prototypes, functional specifications, requirements, visual models, source code files, and change requests. When you're planning your testing, you might look at any of these artifacts to help you decide what exactly needs to be tested and how to test it.
Selecting the test inputs you'll use is the first step to establishing traceability in your project, and it's that traceability that will allow for faster review times. For the purpose of this article, we'll be looking only at requirements from RequisitePro as test inputs, but bear in mind that you should choose your inputs based on the details of testing and the people who'll be reviewing the test plans. If you know you'll have developers reviewing the test plans, include UML diagrams and code as inputs. If you know that the reviewers won't be that technically inclined, it may be best to find higher-level inputs. It's important to tailor your inputs to your review audience.
To add a test input to TestManager, proceed as follows:
- Open TestManager.
- From the menu bar, choose Tools > Manage > Test Input Types...
| |
Figure 1
|
- Select "Rational RequisitePro" and click Edit.
| |
Figure 2
|
- If your project is already listed here, click Cancel and then Close. If your project isn't listed, you must add it as an input. Click the Sources tab and click Insert...
| |
Figure 3
|
- In the New Test Input Source window, enter the name of your input and give a description. Click the Browse... button and navigate to the location where the .RQS file for your requirements is stored.
| |
Figure 4
|
- Click OK, and you should see the input you just added under "Sources."
- Click OK again.
- Once returned to the Manage Test Input Types window, click Close.
Create a test plan.
Now that you have inputs, you're ready to create your first test plan. For this example I'll be creating only one test plan, but you can create as many as your project needs. Try to separate them as much as possible into groups that best fit your target reviewers. For example, if you know that your development staff as well as some of the business staff will review some of the test plans, make a more detailed, technical test plan for your developers and a higher-level, more requirements-based one for your business people. Use UML models, code, and design documents as inputs for the technical one, and business-case specifications, use cases, supplementary requirements, and change requests as inputs for the other. The important thing is that you find what works best for you and the people who'll be reviewing the test plans.
To add a test plan, perform the following steps:
- On the left side of the screen, right-click the Test Plans folder and choose New Test Plan...
| |
Figure 5
|
- On the General tab, enter a name and a description for the test plan. Then click OK.
| |
Figure 6
|
Create folders to organize your test cases.
Within your test plan, you'll want to sort the test cases into logical categories using folders. These folders will allow you to better organize your test cases to fit your audience. Again, try to think of who will be doing the review and tailor your test-case organization to meet their needs. Often it's a good practice to have your folders reflect the structure of the test inputs. This could mean nested folders or just broad topics. Figure 1 is an example of how they could be arranged.
| |
Figure 7: An example of using folders to organize test cases
|
To create a test case folder, perform the following steps in TestManager:
- Double-click the newly created Sample Test Plan One file under the Test Plans folder in the tree on the left. This should bring up a separate window for the test plan.
- Right-click the test plan in the Test Plan window. Choose Insert Test Case Folder... from the menu.
| |
Figure 9
|
- In the New Test Case Folder window on the General tab, enter a name and a description for the folder. Then click OK.
| |
Figure 10
|
Create your test cases.
Now we'll look at adding a test case. Keep in mind that there are dozens of ways to do this in TestManager. It's only important that you know the basics for now; as you get more comfortable with creating test plans in TestManager, the other ways of working within the tool will become apparent.
To add a test case in TestManager, do the following:
- Right-click the test case folder you want to add the test case to and choose Insert Test Case...
| |
Figure 11
|
- In the New Test Case window on the General tab, enter a name and a description for the test case. Then click OK.
| |
Figure 12
|
Establish traceability from your test inputs to your test cases.
At this point you have a test case in your test plan, but it has no traceability and it doesn't test anything. That's what all those other tabs in the New Test Case window are for. We'll now look at establishing traceability from the test inputs to the test case. This is the most important step for expediting test plan reviews because it's what enables the reviewer to quickly identify what specifically a test case is testing -- or vice versa, to look at a requirement and see where it's tested.
To add traceability to a test case, do the following:
- Right-click the test case and choose Associate Test Input... to display the Test Input Selection window. Note that clicking the Test Inputs tab and then the Add... button when you were creating the test case would also have done this.
| |
Figure 13
|
- Expand the Test Input Sources until you can see the source that you want to associate this test case with. Select that source and click OK. (If you want to associate more than one test input with the test case, hold down the Ctrl key and select all the inputs you want to associate.)
- If at any point you want to remove any test inputs or verify which test inputs are associated with a test case, you can do so by right-clicking that test case and choosing Properties, then clicking the Test Inputs tab. This will show all the associated test inputs and allow you to add or remove any from the list.
Set up your test cases so they actually test something.
You should now have your test inputs set up and have created at least one test plan, complete with test case folder(s) and at least one test case, which is linked to at least one test input. Now all you have to do is set up your test case(s) to actually test something (preferably the test input you linked it to) and you're ready to have your test plan reviewed!
To set up a test case in TestManager, you can choose from these options:
- Attach an automated script to the test case
- Attach a Rational ManualTest script to the test case
- Attach an external document (Word, Excel, Word Perfect, or such) to the test case
There are probably more options that I don't know of besides these. In any case, I'll show you the details of these three options, and you can figure out other ways as you become more familiar with the tools.
Attaching an automated script to a test case
If you're like me and love to automate everything you're in luck, because attaching an automated script to a test case is the most painless of the three options. (I know what you're thinking -- you can use this as an excuse to automate more.) Here are the steps:
- In TestManager, right-click the test case and choose Properties...
| |
Figure 15
|
- Click the Implementation tab in the Test Case Properties window. This tab is broken up into three sections: "Manual implementation," "Automated implementation," and "Preconditions," "Post-conditions," and "Acceptance criteria."
| |
Figure 16
|
- Click the Select button under "Automated implementation." This opens up a pop-up window listing the types of artifacts you can attach to the test case. For the purpose of this example I'll show how to attach a GUI script, but feel free to attach whatever artifact you have available and you feel is appropriate. With the exception of Command Line, any of the other options will bring up a selection window like the one shown below.
Select the script that you want to associate with the test case and click OK. You should be returned to the Test Case Properties window, and the script you selected should be visible in the "Automated implementation" field. If there are any preconditions, post-conditions, or acceptance criteria that need to be documented, take the time to do that now. When you've finished, click OK and you'll be returned to the Test Plan window in TestManager.
Now take a close look at your test case (the one you just attached the automated script to). You should see a little gear icon to the left. That means it has an associated automated script.
| |
Figure 18
|
Attaching a manual script to a test case
For those test cases that require a manual approach or just aren't good candidates for automation, you'll enjoy the comfort of knowing that you can also attach a Rational ManualTest test script. Furthermore, not only can you attach this kind of script, you can also create it on the fly as you create your test cases in your test plan. I'll tell you how to do both.
- To create a ManualTest test script from scratch and then attach it to the test case in your test plan, right-click the test case and choose Design... This will display the Design Editor (the same Design Editor that Rational ManualTest uses) window. Simply create your test case and when finished click OK.
- You should be returned to the Test Plan window in TestManager. Right-click the test case and choose Properties...
| |
Figure 20
|
Click the Implementation tab in the Test Case Properties window. To associate the ManualTest test case you just created, click the button labeled Import from Test Case Design... This will bring up a prompt for you to name the ManualTest test script. Once you give it a name it will be saved to the test data store and associated with this test case. If your ManualTest test script already exists, simply click the Select... button, select the correct test case in the list, and click OK.
At this point regardless of which of the above paths you took, you should see the test case listed in the "Manual implementation" field.
| |
Figure 21
|
- Click OK and you'll be returned to the Test Plan window in TestManager.
Now again take a close look at your test case (the one you just attached the ManualTest test script to). You should see a little hand icon to the left. That means it has an associated manual test script.
| |
Figure 22
|
Attaching an external document to a test case
Last but not least, an award should go to Rational for giving all of us who can't seem to convince the rest of the company to live in the Rational Tools like we do the ability to add external documents. If you have your test cases out there somewhere in the nebula we call a network, you can attach them (in whatever format they may be) to your test case inside of your test plan. Not only that -- you can also open them with the click of a button from inside of TestManager. Now this is starting to sound like a commercial, so I'll just show you how to do it and you can judge it for yourself.
- One last time, right-click your test case and choose Properties...
| |
Figure 23
|
- This time click the External Documents tab in the Test Case Properties window.
| |
Figure 24
|
- Click the Add... button. This will display a standard navigation dialog. Navigate to the file you want to attach and click Open. You should see the file listed in the "Associated documents" list.
| |
Figure 25
|
- If you want to add more, simply repeat the last step as many times as necessary. If you want to view the document, click Open and take a look. When you've finished, click OK and you'll be taken back to your test plan.
The Proof of the Pudding: Review Time
Skip ahead a week or two. You and your team have been adding all of your test plans and test cases to the project database via TestManager. You think you're ready for your first review. So how does this review thing work now that everything's organized on the computer?
Instead of sitting at a table with five stacks of paper (formerly two medium-sized trees), your reviewers can look over your test plans from their desktops (TestManager is now included in every Rational Suite for just that reason). Or if you still prefer the roundtable format (sometimes it's nice for everyone to get together and talk about things), simply grab a projector and go. The process of reviewing will consist of outlining the requirements, looking at what should be covered in the test plans, and tracing through the test inputs to see what's tested and where it's tested. If anyone wants to see the script or test-case document that actually does the testing, all you have to do is double-click.
Now, you may be thinking that I just skimmed over an awful lot of important details when I said "outlining the requirements, looking at what should be covered in the test plans, and tracing through the test inputs to see what's tested and where it's tested." Well, you're right. I painted how that process works with the broadest possible brushstrokes, partly because it can be done a lot of different ways and I can't really go into all of them in this article. You can use RequisitePro or TestManager, or a combination of both. You can walk through the GUI or use reports.
Take a look at some of the reporting capabilities available in TestManager. They're all pretty intuitive and easy to use out of the box. You can also create reports in RequisitePro, SoDA, or Crystal Reports (which comes with the suite and can be used to generate new reports should the out-of-the-box reports not meet your particular needs). And you can always search the Rational Developer Network and refer to some of the examples that come in the tutorial for ideas on how to generate reports.
Parting Words
I'll leave you with a couple of pieces of advice. First, don't do away with your old paper system all in one blow. Instead, start small and phase in your use of TestManager gradually. Second, always keep in mind that you need to build the test plans around the people who will be reviewing them. If you do this, you'll be a hundred times better off regardless of whether you even open TestManager once. Good luck!
About the author | | | Mike Kelly is currently a software testing consultant for the Computer Horizons Corporation in Indianapolis. He's had experience managing a software automation testing team and has been working with the Rational tools since 1999. His primary areas of interest are software development lifecycles, software test automation, and project management. Mike can be reached by e-mail. |
Rate this page
| |
