Collaborative Schema Design with IBM Rational ClearQuest

Best Practices to successfully speed up team-based Schema development

Speeding up Schema development often requires that two or more designers work on one Schema. The article describes proven Best Practices to help you manage multiple developers working on your ClearQuest Schema at the same time.

Share:

Christian Glockner (christian.glockner@nl.ibm.com), Staff Software Engineer, IBM

author photoChristian Glockner has been working as Technical Support Engineer with IBM Rational Client Support Europe for more than three years, assisting clients with the use of IBM Rational ClearQuest. Beyond that, he has been involved in several IBM-internal ClearQuest projects, one of which included planning, schema development in a team of two designers, and system deployment and maintenance. In addition, he also coaches colleagues on the Kepner-Tregoe Resolve methodology, which Rational Client Support adopted several years ago. In his spare time he likes to travel, listen to music, and participate in whatever is fun.



27 April 2010

Also available in Chinese Russian

Collaborative (or concurrent) work on an IBM® Rational® ClearQuest® schema, while possible, is not straightforward. However, in some situations, it is necessary for multiple designers to work on a schema at the same time in order to speed up development.

After describing some of the potential challenges you face when developing a ClearQuest schema with the help of several developers, this article introduces some best practices to help you avoid them. Finally, this article shows how to recover from any problems that might occur.

The best practices are based on experience gained during a large IBM-internal ClearQuest project, which involved developing a schema with two designers. These practices are also based on my observations during my job as Technical Support Engineer for ClearQuest at IBM Rational Client Support Europe. More than once have I seen clients run into issues of all sorts because they thought they had done everything correctly, working with multiple schema developers, but in fact they had not.

Although we could always resolve these situations, after reading and following the best practices outlined in this article, you should be able to prevent these problems from happening in the first place.

Serial vs. parallel schema development

Generally speaking, there are two methods for schema development in IBM® Rational® ClearQuest: serial schema development and parallel schema development.

Serial schema development is the standard way of designing a schema that ClearQuest offers. With this method, schema developers work on one schema, one after the other.

Figure 1. Serial schema development workflow
Developer 2 starts work when Developer 1 is done.

This model is simple to use because it is fully supported by the normal workflow in the ClearQuest Designer and it is easy to maintain because only one person is working on the schema at a time. However, the development process is relatively slow and inefficient, because Developer 2 can start working only after Developer 1 has finished his or her work.

Parallel schema development, on the other hand, is more complex to work with, but also much more flexible. Therefore, this is the model that will form the basis for this article.

Figure 2. Parallel schema development workflow
Developer 1 and Developer 2 work at the same time.

Figure 2 shows two developers working on a schema, but each is working on a separate copy of the schema, which allows them to work at the same time. The exact workflow is discussed in more detail later in this article.


ClearQuest tools to support parallel schema development

Although the ClearQuest Designer supports only serial schema development, ClearQuest provides two complementary subcommands that enable multiple developers to simultaneously work on a schema.

  • cqload exportintegration

    The exportintegrationsubcommand of the cqload command exports the changes made in one or more schema revisions to a file.
  • cqload importintegration

    The importintegration subcommand of the cqload command imports changes from a file into a schema and will replay those changes in the destination schema.

For details regarding the use of the cqload command, refer to the section Rational ClearQuest Command Reference in the IBM Rational ClearQuest Information Center.


Challenges

As described earlier, parallel schema development is more sophisticated than its serial counterpart, but it also requires more administration and planning. The most frequently encountered and the most severe problems that might occur during parallel schema development are:

  • Naming conflicts
  • Form overwrites
  • Missing or inconsistent package information
  • Dead links
  • Organizational problems

Naming conflicts or "The name XYZ is already in use"

When you import a schema, you may see the error message The name XYZ is already in use. This error indicates that you have a naming conflict. Typically, you are attempting to create (or rename) an object with a name that is identical to an already existing object. This is not permissible.

All objects of a given type must have unique names. These types include

  • Record types
  • Fields
  • States
  • Actions
  • Record scripts
  • Global hooks
  • Forms
  • Named lists

For example, two record types cannot both be named Defect, and two fields on one record type cannot both be named Headline. Of course you can have a field named Headline on the Defect record type, and another field also named Headline on the EnhancementRequest record type.

If you do not ensure that each object has a unique name, ClearQuest returns the name XYZ is already in use error message.

Form overwrites or "But I changed it!"

Another issue that might occur is that forms are overwritten when performing a cqload importintegration if the file being imported contains form changes.

You will usually know that you have run into this problem if you do not see the changes that a developer made to a form, or if you did see the changes previously but they are gone after you perform the next importintegration.

The reason behind this is that ClearQuest forms are Base64-encoded. That means that any change, even the smallest one, causes a new form definition to be written. In other words, even if you only move a button up one pixel and down one pixel, you still get an entire new form definition. The next time you export your changes using cqload exportintegration, the full form is exported along with all other changes you have made.

Assume that Developer 1 has changed the Submit form of the Defect record type. Developer 2 has accidentally moved a Button on the same Submit form of the Defect record type up and down a pixel. Both export their changes and send them to you.

As part of your workflow, you first import the changes that Developer 1 made. After the import you look at the Submit form of the Defect record type and you can see that Developer 1's changes have been made to the schema.

You then import the changes that Developer 2 made. Since no errors occurred, you look at the schema again and discover that Developer 1's changes are gone. You have just experienced a form overwrite.

Missing or inconsistent package information

Missing or inconsistent package information is probably the most difficult problem to solve. If you have this problem, do not try to fix it yourself, but instead contact IBM Rational Client Support for assistance.

The good news is that this problem is very easy to prevent.

In general terms there are two kinds of packages in ClearQuest.

  • Standalone packages

    These packages offer functionality by themselves, in other words you could apply them to a "Blank" schema and you would have a fully functional schema. One such example is the EnhancementRequest package, which adds a record type (among other things), but does not require any other record types to exist for it to be usable or useful.
  • Dependent packages

    These packages are not useful or usable by themselves, but add functionality to a record type. One such example is the Notes package, which adds fields and hooks to existing record types. Therefore, it is usable only if the record types are in the schema.

The main difference between those two types of packages is how they are installed. Both types of packages are installed with the Package Installation wizard. However, dependent packages also need to be added to one or more record types using the Setup Record Types for Packages functionality in the ClearQuest Designer.

This means that your schema can have the following problems with packages:

  • Your schema might be missing the information that a specific package has been installed
  • Your schema might be missing the information that a dependent package has been applied to a specific record type.

These problems generally occur with older versions of ClearQuest. Although the metadata (such as fields, record types, hooks) was exported correctly when running exportintegration, the information about which packages have been installed in the schema and applied to which record types was not imported at all.

Another potential problem is that different versions of a package might contain different metadata. For example, version 5.0 of the Notes package has different hooks than version 5.1 of the same package (the BASIC hooks have been removed and the Validation hook was added).

Assume that the Notes 5.0 package is applied to the main schema. A developer decides to upgrade his copy of the schema to Notes version 5.1.

As expected, ClearQuest removes the BASIC hooks and adds the new Validation hook that was added in version 5.1 of the package. After this Developer exports his or her changes, the changes made by the package upgrade are also exported and replayed in the main schema at the time of importing the changes.

ClearQuest does not detect that the Notes package was upgraded to 5.1, even though the metadata is . Notes 5.1. Your schema properties indicate that Notes 5.0 is installed, when in fact you have installed 5.1.

The problem is that you might not notice these problems very late, possibly months after you have performed the importintegration, because the schema still works fine. The problem becomes apparent only when you try to upgrade the packages (using the Upgrade Installed Packages functionality in the Designer) If the package information is missing from the schema, ClearQuest will not report any upgradeable packages. If your schema was inadvertently upgraded to Notes 5.1, the package upgrade will fail because ClearQuest will try to remove the BASIC hooks, which no longer exist because they were removed during the package upgrade in the development schema.

Once again, this is considered schema corruption and you should not attempt to correct this problem yourself, but instead let IBM Rational Client Support help you.

Dead links

Note:
In this article, "dead links " in ClearQuest schemas refer not only to REFERENCE or REFERENCE_LIST types of fields but also to any kind of reference from one object to another.

Dead links typically occurs if a reference is valid in the development schema but not in the main schema that you perform the importintegration against.

Dead links potentially affect any object that can refer to other objects in the schema in any way, shape or form, such as :

  • Hooks (GetEntity and other API calls)
  • Forms (form controls for nonexistent fields)
  • Fields (nonexistent named list selected as choice list)
  • Columns of list view controls, when the column refers to a field that does not exist
  • Actions (A Record Script used by a RECORD_SCRIPT_ALIAS action does not exist or an action is intended to transition a record into a destination State that is missing)

These problems are usually relatively easy to correct, but also easy to prevent.

Organizational problems or "Who did that?"

There is a saying, "Too many cooks spoil the broth," and you could probably find numerous examples that prove the point. Actually, organizational problems are not specific to ClearQuest schema development (or parallel schema development, for that matter), but nevertheless these types of problems are common in the schema development process, especially when using a collaborative approach.

A special challenge is the fact that the revision history of the development schemas is lost when performing the importintegration on the main schema. This is because ClearQuest imports all changes (which might have been done during multiple schema revisions) into a single revision in the destination schema.

Therefore, one question you might be asked is, "In which version of which development schema was change X made?"

Figure 3. Example of revision history being lost
Multiple Schema revisions are merged into one.

As you can see from Figure 3, the changes made in revisions 3 and 4 in Developer 1's development schema were combined and resulted in one revision (revision 3) in the Integration schema.

The example is a very simple one, and your situation is likely to be more complicated. Figure 4 shows an example with three developers.

Figure 4. Example of workflow with three developers
Three developers merging their Schemas

As you can see, revision 5 of the Integration schema equals version 2 of the Integration schema plus the changes made by Developer 3, plus the changes made by Developer 2, plus the changes made by Developer 1.

Generally, the more developers that you have, the more complex and difficult it becomes to keep everything together. also In addition, your environment most likely contains not just one Integration schema, but one Acceptance schema and one Integration schema—or an even more complex setup.

Now that you have seen some of the most frequently encountered and most serious challenges you face when developing ClearQuest schemas in parallel, you might conclude that you should not attempt parallel development because too much can go wrong. However, the best practices described in the following section will help you avoid errors.


Best practices

The following best practices enables a team of two or more developers to develop a ClearQuest schema without running into the challenges laid out above. Each practice that you implement increases the likelihood of problem-free schema development. The best practices are in the following areas:

  • Change management
  • Project planning and execution
  • Clear allocation of tasks, discipline
  • Intelligent package installation
  • Predefined name space
  • Version control for Development and Integration schemas
  • Integration tests performed in separate test environment

Change management

You may be wondering, "We are using ClearQuest to do our change management, so why should we be using change management to manage our use of ClearQuest?"

Good question, but the answer is simple: A schema is actually software. Your core business is very likely to be software development and you certainly would not hesitate a second to make the software you are developing subject to change management. (You would not be using ClearQuest otherwise.)

Why then not also make the software that powers the change management for your software subject to change management?

The first time that you cannot answer the question "In which version of which development schema was change X made?", you wish that you had used proper change management for your schema development activities.

The precise information to capture in your change management solution depends on your specific requirements. However, consider capturing the following information:

  • Which object type was changed? (for example, record type, field, action, hook, and so on)
  • Which specific object was changed? (for example, defect, enhancement request, and so on)
  • Who made the change?
  • What exactly was changed?
  • Why was the change made?
  • Which development schema was the change made in?
  • Which revision of the development schema was change made in?
  • Which revision of the Integration schema contains the change?

Capturing this information will allow you to track back any change that affected the Integration schema (which may very well be your production schema) all the way to the developer who made the change and the development schema that contains the change.

Project planning and execution

Project planning is a best practice that is not specific to parallel ClearQuest schema design. However it is especially useful when working in teams.

During the planning phase you should decide on several things, such as requirements (that is, What is the schema supposed to provide?), resources (How many and which developers are available to work on the schema?), and your schedule (When does the schema need to be ready?).

In addition to addressing these concerns, you will also need to consider a few points that are specific to ClearQuest schema development,

  • Which packages are needed to meet the requirements?
  • What are the milestones after the importintegration is run?
  • Which hook language will be used?
  • Which objects depend on which other objects? This information you can avoid dead links.)

At this point, you probably cannot predict exactly which packages you will need in the long run. However, based on your requirements, determine the packages that you know that you will need. Notes, Email, and AuditTrail are good candidates.

You may have realized that the IBM Rational toolset provides the means to manage this process for you, so you can re-use the tools you already have or start using the Requirements Management tools that IBM provides.

Once the planning phase is over, you need to be sure how to execute the plan. The following workflow has been found to work very well.

Figure 5. Creating a copy of the schema for every developer
Creating copies of the Schema for each developer.

In the first step, create a copy of your main schema for every developer. You can either export the schema with the cqload exportschema command or copy the entire schema repository and provide that to your developers.

Figure 6. Importing developer 1's changes into the integration schema
Developer 1 merges changes into the main Schema

In the second step, integrate Developer 1's changes into your Integration schema. Figure 6 shows that Developer 1 started with revision 2 of the schema and created two new revisions before exporting his or her changes.

Figure 7. Importing Developer 2's changes into the integration schema
Developer 2 merges changes into the main Schema

In the third step, import Developer 2's changes into the Integration schema. You see that Developer 2 has only checked in once, exporting their schema revision 3 to the Integration schema (see Figure 7).

After both developers' changes have been imported into the schema repository, install the packages that you need (see Figure 8).

Figure 8. Installing packages
Packages are installed after merges are complete

Revision 5 of the Integration schema contains all changes from both developers and the newly installed packages. It is therefore a good opportunity to use this as a new baseline to start developing from (see Figure 9).

Figure 9. Creating a copy of the new baseline schema
A new baseline is created from the main Schema

After you create the baseline, the process starts again and repeats until the development work is complete.

Clear allocation of tasks, discipline

It is important to understand that record types are atomic. They cannot be split up any further as far as development work is concerned. Consequently, one and only one developer works at a record type at any given time. A developer may, of course, be working on multiple record types at the same time.

Second, you must make sure that no-one modifies Metadata that belongs to someone else. For example, do not change the name of a record type that you do not own. But this rule goes even further: When you create a new REFERENCE or REFERENCE_LIST field on one of your record types, you have the option to automatically create a back-reference. Do not do create a back reference if you do not own the record type that the field is referring to, because this violates this best practice—in effect, the back-reference field is added to a record type you do not own.

Finally, the Integration schema is off-limits. Even if it is tempting to make a minor change on the Integration schema, the impact on future exports and imports is unforeseeable. Therefore, make all changes, as small as they might be, in one of the development schemas, and then export and import those changes into the Integration schema. (Packages are an exception, which is described in the following section.)

Intelligent package installation

As you have seen, many things can go wrong as far as packages are concerned when developing your schema in teams or two or more designers. However you can prevent these problems by being careful about when and how you install packages.

If you have thoroughly planned your project, you will have a good idea of which packages you will need to meet some of the requirements. It is crucial to know this before starting to work with multiple developers. Once you identify the packages,, install them as early in the project as possible – you will need them anyway, so there is no need to delay this task.

Although the Integration schema is off-limits for any schema changes, this rule does not apply to package-related tasks. In fact, the Integration schema is the only place where you may install or upgrade packages or enable record types for packages.

If developers accidentally install or upgrade a package in their development schemas, they will have to remove the package. However, because packages cannot be removed, the developers are forced to delete the revision in which the package was installed or upgraded plus all later revisions. Since this can cause a large amount of work to be lost, make sure your developers follow to the rule of not making package-related modifications in their own schemas.

Predefined name space

A good practice to avoid name conflicts is to define your own name space before starting to work on the project. Agree on the names of record types, fields, global scripts, and so on, so that every project member knows which names to use. If you need to change a name after you have started work on your schema, make sure that everyone is aware of that change.

You may also want to use a company- or project-specific prefix with your schema object names. For example, instead of creating a record type named Customer, you might want to call it IBMCustomer. In the ALM package, object names are prefixed with ALM, which helps avoid name collisions when installing the package into a schema where a record type named Request already exists, but ALMRequest does not.

If you use prefixes, be aware that the maximum length of object names in ClearQuest is 25 characters. InternationalBusinessMachinesCustomer would be a bit too long, so stick to IBMCustomer instead.

Version control for development and integration schemas

Again you might wonder why you should put your development and Integration schemas under version control. After all ClearQuest already manages revisions of schemas for you.

And once again the answer is simple: A schema is also just software. If you put the software that you are developing using ClearQuest under version control (which you are likely to do with IBM® Rational® ClearCase®), then you should also put the software that powers the software you are using in your development under version control.

This ties in with the change management that you are already using with your schema, and it also provides you with another backup copy of your development and Integration schemas.

To get the best results, export every development schema using the cqload exportschema command after every schema check-in. Then check in the resulting file into your version control system, such as ClearCase. You should do the same with your Integration schema every time you have run the importintegration successfully.

Integration tests performed in separate test environment

It probably goes without saying that you should first test every importintegration in a separate test environment before actually using it with your production schema.

Using a separate test environment is also a good opportunity to see if the imports of the changes from the development schemas will actually work in the order you were planning to import them or not. If they fail, you can revise the order until the import of all changes is correct. You will now be able to perform the imports in your production environment in an order that you know will work.

Summary

Now that you are aware of the best practices to use, look at the following table to see how each best practice helps you avoid specific issues.

Table 1. Issues that can occur and the best practices to avoid them
IssueBest practice
Naming conflictsPredefined name space
Clear allocation of tasks, discipline
Project planning and execution
Form overwritesClear allocation of tasks, discipline
Missing or inconsistent package informationIntelligent package installation
Discipline
Dead linksProject planning and execution
Change management
Clear allocation of tasks, discipline
Organizational problemsChange management
Clear allocation of tasks, discipline
Project planning and execution

What if?

Even though you follow these best practices, you might still encounter problems. To resolve them, take the following steps:

  • Think
  • Look closely at the error message
  • Try the integration the other way around
  • Contact Rational Client Support

Think

"Think" is IBM's oldest motto, and it is still a very powerful one.

If you have managed your changes correctly, you know which changes are contained in the files that you are trying to import. Taking this into account, ask yourself:

  • Do the objects that are supposed to be altered or deleted actually exist?
  • Do the objects that are supposed to be created exist already?

Look closely at the error message

Error messages that you encounter while trying to import changes are typically try to explain the problem. Consider the following examples of error messages:

  • The name "Defect" is not a valid entitydef
    This error message tells you that the record type Defect does not exist.
  • There is a Reference to an object that does not exist. Object Type: Field. Object Name: Test
    This error message tells you that something is referring to the nonexistent Field Test.
  • The name Defect_Base is already in use.
    This message tells you that there already is an object with the name Defect_Base.

Your corrective actions will vary based on the error message, but generally the message gives you a good idea of what is wrong.

Try the integration the other way around

At some point in your workflow, your developers will send you files containing the changes they made to the schema, created using the cqload exportintegration command.

The changes made in one file that you are trying to import into your Integration schema could depend on the changes made in a subsequent file.

Maybe the objects that export file from developer 1 is trying to change are created only in the export file from developer 2. In that case, import file 2 first, followed by file 1.

Alternatively, maybe the objects that export file 2 is trying to create are deleted only in export file 1. To resolve this situation, import file 1 first and then import file 2.

The more files you have, the more complex the reordering will be, but good change management and allocation of tasks will help you tackle such situations. Also, running the integration tests in your test environment first will allow you to determine the correct order for importing your changes.

Contact Rational Client Support

It is good to know that you can ask someone for help if all else fails. If you cannot resolve the issues you experience when you import changes, ask the specialists in IBM Rational Client Support for assistance.

The best way to submit your service request is via the IBM Support Portal. Click "Open a Service Request" on the bottom right for the IBM Support Portal homepage to submit your service request.

When you submit your request, provide the following information to the specialist:

  • The version of ClearQuest you are using
  • The list of commands that you have executed (cqload exportintegration/importintegration)
  • The exact error message, possibly screenshots and log files
  • An export of the destination schema or a backup of the database that holds the schema repository
  • All export files created with cqload exportintegration that you are trying to import
  • A description of the order of importing the export files into the destination schema

Conclusion

As you have seen, working in parallel on ClearQuest schemas involves careful planning and communication to avoid challenges. However, following the best practices in this article greatly increases the chance for first-time success of your project. If, for whatever reason, something goes wrong in spite of all preparations, you still have a good chance of fixing the error.

And if you cannot fix the error yourself, you can rely on IBM Rational Client Support to help you.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=485180
ArticleTitle=Collaborative Schema Design with IBM Rational ClearQuest
publish-date=04272010