Adaptive workflow in ClearQuest (or: How to create a dynamic state machine)

In ClearQuest the workflow defined by the state transition matrix is a static process - user interaction causes the transition of a record from one state to another. In his latest article, ClearQuest guru Shmuel Bashan describes how to create a dynamic state machine in which workflow can vary depending on different conditions.

Share:

Shmuel Bashan, SMB IT Architect, IBM, Software Group

The author is an employee at IBM.



06 July 2004

Introduction

In IBM® Rational® ClearQuest® the state transition matrix defines the core workflow of a record-type. Transition from one state to another is done by actions performed by user selection. It means that the workflow is static. In many cases a dynamic state machine is required, so that the flow will vary depending on different conditions.

We shall demonstrate a technique that will enable a user (having the appropriate role) to change the default workflow for each and every record.


Background

In ClearQuest you may have several types of change requests, which are called record types. Usually they will be one of the following: issue, defect, enhancement or feature (although many other record types may be created). A company may want to have only one record type to deal with these types of requests (and there are many good reasons why to do it this way), but a static state transition matrix limits the implementation. Let's see some simple examples:

  1. For new features and enhancements you would probably want a Change Control Board (CCB) to decide whether to implement them or not, but for a bug fix you do not need the CCB to make this decision, and the flow can be shorter and faster.
  2. In small projects, testers are often familiar with the development teams; they know who the developer is that deals with the defective component they have just tested, so there is no need to go through some states to assign the defect to that developer; the tester can select the assignee during the submit action.
  3. For features that pose high risk to the project, the project manager may want to have extra actions like design review and code review. For simple features these actions may be omitted.

Each extra action is, in the best case, a waste of time for the user performing the action. A more severe consequence is when the issue is delayed for long time, or even forgotten. In the worst case, someone will make a mistake and will move the issue to the wrong state causing unnecessary delays to several users, and inefficient work of the group.


Definitions

Issue: A synonym to Change Request
The term issue defines a general change request. In ClearQuest that is a record type. Its name is usually "Issue", "CR", "SR" or some other name, according to the needs of the organization.

In most cases this record type will include a field called something like "Issue_Type" that will have values like: Bug, New_Feature, Field_Request, Enhancement and so forth.

Workflow: The Rational Unified Process (RUP) definition of a workflow is a sequence of activities that produces a result of observable value.

A workflow defines:

  • The activities
  • The roles that perform those activities
  • The states which the artifacts may go through (in our case the artifacts are issues)
  • The interactions between roles.

In ClearQuest terms, the workflow consists of the state transition matrix, the set of actions, and action hooks that define the rules (especially the Access_Control hook).

Role: Set of activities to be performed by some users.

The RUP defines roles as follows.

  • A role is an abstract definition of a set of activities performed and artifacts owned.
  • Roles are typically realized by an individual, or a set of individuals working together as a team
  • Roles are not individuals; instead, they describe how individuals behave in the business and what responsibilities these individuals have.

Adaptive workflow

In this section we shall first explain the programming technique, and show three examples in which each one has higher complexity level.


The technique

In ClearQuest, performing an action of the type 'change_state' will always move the record to a specific single state, as defined in the destination state of the action dialog box, shown in Figure 1.

Figure 1. The dialog box for the Assign action shows the defined destination state
The dialog box for the Assign action shows the defined destination state

Let's say that we want to use a hook to move a record to another state, one that is defined in the state transition matrix.

To change the record state with the ClearQuest API one must use the session method EditEntity(). Beginners sometimes try to change the state using SetFieldValue("State", "NewState"), but this call is forbidden and will not work. Also, trying to call another action in a validation hook or a commit hook will result in failure. When the notification hook is fired, the record is already committed to the database and no changes to the record are possible. So it seems as if there is no way to move to another state other then the current action destination state.

However, the fact that the notification hook is executed after the record is committed to the database, and the fact that it is possible to open a new session in that hook, enables us to perform the required action in this new session. So the method of adaptive workflow is based on this idea.

NOTE: It is important to mention that the new session logon has some security risk. See section 6.1 in the appendix for more information.

Automatically move to another state

You are probably familiar with the predefined schema in ClearQuest. In all of them the initial state is "Submitted" and no "Owner" is assigned. The default action in this state is "Assign" which will move the record to the "Assigned" state and will require you to fill the fields "Owner" and "Priority".

In many cases, the submitter can fill the fields "Owner" and "Priority", and would like to move the record to the state "Assigned" automatically.

The only way to do this is by performing the action "Assign" in a new session in the notification hook.

			Sub Defect_Notification(actionname, actiontype)
' actionname As String
' actiontype As Long
' action is Submit
' record type name is Defect
    
' - Purpose: Change defect state to "Assign" if 
owner&priority were set on submit
' - Created by SB on 11/11/2003
' - Date modified:

Dim newses , EntityObj
' - check if both owner and priority have values
if ( "" <> GetFieldValue("Owner").GetValue AND   _
      "" <> GetFieldValue("Priority").GetValue ) then
		      ThisID = GetDisplayName 
' - create new session and log as user (See security note 
in the appendix)
     set newses = CreateObject("CLEARQUEST.SESSION")
' - parameters for next line should be modified per a 
specific database
     newses.UserLogon "user", "pass", "db_name", 
AD_PRIVATE_SESSION, "db_set"
' - Get, Edit and Validate the record 
     set EntityObj = newses.GetEntity("defect", ThisID)
     newses.EditEntity EntityObj , "Assign"
     status = EntityObj.Validate 
     if (status <> "") then 
        newses.outputdebugstring "SB>>>: Fail validating in 
action assign: " & _ 
                                                      
status & vbCrLf 
        EntityObj.Revert
    else
        newses.outputdebugstring "SB>>>: Success validating 
in action assign: " & _ 

status & vbCrLf 
        EntityObj.Commit 
  end if  ' pass validation
end  if  ' auto assign

End Sub

One record-type having several state machines for each issue type

Now let's take a step forward and see a more complicated scenario. The ClearQuest schema has one record type called Issue, which deals both with defects, enhancements and features. It is desired to include more states when the type of the issue is enhancement or feature. For the sake of the example let's assume that it is required to go through the additional states: Design_Review and Code_Review.

How do we change the workflow to force the transitions to the above-mentioned states?

Figure 2. Portion of a workflow diagram
Portion of a workflow diagram

Let's consider the above diagram as a portion of the workflow of the state machine. From the state Opened it is possible to move to the Resolved state using the action Resolve, or to the CodeReviewed state with the action CodeReview. Some organizations may leave this decision to the developer, trusting his/her judgment or their familiarity with the company methodology. However we propose to automate it.

As mentioned earlier if the Issue is a defect, code review is not necessary, but for new features, code review is obligatory.

So when the developer has finished his coding and tested it, he should move to the next state, and to do this he will select a special action we called "Next".

We have added an action to the Issue record-type; its type is modify and its name is "Next".

The notification hook of this action will determine the required final state, thus the required action, of type change-state, and will activate it in another session.

Let's see a code example to perform the required change state. Notice the Case "Opened" section for the implementation of the above mentioned section.

Sub Issue_Notification(actionname, actiontype)
' actionname As String
'  actiontype As Long
' action is Next
' record type name is Issue
    
' - Purpose: Change issue state to the required state
' - Created by SB on 01/12/2003
' - Date modified:

Dim newses , EntityObj
' - check the current state
Cur_state = GetFieldValue("State").GetValue
Issue_Type = GetFieldValue("Issue_Type").GetValue
ThisID = GetDisplayName 
' - create new session and log as user (See security note in 
the appendix)
     set newses = CreateObject("CLEARQUEST.SESSION")
     newses.UserLogon "user", "pass", "db_name", 
AD_PRIVATE_SESSION, "db_set"

' - find the required action
  Select Case Cur_state
  Case "Submitted"
		NextAction = "Analize"
  Case "Analyzed"
		NextAction = "Assign"
  Case "Assigned"
	If  ( Issue_Type = "Defect" ) then
		NextAction = "Open"
	Else
		NextAction = "DesignReview"
	End if		
  Case "DesignReview"
		NextAction = "Open"
  Case "Opened"
	If  ( Issue_Type = "Defect" ) then
		NextAction = "Resolve"
	Else
		NextAction = "CodeReview"
	End if		
  Case "CodeReviewed"
		NextAction = "Resolve"
  Case "Resolved"
		NextAction = "Verify"
  Case "Tested"
		NextAction = "Close"
  Case "Closed"
End Select



' - Get, Edit and Validate the record 
set EntityObj = newses.GetEntity("Issue", ThisID)
' - We now use the action previously selected 
     newses.EditEntity EntityObj , NextAction
     status = EntityObj.Validate 
     if (status <> "") then 
                newses.outputdebugstring "SB>>>: Fail validating 
action " & NextAction  & _ 
                                                      ", the 
error is: " & status & vbCrLf 
        EntityObj.Revert
    else
        newses.outputdebugstring "SB>>>: Success validating 
action " & NextAction  & _ 
                                                      ", the 
error is: " & status & vbCrLf 
        EntityObj.Commit 
  end if  

End Sub

A state machine for each Issue (private flow for record)

Now let's see a more complicated scenario. The flow rules will be set manually by the CCB (or other authority). Consider a new feature arriving to the CCB. The CCB may reject, postpone or accept the feature. If accepted, the CCB requires that the feature pass through some additional states, based on the risk it poses to the project (There may be other reasons to set different workflows, for example, the stage of the project.)

The solution we propose is as follows:
In the schema we include fields to define if a state is mandatory. For example, if the field Req_CodeReview has the value "Yes" the state CodeReviewed is mandatory. A person with hub role (or other authority) will fill these additional fields. Figure 3 shows an example of how to select the required states in a tab form called "Navigation Flow":

Figure 3. Selecting required states for the transition
Selecting required states for the transition

As in the previous example, we define an action of type modify (let's call it "Next_Action" or just "Next"). The notification hook of this action will open a new session and will perform the required action that changes the state according to the rules previously set on the form.

See section 6.2 in the appendix for more information on hiding actions.

The code for this hook is:

Sub Issue_Notification(actionname, actiontype)
' actionname As String
' actiontype As Long
' action is Next
' record type name is Issue
    
' - Purpose: Change issue state to the required state
' - Created by: SB on 12/12/2003
' - Date modified:

Dim newses , EntityObj
' - check the current state and the type of the issue
Cur_state = GetFieldValue("State").GetValue
Issue_Type = GetFieldValue("Issue_Type").GetValue

ThisID = GetDisplayName 
' - Create new session and log as user (See security note in 
the appendix)
     set newses = CreateObject("CLEARQUEST.SESSION")
     newses.UserLogon "user", "pass", "db_name", 
AD_PRIVATE_SESSION, "db_set"

' - find the required action
  Select Case Cur_state
  Case "Submitted"
	If  (GetFieldValue("Req_CCB").GetValue = "Yes" ) then
		NextAction = "toCCB"
	Else
		NextAction = "Analize"
	End if		
  
  Case "Analyzed"
		NextAction = "Assign"
  Case "Assigned"
	If  (GetFieldValue("Req_DesignReview").GetValue = "Yes" ) 
then
		NextAction = "DesignReview"
	Else
		NextAction = "Open"
	End if		
  Case "DesignReview"
		NextAction = "Open"
  Case "Opened"
	If  (GetFieldValue("Req_CodeReview").GetValue = "Yes" ) 
then
		NextAction = "CodeReview"
	Else
		NextAction = "Resolve"
	End if		
  Case "CodeReviewed"
		NextAction = "Resolve"
  Case "Resolved"
		NextAction = "Verify"
  Case "Tested"
		NextAction = "Close"
  Case "Closed"
End Select



' - Get, Edit and Validate the record 
set EntityObj = newses.GetEntity("Issue", ThisID)
' - We now use the action previously selected 
     newses.EditEntity EntityObj , NextAction
     status = EntityObj.Validate 
     if (status <> "") then 
                newses.outputdebugstring "SB>>>: Fail in 
validating action " & NextAction  & _ 
                                                      ", the 
error is: " & status & vbCrLf 
        EntityObj.Revert
    else
        newses.outputdebugstring "SB>>>: Success in validating 
action " & NextAction  & _ 
                                                      ", the
error is: " & status & vbCrLf 
        EntityObj.Commit 
  end if  

End Sub

But there is a flaw in this method: the field's 'requiredness' defined in the behavior table will not have any effect. The action the user selected is of type modify and it is not clear during the record editing what is final state. Therefore, it is necessary to set the requiredness of the fields before the record is edited, that is, in the initialization hook of the action "Next".

Here is an example of how to do it:

Sub Issue_Initialization(actionname, actiontype) 
' actionname As String
'  actiontype As Long
'  action is Next
'  record type name is Issue
   
' - Purpose: Change fields Requiredness based on final state
' - Created by: SB on 18/12/2003
' - Date modified:


Dim newses , EntityObj
'  check the current state
Cur_state = GetFieldValue("State").GetValue
IType = GetFieldValue("Issue_Type").GetValue
ThisID = GetDisplayName 

'   find the required action
  Select Case Cur_state
  Case "Submitted"
  	If  (GetFieldValue("Req_CCB").GetValue = "Yes" OR IType = 
"Feature" ) then

	Else
  		SetFieldRequirednessForCurrentAction "Owner", 
AD_MANDATORY
  		SetFieldRequirednessForCurrentAction "Priority", 
AD_MANDATORY
	End if		

  Case "CCB"
  		SetFieldRequirednessForCurrentAction "Owner", 
AD_MANDATORY
  		SetFieldRequirednessForCurrentAction "Priority", 
AD_MANDATORY

  Case "Opened"
  		SetFieldRequirednessForCurrentAction "Resolution", 
AD_MANDATORY
  Case "CodeReviewed"
  		SetFieldRequirednessForCurrentAction "Resolution",
AD_MANDATORY
  
End Select

End Sub

Summary

We usually say that the ClearQuest state machine is static. In this article we explained how to overcome this limitation and make the state machine dynamic using some scripting. The flexibility of ClearQuest enables us to go beyond the basic functionality and allow organizations to create a (dynamic) workflow according to their needs. We have demonstrated how the flow can be automated in some cases; while in other cases a general user action (that we have called "Next"- the implicit meaning is that the user has finished the activity) will cause the CR to move to the required state, thereby enforcing the workflow rules. As a more complex example we have showed how we can adapt the flow for each and every record.

The ideas presented here can be developed and enhanced to answer the needs of a wide variety of organizations or projects. This is the beauty of ClearQuest, most of the customization can be done using the GUI, and the complex requirements can be developed using scripting.

Attached is a schema that is an example for implementation of the ideas raised in the article.

Security note: In the notification hook we have created a new session and log on as user, as follows:

     set newses = CreateObject("CLEARQUEST.SESSION")
       newses.UserLogon "user", "pass", "db_name", 
AD_PRIVATE_SESSION, "db_set"

The second parameter of the UserLogon session method is the user password. This is the visible password, which may cause a security risk to some organizations. The current ClearQuest version does not allow for encrypted password logon. The risk is that any user can open the ClearQuest Designer and see the password.

One solution is to use external libraries for encryption. ClearQuest Web users have no way to see the schema and password; this is a solution for some organizations. Other organizations remove the Cqdesign.exe after the client installation so regular users will not have access to the schema. Also, make sure that the user that was defined for the session logon does not have unnecessary privileges.

Hiding actions

As explained in sections 4.3 and 4.4, it is intended that the user will use the action "next", but the alternative actions (of type change-state) will appear in the action list, so user can take the wrong path by mistake.

There are some solutions to this issue:

  • First and very important is proper training and deployment. Users must know that whenever the action "next" is in the list this command should be used, and thus the final state will be the one set by the rules.
  • Another way is by defining a special group having limited membership, like the users "admin" and "user" (the user who logged on to the session we opened). Then limit the access_control of the change state actions to that group. In this way the users will see only the "next" action.
  • A third method is writing an access_control hook that will define who and when is allowed to perform the change state action. If a user tries to activate an action of type change state instead of the action "next" he will get an error.

The proposed solutions and scripts mentioned here are not included in this article or in the attached schema.

Sample schema

To load the attached sample schema, do the following:

  1. Expand the attached zip file to your local c:\temp directory.
  2. Open a command prompt window.
  3. Type the following DOS command:
    cqload importschema [-dbset connection_name] admin admin_password c:\temp\schema.txt

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=5224
ArticleTitle=Adaptive workflow in ClearQuest (or: How to create a dynamic state machine)
publish-date=07062004