Managing persisted data

Whenever data is persisted into a database, a time comes when its important to delete or archive some of that data. Periodically cleaning up saves disk space and improves performance.

Note: Back up your data before you archive and purge that data.

The data

IBM® Business Automation Workflow retains the following kinds of data. All of them are candidates for regular purging or archiving. The tables provide pointers to more specific guidance.

Standard deployment environment
Workflow Center holds snapshots of process applications and toolkits as they are developed.
Workflow Server holds snapshots of process applications that have been deployed to it.
Workflow Server also holds process instances and their user tasks.
Workflow Center and Workflow Server both hold historical EPV values as they are updated.
BPM document store is a built-in storage location for document attachments and case properties.
The Performance Data Warehouse holds event data collected from tracking groups and auto-tracking of running process instances.
Durable subscription events for intermediate messages are retained.
Installation messages are stored in the system temporary folder (%temp%).
Some index data is stored in memory indices to support user interactions.
Shared business objects are retained.
Advanced deployment environment
In addition to the data mentioned for a Standard deployment environment, an Advanced deployment environment stores information about service functions.
In addition to process instances and their user tasks, Workflow Server holds instances of BPEL processes and their human tasks.
Workflow Server also stores information about failed events. You can view that information through the Failed Event Manager.
WebSphere Application Server
System and trace logs are stored during production and when doing additional traces.
WebSphere Transaction Manager saves transaction logs from inflight transactions.
The system also saves transaction logs from databases used by Business Automation Workflow.

Workflow Center

Table 1. Cleanup activities for Workflow Center administration
Activity Description Resources
Archiving in Workflow Center Archiving a project does not delete it or reclaim its space in the database. Archiving marks the project so that it does not show by default in the Workflow Center project list. Archiving a project archives all its snapshots and artifacts.

You can archive individual named snapshots instead of archiving an entire project.
Deleting historical EPV values Historical EPV values are kept in the database. When EPV values are rapidly updated, the corresponding database rows can quickly accumulate and adversely impact the database performance. Use the BPMEPVHistoryCleanup wsadmin command to purge unneeded EPV values from the database, as described in the topic BPMEPVHistoryCleanup command.
Deleting in Workflow Center (Standard deployment environment) To delete a project, you must first archive it. Then use the BPMRemoveProcessApplication wsadmin command to remove a process application. Removing a project deletes all snapshots and process instances. Use the BPMRemoveToolkit wsadmin command to remove a toolkit.

To delete a project with the user interface, select the archived projects; then click Delete next to the project that you want to remove. Deleting a project deletes all snapshots and process instances.

An unnamed snapshot is created every time you save artifacts in Process Designer. A named snapshot is simply an unnamed snapshot that has been given a name. These unnamed snapshots are saved in the database to provide a project history, but they should be regularly purged. The LSW_PO_VERSIONS table incurs the most growth when unnamed snapshots are not regularly deleted.

You can delete any named snapshot manually. You can delete named or unnamed snapshots using the wsadmin commands. You can also configure Business Automation Workflow to delete unnamed snapshots automatically when they are no longer needed.

You can delete snapshots using the BPMSnapshotCleanup wsadmin command. To delete a named snapshot, you must first archive it.

Deleting in Workflow Center (Advanced deployment environment) In addition to the actions described for a Standard deployment environment, you need to be aware of business level applications (BLAs) and enterprise applications that are created on the Workflow Center playback server. If process applications or toolkits in Workflow Center contain a module or library (advanced content), either directly or through a toolkit, a BLA is created for the tip of every process application or toolkit and for every named snapshot that contains advanced content, if one of the following conditions is met:
  • Other projects are associated with the process application or toolkit (other modules, libraries, or any type of IBM Integration Designer, Rational® Application Developer, or Eclipse project).
  • The default module contains executable content: an import or export statement, or a component.

For example, normally a process application or toolkit contains a default module and default library. If the user adds a component, import, or export to the module, or associates any other project, the process application or toolkit is deployed and a BLA is created.

Such advanced content can quickly accumulate, and that accumulation can affect the time it takes to start the server, memory use, and general performance.

If a snapshot contains Advanced Integration Services (for example, SCA modules or BPEL processes), you can use the Workflow Center console to undeploy it from the Workflow Center server.

If you are following the façade pattern, You usually only need one BLA for the most current snapshot of a process application. To delete BLAs, click the Undeploy option on the Snapshots page of Workflow Center. To delete named snapshots, click Deactivate then Undeploy.
Deleting BPMN process instances When you delete process instances, task instances are also deleted. The BPMProcessInstancesPurge wsadmin command allows you to delete specific instances or delete instances that completed within a specified date range.
Undeploying a process application or toolkit tip (Advanced deployment environment) The tip is the current working version of a process application or toolkit. When you undeploy a tip, all Advanced Integration Service artifacts and the associated business level application (BLA) are removed from the server. The process application definition and other artifacts are still present in the repository.

If you are using the façade pattern and testing the advanced content in an Integration Designer Unit Test Environment (UTE), you almost never need the tip to be deployed. However, since the tip is automatically deployed when changes occur, you need to undeploy the tip if it is not really needed.

Workflow Server

Table 2. Cleanup activities for Workflow Server administrators
Activity Description Resources
Deactivating a snapshot Use the BPMDeactivate command to deactivate a snapshot. Deactivating allows all existing instances to complete processing, but no new process instances are created. In a network deployment environment, you must run this command on the node containing the application cluster member that handles Workflow Server or Workflow Center applications. Do not run this command from the deployment manager profile.

You can deactivate a running process application snapshot on the Workflow Center server or a Workflow Server. If your process application uses a BPEL process as the main entry component, you must stop the corresponding BPEL template in the WebSphere administrative console. If this BPEL process invokes a BPD, you must allow any existing instances to complete after you stop the template but before you deactivate the snapshot.

Deleting snapshots from IBM BPM Standard on Workflow Server Over time, a workflow server can accumulate many snapshots of a process application. You can delete snapshots that are no longer in use. When you delete a snapshot, you also delete any business process definitions that are associated with it.

The snapshot cannot have any running instances. The snapshot cannot be active.

Toolkit snapshots are not automatically deleted when process application snapshots are deleted. Use the BPMShowSnapshot command to obtain a list of toolkit snapshots on which a process application depends. If those toolkit snapshots are not needed by any other process application or toolkit, delete them using the BPMDeleteSnapshot command.
Deleting BPMN process instances When you delete process instances, task instances are also deleted. The BPMProcessInstancesPurge wsadmin command allows you to delete specific instances or delete instances that completed within a specified date range.
Deleting system and decision tasks when the tasks complete By default, data about system and decision tasks is saved after the task is complete. Use the BPMTasksCleanup wsadmin command to delete completed tasks from running process instances on the workflow server. If you do not want to store audit data for system and decision tasks on the workflow server, select Delete task on completion when you create a system or decision task in Process Designer.
Deleting completed tasks from the runtime database Long running process instances can produce a number of completed tasks in the runtime database. When the number of completed tasks gets large enough, they can affect the performance of the runtime database. You can resolve the problem by purging the runtime database of the completed tasks, which are no longer needed at run time. Note that deleting the completed tasks does not delete them from the system. The completed tasks are still in the archive database. Deleting completed tasks is also useful if you are migrating process instances. By removing completed tasks, you are migrating only the information needed by the process instances in their new location.
Deleting execution data about user, decision, or system tasks when the task completes By default, the runtime execution state of an activity is saved after it is complete. This data cannot be deleted after it has been saved. If you do not want to store execution data (such as variable values) for viewing after the process is complete, select Clean State from the Task Header section of the process diagram in Process Designer.
Deleting snapshots on Workflow Server (Advanced deployment environment) If a process application contains content from Integration Designer, a business-level application (BLA) with EAR files is created. After the process application is installed on the workflow server, the BLA and its constituent EAR files are deployed on the server. You need to stop the processes and undeploy the content before you delete the snapshot.
Deleting BPEL process instances (Advanced deployment environment) Use the deleteCompletedProcessInstances.py administrative script to selectively delete from the BPC database or the Business Process Archive database any parent BPEL process instances that have reached an end state of finished, terminated, or failed.
Deleting BPEL human task instances (Advanced deployment environment) Use Business Process Choreographer Explorer to delete human tasks individually. You can also use the Jython scripts deleteCompletedTaskInstances.py or deleteCompletedProcessInstances.py.

Use the cleanup service in the Human Task Manager or Business Flow Manager administrative console pages to schedule jobs to delete task or process instances.
Deleting Business Process Choreographer objects (Advanced deployment environment) Various database objects accumulate in a running system, for example, audit log entries, task and process instances, task and process templates, and people queries. Regularly running administrative scripts to delete objects that are no longer needed from the Business Process Choreographer databases can help prevent wasting storage space.
Deleting groups
  • Security groups come from a configured user repository, such as LDAP. Security groups are never deleted when process instances or tasks are deleted.
  • A team, formerly called a participant group, is associated with a snapshot.
  • A temporary group is formed if you assign a task to a list of users or groups. Temporary groups are deleted from the server automatically when no task uses them any more.
  • Dynamic groups are defined by an expression, such as all users from Canada. The membership of the group can change each time the server is started, Dynamic groups are scoped to a snapshot.
  • An internal group can contain users from more than one user repository. Internal groups are created and deleted from the Process Admin Console.
Deleting unused ad hoc groups An ad hoc group is shared among tasks that make use of it. Because it can be shared, an ad hoc group is not deleted when the task is deleted. You can clean up unused ad hoc groups in the following ways:
  • Use the Process Admin Console during task cleanup.
  • Delete unused ad hoc groups while deleting a process instance.
Deleting durable messages Durable subscription messages exist and persist in the LSW_DUR_MSG_RECEIVED database table if durable subscriptions are enabled for certain events.
Cleaning up shared business objects When a new version of a shared business object is saved, the older versions of the shared business object that are no longer required are automatically deleted. The system performs the cleanup based on these configuration properties:
cleanupMaxVersionCount
Specifies the number of versions that should be kept. For example, a cleanupMaxVersionCount of 4 indicates that at least 4 versions should be kept.
cleanupMinAge
Specifies a time in hours that prevents recent versions from being deleted. For example, a cleanupMinAge of 48 indicates that a version that was created less than 48 hours ago should not be deleted, regardless of the cleanupMaxVersionCount specified.

Performance Data Warehouse

Table 3. Cleanup activities for database administrators
Activity Description Resources
Purging old instances Task and process instances are recorded in the database, even after the task and process have been completed.
Archiving data in Performance Data Warehouse You can archive snapshots and mark all the metadata in those snapshots with an ARCHIVED time stamp. Business Automation Workflow does not use archived metadata when it generates Performance Data Warehouse schema and views.
Purging data in Performance Data Warehouse BPD events that are being tracked are saved in the database of Performance Data Warehouse. Use the prune command to remove data that you no longer need from the Performance Data Warehouse database.

File system

Table 4. Cleanup activities for the temporary directories
Activity Description Resources
Purging the Business Automation Workflow temporary directory During installations and during some other runtime operations, files are placed in the system temporary directory (%temp%). You can delete those files when the installation has completed. The server must be stopped when you delete files from the temp directory.  
Managing WebSphere log files IBM Websphere Application Server creates temporary files in the following directories:
  • profile root/profile name/config/temp
  • profile root/profile name/temp
  • profile root/profile name/wstemp

You can delete the temp files after stopping the servers on the profile node. You can delete the content of the wstemp directory after shutting down the deployment manager.

Some other WebSphere log files purge data automatically.
  • The SystemOut.log file contains messages about significant events from the JVM process. Log files automatically roll over and purge the oldest data. You can use a background batch process to capture the oldest logs before rollover.
  • Never delete the transaction log. It is self-cleaning.