This topic applies only to the IBM Business Process Manager Advanced configuration.

Archiving completed BPEL process and task instances

The archive.py administrative script moves completed top-level BPEL process instances and human task instances, including their associated data, from a Business Process Choreographer database to an archive database.

Location

The archive.py script is located in the Business Process Choreographer admin directory:

install_root/ProcessChoreographer/admin
install_root\ProcessChoreographer\admin

Running the script

The configuration script is run using the wsadmin command.

The script must be run in connected mode to your Business Process Choreographer configuration. The source and destination clusters and the deployment manager must be running.
Include the wsadmin -user and -password options to specify a user ID that has WebSphere system administrator authority.
If you are not working with the default profile, use the wsadmin -profileName profile option to specify the profile.
If the script is already running, further invocations will be serialized.

Enter the following command:

install_root/bin/wsadmin.sh -f archive.py parameters

Enter the following command:

install_root\bin\wsadmin -f archive.py parameters

Parameters

You can supply the following parameters:

-fromCluster fromClusterName
-toCluster   toClusterName
(-tasks | -processes)
(-all | [-finished] | [-terminated] | [-failed] | [-expired])
[(-templateName templateName [-nameSpace namespace] [ -validFromUTC timestamp ])]
[-kind ( todo | invocation | collaboration ) ]
[-completedAfterUTC  timestamp | -completedAfterLocal  timestamp ]
[-completedBeforeUTC timestamp | -completedBeforeLocal timestamp ]
[-startedBy userId | -createdBy userId]
[-slice numberOfEntities]
[-limit numberOfEntities]

-fromCluster fromClusterName: The name of the cluster where Business Process Choreographer is configured identifies the source database.
-toCluster toClusterName: The name of the cluster where Business Process Archive Manager is configured. This identifies the destination archive database
-tasks | -processes: Include one of these options to select whether tasks or processes will be moved to the archive. Specifying both will cause an error. Subprocesses are archived with the related process instances that they belong to. Process templates and task templates are copied with the related instance that depend on them.
-all | [-finished] [-terminated] [-failed] [-expired]: Specifies which process instances are to be moved according to their state. The -all option means all end states; finished, terminated, failed, and expired. If you do not specify -all, you must specify one or more of the end states. The -expired option is only valid for the -tasks option.
The set of instances to be archived can be restricted further by specifying further parameters.
-templateName templateName: Optionally, specifies the name of the process or task template whose instances will be moved. If there are multiple templates sharing the same name but with different validFrom dates the instances for all templates with that name are moved unless you use the -validFromUTC parameter to specify a particular template. If you use the -tasks option, you can further restrict the instance selection by also specifying one or both of the -nameSpace and -kind parameters.
-nameSpace nameSpace: Optionally, specifies the namespace of the task template whose instances will be moved.
-kind ( todo | invocation | collaboration ): Optionally, restricts the archiving to a specific kind of task.
-validFromUTC timestamp: The date from which the template is valid in Coordinated Universal Time (UTC). This option can only be used with the templateName option. The timestamp string has the following format: 'yyyy-MM-ddThh:mm:ss' (year, month, day, T, hours, minutes, seconds). For example, 2009-11-20T12:00:00.
-startedBy userId|-createdBy userId: Optionally, only moves completed task instances that were created by a specified user, or process instances that were started by a specified user ID.
-completedAfterLocal timestamp: Optionally, specifies that only instances that completed after the given local time are archived. The timestamp string has the following format: 'yyyy-MM-ddThh:mm:ss' (year, month, day, T, hours, minutes, seconds). For example, 2009-11-20T12:00:00. If you only specify a date, the time will default to 00:00:00 local time on the servers.
-completedAfterUTC timestamp: Optionally, specifies that only instances that completed after the given time in Coordinated Universal Time are archived. The timestamp string has the following format: 'yyyy-MM-ddThh:mm:ss' (year, month, day, T, hours, minutes, seconds). For example, 2009-11-20T12:00:00. If you only specify a date, the time will default to 00:00:00 UTC.
-completedBeforeLocal timestamp: Optionally, specifies that only instances that completed before the given local time are archived. The timestamp string has the following format: 'yyyy-MM-ddThh:mm:ss' (year, month, day, T, hours, minutes, seconds). For example, 2009-11-20T12:00:00. If you only specify a date, the time will default to 00:00:00 local time on the servers.
-completedBeforeUTC timestamp: Optionally, specifies that only instances that completed before the given time in Coordinated Universal Time are archived. The timestamp string has the following format: 'yyyy-MM-ddThh:mm:ss' (year, month, day, T, hours, minutes, seconds). For example, 2009-11-20T12:00:00. If you only specify a date, the time will default to 00:00:00 UTC.
-limit numberOfEntities: This optionally specifies the maximum number of top-level instances that will be archived by the current script invocation. The default value is 1. If you invoke the script using the -limit 0 option, no instances will be archived, and only the consistency check and any necessary recovery actions will be performed.
Important: Be careful increasing the value for numberOfEntities. After this script is started, it is not possible to stop it. Depending on your environment and the size of the BPEL processes, and the number of subprocesses and human tasks, each top-level process instance can take a long time to archive.
-slice numberOfEntities: Specifies how many object instances are copied or deleted in each database transaction. The default value is 1. Using larger slice sizes can result in better performance, but requires more database resources, such as locks, for the archiving operation.
Restriction: If you are using an Oracle database, specifying a very large slice size, for example, 1000, can cause Oracle error ORA-01795.

Results

The script reports which server the script is running on:

The archive operation is running on server 'runtimeClusterMember1' 
on node 'runtimeNode01'. Check the log files of the server to get 
information about the progress and results of the archiving operation.

When the script has finished, it outputs the following message:

archive.py finished.

It also reports how many top-level instances were moved to the archive or a warning that no instances were archived. For example, one of the following messages is output:

```
Top-level instances archived: 15
```

WARNING: The selection criterion returned no results. 
No instances archived.

For more detailed information, check the SystemOut.log file on the cluster member where the workload manager ran the script.

Troubleshooting and recovery

Because the archiving script must be run connected to the Business Process Choreographer configuration that the data will be archived from, that is where the API event handlers are called, and where events and audit log entries are generated.

After the script has been started, it cannot be stopped. Especially if you specify large value for the -limit parameter, the script might run for a long time before completing, which can cause the following warnings and errors:

If you get the error java.net.SocketTimeoutException: Read timed out or ADMC0009E: The system failed to make the SOAP RPC call: invoke, it probably means that a SOAP connection timeout happened before the archiving finished. In this case, the archiving continues, and you must check the log file of the cluster member where the workload manager ran the script, to see if it completed successfully. You can ignore these timeout errors, but to prevent them you must increase the timeout values, as described in Connection timeout when running a wsadmin script.
If the archiving operation takes a long time, it is normal for warnings like ThreadMonitor W WSVR0605W: Thread "SoapConnectorThreadPool : 0" (00000032) has been active for 611322 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung. to be written to the SystemOut.log file of the runtime deployment target. If the archiving operation completes, you will see another message like ThreadMonitor W WSVR0606W: Thread "SoapConnectorThreadPool : 0" (0000002d) was previously reported to be hung but has completed. It was active for approximately 3958253 milliseconds. There is/are 0 thread(s) in total in the server that still may be hung.
If you get the error CWWBB0665E: Archiving error with Oracle error ORA-01795, reduce the size of the slice parameter.

If archiving fails for any reason, for example, because the server is restarted during archiving, any unfinished archiving is not automatically completed. You will have to invoke the script again.

Because the script works in two phases; first copying the selected instances to the target archive database, and then deleting the instances from the source database, if the script fails during archiving, it is possible that either the copying was not completed or the deleting was not completed. This can mean that the same instances exist and are visible in both databases.

Having duplicate instances should not cause any problems, and will be corrected the next time that the script is run. After you have fixed the problem that caused the failure, invoke the archive.py script again using the same invocation parameters. If you invoke the script using the -limit 0 option, no instances will be archived, and only the consistency check and any necessary recovery actions will be performed.

If the copying phase did not complete, the script will delete the duplicate instances from the destination archive database.
If the copying phase completed, but the deletion phase did not complete, the script will continue deleting the duplicate instances from the source database.

Restriction: The following restrictions apply:

You cannot transfer objects from an archive database back to a Business Process Choreographer database, nor to another archive.
The first time that you archive instances to a new archive database, the identity of the Business Process Choreographer configuration is written to the database, and in future, only instances from that configuration can be archived to that archive database.
When instances are successfully moved to the archive, they are deleted from the Business Process Choreographer database, which generates a deletion event for the common event infrastructure (CEI) and the audit log. But it is not possible to identify that the deletion event was caused by an archiving action rather than by some other delete action, for example, cleanup service, user initiated delete action, delete script, or automatic deletion after successful completion.
You cannot archive to different archives at the same time. Parallel invocations of the archive.py script are serialized.
You cannot archive a process instance that has the same process name as any other process instance in the archiving database.
You cannot archive a process instance that has the same values for its correlation set as another process instance in the archiving database.
If you archive instances of a process template, then undeploy and redeploy the identical process template with the valid from date unchanged, you cannot archive any new instances of that process template to the same archive database. This is not an issue for normal process template versioning, where a different valid from date is used.

However, even if one of the restriction prevents you from archiving certain process instances to one archive database, you can archive those process instances to a different archive database, for which the restriction conditions are not true.

Example invocation

To move up to 500 finished and terminated BPEL process instances from the database for the Business Process Choreographer configuration on cluster "ProcessCluster" that completed before the year 2010 to the archive database for the Business Process Archive Manager configured on cluster "SupportCluster", perform the following action.

Enter the following command:

install_root/bin/wsadmin.sh 
    -f install_root/ProcessChoreographer/admin/archive.py 
    -fromCluster ProcessCluster -toCluster SupportCluster 
    -completedBeforeLocal 2010-01-01T00:00:00
    -processes
    -finished -terminated -limit 500

Enter the following command:

install_root\bin\wsadmin 
    -f install_root\ProcessChoreographer\admin\archive.py 
    -fromCluster ProcessCluster -toCluster SupportCluster 
    -completedBeforeLocal 2010-01-01T00:00:00
    -processes
    -finished -terminated -limit 500