Managing a large set of Content Platform Engine queue sweep entries

Content

Note: The information in this techdoc has been incorporated into the Queue sweep troubleshooting section of the IBM FileNet P8 Platform and the IBM Content Foundation documentation in IBM Knowledge Center. The information in IBM Knowledge Center is the most current information on this subject.

• For FileNet P8 Platform, see Queue sweep troubleshooting.
  
• For Content Foundation, see Queue sweep troubleshooting.

This article explains the new queue entry status values and how to resolve failed entries, either individually or in large sets. "Queue entry" refers to any subclass of the abstract queue entry, which is processed by the sweep framework queue sweeps. This article does not apply to queue entries such as ContentQueues, QueueItems, and IndexRequests.

This article uses the Thumbnail Request Sweep and the associated Thumbnail Request queue entry as an example.

Queue sweeps in V5.2.1
In V5.2.1, queue sweeps allow for the unlimited retries of failing entries and the automatic deferral of processing certain entries. These changes lessen the amount of administrative effort that is needed to keep the sweeps running smoothly.

If problems occur, this document covers how to either retry or delete queue entries. The entries for a sweep can be found under the Requests tab. The columns are:

• Id: The id of the Queue entry.
• Date Created: The date the queue entry was created.
• Status: The queue entry status, whose values are described in the next section.
• Failure Count: The number of times that processing the queue entry has failed.
• Deferral Count: The number of times that processing the queue entry was deferred.
• Held Until Date: The date the sweep attempts to process the entry.
• Reason for Last Failure: A description of the last failure.

Queue entry status values
The status of an individual queue entry has four possible values:

• Waiting: This entry is queued for processing by the sweep handler.
• In progress: This entry is being processed by the sweep handler.
• Retry wait: The entry failed or was deferred. It is reprocessed on the date specified by the Held Until Date property.
• Failed: Processing of the entry failed enough times (equal to the sweep's Maximum Failures property) that the entry will no longer attempt to be processed.

Maximum failures
Queue sweeps have a Maximum Failures property that determines how many times the sweep tries and fails to processes a queue entry before no longer attempting to do so. A value of zero means that there is no maximum, and the sweep continuously retries to process queue entries. New queue sweeps have a value of zero by default.

There are two ways to resolve a failed status: either by deleting the queue entry or by resetting the failed status and allowing the entry to attempt to be reprocessed. Both methods are described in this document.

Note: The failure count is incremented even if the maximum failure property is set to zero.

Deferred count
New to V5.2.1 is the Deferral Count property for queue entries. This count is incremented when the queue sweep handler determines that the entry cannot currently be processed. For example, when an advanced storage area is used, a content deletion request is deferred if the content is on a storage device that the Content Platform Engine has determined to be offline. Deferred items are scheduled for retry in 5 minutes (specifically, the Held Until Date is set to the current time plus 5 minutes).

The deferred count is separate from the failure count, and does not affect the maximum failures. A deferral is an expected problem with performing the sweep, whereas a failure is an unexpected problem. For example, a thumbnail request fails if the image file no longer physically exists in the storage area.

Examining queue entries for troubleshooting
When you deal with large sets of queue entries, you can filter which entries are displayed to only those entries that you are interested in. To do so, open the Requests tab of the queue sweep and click Actions > Options.

From this window you can set how many entries are shown and what status or counts they must have in order to display them. As an example, if you want to see the first 200 entries that have a status of Waiting and a deferral count greater than or equal to 50, set the fields as follows:

• Show entries: 200
• Queue entry status: Waiting
• Failure count: Not set
• Deferral count: Greater than or equal to: 50
• Time limit: 180 seconds

API searches
Although the queue entry screen provides a useful overview of possible failures, it might be necessary to look at more properties than the Options screen allows for to properly troubleshoot the issue. You can do so from the Search screen with an object store search.

For example, assume that you want to do perform the same search as you did by using the Queue Entry Options screen, but only for documents that were created for the week that starts on September 1. You can set the criteria on the Simple View tab as follows:

• Class: Thumbnail Request
• Date Created: Greater Or Equal to 9/1/2014
• Date Created: Less Or Equal to 9/7/2014
• Queue Entry Status: Equal to 0
• Deferral Count: Greater Or Equal to 50

Note: The integer values for the Queue Entry Status property are: Waiting (0), In progress (1), Retry Wait (2), Failed (3).

Resetting failed entries
If a queue entry is failed but the underlying problem is fixed, resetting the failure count sets the value to zero and allows the queue sweep to attempt to process it again.

If the number of failed requests is small enough to be handled from the requests screen, select the documents that you want to reset and click Actions > Reset Failure Count.

Otherwise, a custom job can be set up. For more information, see Resetting failed queue sweep entries.

Deleting entries
If a queue entry is failed and there is no way to fix the underlying problem, queue entries can be deleted. This can be done manually through the Requests screen by selecting the entry and clicking Delete. For large sets, a Content Disposal Policy can be used.

When you create this policy, the filter expression must include QueueEntryStatus=3 (that is, the queue entry's status is failed) in addition to any other filtering parameters. As this is a policy and not a job, remember to delete or disable it after it has cleared out the set of failed entries, or optionally set an end date on the policy to prevent it from continuing to process.

Changes from V5.2.0
The Maximum Failure property changed in V5.2.1 to allow for retrying queue entries an unlimited number of times, which was not possible in V5.2.0. To accomplish this, set the property's value to zero. The maximum failure count for most internal queue sweeps, such as the advanced storage queue sweeps, are set to zero. This is because these sweeps are intended to contain only entries that can either be successfully processed or can be dismissed by the queue sweep as being no longer needed, for example, a content replication request for a content item of a document that was deleted. The Thumbnail Request queue is an exception to this, where the Maximum Failure property is set to 3.

In V5.2.0, it was possible to trigger reprocessing of queue entries that reached the maximum failure count by setting the maximum failure property to a lower value. This no longer triggers reprocessing because entries that are in the failed state are not selected for processing. Instead, follow the steps in “Resetting failed entries”.

Deferrals (described above) are also new for V5.2.1. See “Deferred Count” for a description of this field. Deferring entries allow for automatic retries of certain issues.