Monitoring queues for Db2 Text Search index updates

You can gather monitoring information to tune the Db2® Text Search server configuration if you experience indexing performance issues.

For Db2 versions 10.5 and later, the -monitorQueues flag is deprecated. Information about the current state of both input and output queues is now available in the monitorN.csv file. The InputQueueSizes.csv and OutputQueueSizes.csv files are no longer required.

You do not need to manually enable queue logging. Instead, refer to the monitorN.csv file. For more details about monitorN.csv, see the table below


Use the information in the following table to troubleshoot indexing performance that is based on queue status.

Table 1. Queue monitoring status
Application type Related information
Time The current time in seconds. This information is not displayed in the IBM System Dashboard for Enterprise Content Management.
Total number of processed documents The total number of indexing requests processed by Db2 Text Search for all collections since the server started. This includes add, update, and delete requests. Deletions issued through the Query.delete method are excluded. All indexing requests are counted, regardless of if the processing was a success or failure.
Total size of processed documents The total memory size in kilobytes of all indexing requests processed by Db2 Text Search since the server started.
Total number of failed add requests The number of failed add requests processed by Db2 Text Search since server start.
Total number of successful add requests The number of successful add requests processed by Db2 Text Search since server start. This information is not visible in the IBM system dashboard for enterprise content management.
Total size of successful add requests The total memory size in kilobytes of successful add requests processed by Db2 Text Search for all collections since server start.. This information is not visible in the IBM system dashboard for enterprise content management.
Total number of failed delete requests The total number of failed delete requests processed by Db2 Text Search for all collections since server start. This information is not visible in the IBM system dashboard for enterprise content management.
Total number of successful delete requests The number of successful delete requests processed by Db2 Text Search for all collections since server start. This information is not visible in the IBM system dashboard for enterprise content management.
Documents in input queue The number of documents in the Db2 Text Search input queue.
Input queue size The memory size (in bytes) of documents in the Db2 Text Search input queue.
Documents in the output queue The number of documents in the Db2 Text Search output queue.
Output queue size The memory size (in bytes) of documents in the Db2 Text Search output queue.
Documents waiting for preprocessing The number of documents in the initial stage of the Db2 Text Search indexing pipeline that are waiting for preprocessing.
Documents currently in preprocessing The number of documents in the Db2 Text Search indexing pipeline, in the second stage of preprocessing (text extraction, tokenization, and language analysis)
Documents waiting for indexing The number of documents in the third stage of the Db2 Text Search indexing pipeline that are waiting to be indexed.
Documents currently being indexed The number of documents in the final stage of the Db2 Text Search indexing pipeline.
Number of concurrent queries The number of ongoing queries that are currently running in the system. This number includes all searches that have started but not yet completed at the time of measurement. This information is not visible in the IBM system dashboard for enterprise content management.
Total number of queries The total number of search requests that were processed by Db2 Text Search since the server started. This information is not visible in the IBM system dashboard for enterprise content management.
Used heap memory The amount of heap memory in megabytes that is used by the JVM before Java™ memory garbage collection. This information is not visible in the IBM system dashboard for enterprise content management.
Thread count Number of threads that are used by the Db2 Text Search server. This information is not visible in the IBM system dashboard for enterprise content management.
System load Provides an indication of the average system load for the previous minute, as provided by the JVM. This information might not be available on all platforms. This information is not visible in the IBM system dashboard for enterprise content management.
Open file descriptors The number of open operating system file descriptors. This information is available only for AIX, Linux®, and Solaris systems on which the lsof utility is installed.
Free physical memory Provides an indication of the free physical memory on the computer, as provided by the JVM. This information might not be available on all platforms. This information is not visible in the IBM system dashboard for enterprise content management.
Batches in progress In the monitor.csv output file:

Provides information about the indexing batches that are currently being processed. For each batch, the following information is provided in brackets, for example, [1;A;L;1000;1000]:

  1. Batch ID
  2. Type of batch with possible values:
    1. A: Add or Update
    2. D: Delete
  3. State of the batch with possible values:
    1. Initial (I): The server did not yet accept the entire batch from the client.
    2. Last document was received (L): The server accepted the entire batch from the client.
  4. Number of documents that were received so far for this batch
  5. Number of documents from this batch that are in the input queue.
  6. IBM system dashboard for enterprise content management: Displays the number of indexing batches (tasks) that are currently processed.
Active merges The number of index segment merges that are currently taking place.
Merge size The total size (in megabytes) of index segment merges that are currently taking place.

If tuning is required, open a support case with IBM Support for Db2. Few parameters could be maladjusted which will negatively affect the performance.