Maximum heap size configuration

You can improve the performance of your Db2 Text Search configuration by adjusting the maximum heap size of the integrated or standalone Db2 Text Search server.

When a document is received by the document ingestion thread, its content is placed in the document queue. Documents placed on the document queue remain there until an active indexing thread indexes it. In a typical operation, the speed of placing documents on the document queue is faster than the time required to parse and index the document. Therefore, at some point in time, the document queue reaches its capacity, and the document ingestion thread is blocked until another slot is freed from the document queue.

As the document queue fills with unprocessed documents, it consumes heap memory. Further memory is consumed for document processing like parsing and indexing. The combined heap memory consumption must be less than the maximum heap size of the process. By default, the heap size is configured to be 1500 MB.

Also, consider the ratio between the input and output queue memory size and the heap memory. The queue size is determined by the memory consumption of the documents in the queue. If you intend to process long documents, like 20 MB each, and decide to increase the queue memory size, consider increasing the heap size.

The maxHeapSize variable sets the maximum allowed heap size for the integrated or the standalone Db2® Text Search server. The default maximum heap size is 1.5 GB. This value must be a number between 1.5 GB and the maximum amount of memory allowed by your operating system and JVM version. Consider the following examples:
  • If you have a Windows system with a 32-bit JVM, then a process can have a maximum heap size of 2 GB. Therefore, your maxHeapSize parameter must be set to less than 2 GB. For example, 1.8 GB.
  • If you have an AIX® system with a 64-bit JVM, then the maximum heap size is limited only by the amount of virtual memory configured on the system. If many large documents with an average size of 20 MB must be processed continuously, then increase the maxHeapSize parameter to approximately 4 GB.

You can set the maximum heap size when you install or upgrade the stand-alone Db2 Text Search server by specifying the IA_MAX_HEAP_SIZE parameter in the response file. When you set the maximum heap size to a value greater than 2 GB during the installation or upgrade of the stand-alone text search server on a 64-bit operating system, file size limits for text, XML, and binary documents are increased for new collections. For each 8.3 MB of heap memory over 2 GBs, the values of the file size limits (60 MB by default) are increased by 1 MB (starting from 60 MB up to 400 MB):

60 MB + (heap memory - 2 GB)/8.3
Table 1. Examples of maximum heap size versus file size limit settings
Maximum heap size File size limits
2 GB 60 MB
3 GB 180 MB
4 GB 300 MB
Attention: When you modify the maximum heap size by using the configuration tool after installation, you must manually adjust the file size limits. Use the configuration tool to specify the maxBinaryTextSize, maxTextSize, and maxXmlTextSize parameters. During installation and upgrade, specifying the IA_MAX_HEAP_SIZE parameter in the response file for the standalone Db2® Text Search server automatically adjusts file size limits.
To change the maximum heap, issue the following command: 
configTool set -system -maxHeapSize <value> -configPath <full-path-to-configuration-folder>
where, <value> is the corresponding file size value and <full-path-to-configuration-folder> is the full path to the configuration.xml file for Db2 Text Search server.
Note: The output for the command configTool printAll -configPath <full-path-to-configuration-folder> lists the value for the maxHeapSize parameter as startupHeapSize.
To change the file size limits, issue the following command: 
configTool set -system -configPath <full-path-to-configuration-folder> -maxBinaryTextSize <value> -maxTextSize <value> -maxXmlTextSize <value>
where, <value> is the corresponding file size value and <full-path-to-configuration-folder> is the full path to the configuration.xml file for Db2 Text Search server.
On a 32-bit operating system, the typical configuration is:
  • Maximum heap size: 1.8 GB
  • Queue sizes: 90 MB each
  • File size limits: 60 MB
On a 64-bit operating system, the typical configuration is:
  • Maximum heap size: 3 GB
  • Queue sizes: 150 MB each
  • File size limits: 200 MB