Tuning the memory

If you plan to process documents with a large number of pages, you might need to tune your Content Platform Engine and Document Processing engine components.

• Tuning Content Platform Engine
  If your Content Platform Engine SweepDispatcher runs into an out of memory error when processing documents, with a message such as:

  FNRCE0000E - ERROR SweepDispatcher _unitOfWork threw a non-Engine exception java.lang.OutOfMemoryError: Java heap space

  You can solve this problem by increasing the Content Platform Engine Java™ heap size.
  1. See Updating script-installed deployments to update your CR YAML file.
  2. In your CR file, under ecm_configuration.cpe.cpe_production_setting, update the jvm_max_heap_percentage parameter to a higher percentage, for example 66 (the default is 33). For more information, see Content Platform Engine parameters.
  3. The operator reconciliation loop might take up to 30 minutes or more, and the Content Platform Engine pod(s) will be restarted.
• Tuning the Document Processing engine setup pod
  If your setup container runs into a WorkerLostError error, it is most likely due to an out of memory error during the processing of documents. You might get the following message:

  | ERROR | <tenantId> | <ontology> | <transID> | celery.worker.request | Task handler raised error: WorkerLostError('Worker exited prematurely: signal 9 (SIGKILL) Job: 0.',)
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1267, in mark_as_worker_lost
      human_status(exitcode), job._job),
  billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL) Job: 0.

  You can increase the memory limit of the setup pod to solve this problem:
  1. See Updating script-installed deployments to update your CR YAML file.
  2. In your CR file, under ca_configuration.global.setup.resources.limits, increase the memorylimit parameter to a higher number, for example 1024Mi (the default is 700Mi). For more information, see ADP Configuration parameters (ca_configuration).
  3. The operator reconciliation loop might take up to 30 minutes or more, and the setup pod(s) will be restarted.
• Tuning Document Processing engine classifyprocess-classify pod
  If your classifyprocess-classify container runs into a WorkerLostError error, it is most likely due to an out of memory error during the processing of documents. You might get the following message:

  | WARNING | <tenantId> | <ontology> | <transID> | py.warnings | /usr/local/lib64/python3.6/site-packages/scipy/sparse/_index.py:84: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
    self._set_intXint(row, col, x.flat[0])
  Process 'ForkPoolWorker-5:10' pid:45717 exited with 'signal 9 (SIGKILL)'
  |   ERROR | PDB1 | ONT2 | 6c4e0346-74e7-4726-86b1-46d43c6015b9 | classify_doc | classify_doc task was terminated because it threw an error for page .
  |   DEBUG | PDB1 | ONT2 | 6c4e0346-74e7-4726-86b1-46d43c6015b9 | classify_doc | Worker exited prematurely: signal 9 (SIGKILL) Job: 49.
  Traceback (most recent call last):
    File "./classify_doc/tasks.py", line 43, in classify_pool
    File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1791, in get
      raise self._value.exception
    File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1267, in mark_as_worker_lost
      human_status(exitcode), job._job),
  billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL) Job: 49.

  You can increase the memory limit of the classifyprocess-classify pod to solve this problem:
  1. See Updating script-installed deployments to update your CR YAML file.
  2. In your CR file, under ca_configuration.global.classifyprocess.resources.limits, increase the memorylimit parameter to a higher number, for example 3Gi (the default is 2Gi). For more information, see ADP Configuration parameters (ca_configuration).
  3. The operator reconciliation loop might take up to 30 minutes or more, and the classifyprocess-classify pod(s) will be restarted.
• Tuning the Document Processing engine postprocessing pod
  If your postprocessing container runs into a WorkerLostError error, it is most likely due to an out of memory error during the processing of documents. You might get the following message:

  | ERROR/ForkPoolWorker-1] Process 'ForkPoolWorker-1:9' pid:79732 exited with 'signal 9 (SIGKILL)'
  | icp4adeploy-postprocessing-589b8dd99f-7285z | PDB4  | ONT4 | f844a9c0-16f9-11ec-9e39-71406e39d67c  | ERROR | post processing task was terminated because it threw an error for page 3.
  | ERROR/ForkPoolWorker-1] post processing task was terminated because it threw an error for page 3.
  | DEBUG/ForkPoolWorker-1] Worker exited prematurely: signal 9 (SIGKILL).
  Traceback (most recent call last):
    File "./postprocessing/tasks.py", line 132, in postprocessing
    File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1791, in get
      raise self._value.exception
    File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1267, in mark_as_worker_lost
      human_status(exitcode)),
  billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL).

  You can increase the memory limit of the postprocessing pod to solve this problem:
  1. See Updating script-installed deployments to update your CR YAML file.
  2. In your CR file, under ca_configuration.global.postprocess.resources.limits, increase the memorylimit parameter to 1024Mi or higher (the default is 800Mi). For more information, see ADP Configuration parameters (ca_configuration).
  3. The operator reconciliation loop might take up to 30 minutes or more, and the Postprocess pod(s) will be restarted.