IBM Support

QRadar: Upgrade failed due to existing heap dumps under the /store/jheap/ directory

Troubleshooting


Problem

During the upgrade of a QRadar Appliance, the process fails due to the presence of heap dumps in the /store/jheap/ directory, resulting in an error message similar to the following:
Checks the /store/jheap directory for any heap dump files that need to be removed.
[FAILURE]
     The following dump file(s) were found:
     /store/jheap/hostcontext.hostcontext.oomdump.tar.gz
     /store/jheap/ccpp-2023-04-05-08:54:56-xxxx/coredump
     /store/jheap/ccpp-2023-04-05-08:54:56-xxxx/core_backtrace
     /store/jheap/ccpp-2024-06-12-12:30:39-xxxx/coredump
     /store/jheap/ccpp-2024-06-12-12:30:39-xxxx/core_backtrace
[REMEDIATION]
     Remove the dump files to avoid issues during the upgrade. 
     For each file found, run: rm <dump_file>

Cause

This failure occurs when there are dump files under /store/jheap/ during an upgrade.

Diagnosing The Problem

Administrators run the following steps to diagnose the issue:
  1. Use SSH to log in to the Affected QRadar Host as the root user.
  2. Ensure that the SFS patch file is mounted in the /media/updates/ directory.
  3. The cliniq health check should be run to see what are the dump files interfering with the upgrade: 
    /media/updates/cliniq/cliniq
    Output example: 
    Checks the /store/jheap directory for any heap dump files that need to be removed.
[FAILURE]
     The following dump file(s) were found:
     /store/jheap/hostcontext.hostcontext.oomdump.tar.gz
     /store/jheap/ccpp-2023-04-05-08:54:56-xxxx/coredump
     /store/jheap/ccpp-2023-04-05-08:54:56-xxxx/core_backtrace
     /store/jheap/ccpp-2024-06-12-12:30:39-xxxx/coredump
     /store/jheap/ccpp-2024-06-12-12:30:39-xxxx/core_backtrace
[REMEDIATION]
     Remove the dump files to avoid issues during the upgrade. 
     For each file found, run: rm <dump_file>
    Result
    The administrator is aware of existing dump files and can continue to the Resolving the Problem section if required.

Resolving The Problem

Administrators run the following steps to solve the issue:
  1. Use SSH to log in to the QRadar Console as the root user.
  2. Run this command to find and remove dumps under the /store/jheap/ directory: 
    /opt/qradar/support/all_servers.sh -Ck 'find /store/jheap/ -type f \( -name "*core*" -o -name "*dmp*" -o -name "*dump*" -o -name "*gz*" \) -exec rm -fv {} \;'
  3. Run the cliniq health check in the affected QRadar host to ensure that there are no more dump files. The procedure is stated in Diagnosing The Problem section.
  4. If there are no remaining dump files to remove, then the administrator can proceed to re-run the installer: 
    /media/updates/installer
    Result
    The patch starts with no jheap dump failures. If the issues persist, contact QRadar Support for assistance.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtdAAA","label":"Upgrade"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Document Information

Modified date:
01 August 2024

UID

ibm17160775

Page Feedback