IBM Support

QRadar: Upgrading large deployments in parallel can cause upgrade failed on some hosts.

Troubleshooting


Problem

Upgrading large deployments in parallel can sometimes cause upgrade failures on some managed hosts.

Symptom

Some managed hosts fail the upgrade during pre-check.

Cause

The problem is due to the managed hosts need to connect to the console's tomcat service during the initial upgrade pre-check. With so many managed hosts upgrading in parallel and trying to connect to console's tomcat service at the same time, sometimes the connection can fail on some managed hosts. The upgrade pre-check will not reattempt to connect and fails the upgrade instead.  This issue is more likely to occur when upgrade using all_server command to initiate the installer.

Environment

Large deployments of over 50 managed hosts

Diagnosing The Problem

On the managed host, in the /var/log/setup.xxxxxxxx/patch.log:

Jul 27 09:30:44 2023: Jul 27 09:30:44 2023:[ERROR] Failed to connect to tomcat

Resolving The Problem

Manually re-run the installation on the managed host:
  1. SCP the upgrade file from console to the managed host:
    scp /storetmp/<upgrde file>  managedHostIP:/storetmp
  2. To create the /media/updates directory, type the following command:
    mkdir -p /media/updates
  3. To mount the file to the /media/updates directory, type the following command:
    mount -o loop -t squashfs /storetmp/<upgrade file> /media/updates
  4. To run the patch installer, type the following command:
    /media/updates/installer 

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtdAAA","label":"Upgrade"}],"ARM Case Number":"TS013347837","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Document Information

Modified date:
11 December 2023

UID

ibm17082703