IBM Support

QRadar: Important auto update server changes for administrators

Troubleshooting


Problem

IBM® is migrating QRadar SIEM auto update servers to a new location in the IBM Cloud®. This notice is intended to remind administrators that they must change their auto update configuration to use a new IBM Cloud® web server to avoid interruptions with daily and weekly software updates. Administrators who use IP-based firewall rules in their organization must also update their corporate firewall rules to allow traffic to the IBM Cloud auto update web server.

Resolving The Problem

Notice: Administrators must change their auto update server configuration before 30 November 2020 to avoid interruptions with your auto update download. A benign error message can be displayed in the user interface when you update your configuration as described in APAR IJ29298.

About

IBM® is migrating the QRadar® auto update servers to the IBM Cloud to better serve updates globally starting on 27 July 2020. Administrators with firewalls that use IP-based rules will experience interruptions to auto update downloads after November 30,2020. This server change allows a single host name and IP address to be leveraged through IBM Cloud® clusters for all QRadar auto updates globally.
 
Server changes Web server hostname Static IP address and port Location Description
New server cluster https://auto-update.qradar.ibmcloud.com/ 169.47.251.244:443 Global New server active on 27 July 2020
Legacy server https://qmmunity.q1labs.com/ 69.20.113.167 United States Active until 30 November 2020
Legacy server https://qmmunity-eu.q1labs.com/ 212.64.156.13 Europe Active until 30 November 2020

Affected versions

All QRadar® products and versions are impacted by this change.

IMPORTANT: Administrators who fail to update their corporate firewalls might experience an interruption in service after 30 November 2020. QRadar® Support recommends that all administrators update their QRadar Console's auto update settings during a maintenance window and confirm that auto updates complete successfully.

Summary

Administrators can update their QRadar auto update servers to use the new server location starting on 27 July 2020. Administrators must contact their corporate firewall team to ensure that any IP-based firewall rules are updated before November 30, 2020 to use the new static IP address at 169.47.251.244. QRadar Support recommends that administrators can add both the auto update host name and the static IP address to their corporate firewall policy rules to allow traffic:
 
Web server Static IP address and port Location Description
https://auto-update.qradar.ibmcloud.com/ 169.47.251.244:443 Global New server active on 27 July 2020

 
 

Action Required: Configuring your auto update server address

All QRadar administrators are expected to update their auto update server configuration by 30 November 2020 to prevent an interruption in auto update downloads.

  1. Log in to the QRadar Console as an administrator.
    Notice: QRadar on Cloud administrators are not required to make this change. The QRadar on Cloud DevOps team implemented a change on QRadar on Cloud Console appliances to use the new IBM Cloud weekly auto update server. 
  2. Click the Admin tab.
  3. In the System Configuration section, click Auto Update.
    image 5134
  4. Click Change Settings.
  5. Click the Advanced tab to configure the update server and backup settings.
  6. In the Web Server field, type the following address: https://auto-update.qradar.ibmcloud.com/
    image 5652

    Note: The Web Server field must include a trailing forward slash '/' character to prevent Invalid format for server errors. If you receive an invalid format error message, verify your auto update server URL ends with a forward-slash.
    image 5136
  7. If prompted, click Yes to load the auto update settings. This service restart does not stop any services, only reloads the configuration and refreshes the existing configuration.
  8. To test the auto update configuration, click Check for Updates.
  9. Click Get New Updates.
    image 5139
  10. Wait for the auto update server to update files.
    image 5143

    Results
    A system notification is generated to administrators that the auto update is complete. Optionally, administrators can confirm that updates were applied from the View Update History page or can use the command line to verify they are using the new auto update server at https://auto-update.qradar.ibmcloud.com/. For any issues, see the troubleshooting section.

Troubleshooting: SSL inspection requirements

Connections to the auto update server can fail to connect when proxy SSL inspection is enabled for the URL of the QRadar auto update server. When SSL inspection is enabled, the auto update server returns the error "vendor_manifest_list_512": 400 Bad Request as an incorrect CA is returned. To resolve this issue, administrators can disable SSL inspection on their proxy for https://auto-update.qradar.ibmcloud.com/.

Review /var/log/qradar.log on your QRadar Console to confirm auto update or tomcat.tomcat error messages. 
 
Sep 17 12:46:02 hostname AUTOUPDATE[116470]: Autoupdate 9.6 initialized.
Sep 17 12:46:02 hostname AUTOUPDATE[116470]: Do we need to turn on SSL Cert
Sep 17 12:46:05 hostname AUTOUPDATE[116470]: Could not retrieve "vendor_manifest_list_512": 400 Bad Request
Sep 17 12:46:06 hostname AUTOUPDATE[116470]: Could not retrieve "dau/dau.manifest.xml.asc": 400 Bad Request
Sep 17 12:46:06 hostname AUTOUPDATE[116470]: Could not retrieve signature for the manifest file.
Sep 17 12:46:08 hostname AUTOUPDATE[116470]: Could not read company
OR 

Sep 17 12:46:08 hostname tomcat [6599]: 2020-11-12 18:48:21,006 [QRADAR] [hostname@IPAddress (5562) 
/console/do/qradar/autoupdateSettings] org.apache.commons.httpclient.HttpMethodBase: 
[INFO] Response content length is not known
Sep 17 12:46:18 hostname [tomcat.tomcat] [hostname@IPAddress (5562) /console/do/qradar/autoupdateSettings] 
com.q1labs.autoupdate.ui.services.UIAutoupdateService: 
[INFO] [IPADDRESS/- -] Connected to the autoupdate server, but cannot recognize the certificate. 
We take this as good enough for validation purpose.

To verify whether SSL inspection is enabled, administrators can attempt to curl the QRadar auto update server and compare the returned SSL CA certificate. If the returned certificate lists their SSL inspection provider in the company name field, it indicates that SSL inspection needs to be disabled for the QRadar auto update server URL.
 

Procedure

  1. Log in to the QRadar Console as the root user.
  2. To verify the SSL CA certificate, type:

    For nonproxied connections: 
    • curl -v https://auto-update.qradar.ibmcloud.com/autoupdates/manifest_list
    • curl -kv https://auto-update.qradar.ibmcloud.com/autoupdates/manifest_list

    For anonymous proxy connections: 
    • curl -v -x https://proxy_server:proxy_port https://auto-update.qradar.ibmcloud.com/autoupdates/manifest_list
    • curl -kv -x https://proxy_server:proxy_port https://auto-update.qradar.ibmcloud.com/autoupdates/manifest_list
      Note: If the command succeeds with the -kv option, but fails with the -v option, it indicates that SSL inspection is enabled.
  3. Verify the returned SSL CA certificate to determine whether the company name is displayed.
  4. Disable SSL inspection for https://auto-update.qradar.ibmcloud.com/ on your proxy.

    Results
    After SSL inspection is disabled, administrators can run an auto update to retrieve the latest auto update. If you continue to experience issues, see the other troubleshooting sections or contact QRadar Support for assistance.

Troubleshooting: How to validate auto update settings from the command line

Administrators who prefer the command line can SSH to the QRadar Console as the root user to verify the connection to the new auto update server.

  1. Use SSH to log in to your QRadar Console as the root user.
  2. Type the following command to verify the connection: /opt/qradar/bin/UpdateConfs.pl -testConnect 1 0
    - If successful, the following message is displayed:[AUTOUPDATE] [TESTCONNECT] Test downloaded successfully! 
    image 5145

    - If unsuccessful, the following message is displayed and the administrator need to verify their proxy configuration:[AUTOUPDATE] [TESTCONNECT] Could not download manifest list.
    image 5149

Troubleshooting: Proxy validation and SSL 500 error messages

Users with a proxy configured in their auto update settings in QRadar who are unable to receive automatic updates where the auto update log displays the error: Could not contact the update server: 500 SSL negotiation failed: Could not download manifest list.

User interface error message:
image-20200317150131-2

Error log example: 
  Fri Mar 6 03:34:03 2020 [WARN] Could not retrieve "manifest_list_512": 500 
Can't connect to auto-update.qradar.ibmcloud.com:443 (Crypt-SSLeay can't verify hostnames)  
Fri Mar 6 03:34:03 2020 [DEBUG] Set error_code to 4  Fri Mar 6 03:34:03 2020 [DEBUG] Previous 
Value: 6  Fri Mar 6 03:34:03 2020 [DEBUG] Updating DB  Fri Mar 6 03:34:03 2020 [DEBUG] Successfully 
Updated DB error_code to 4  Fri Mar 6 03:34:03 2020 [WARN] Could not download manifest list.  
Fri Mar 6 03:34:03 2020 [DEVEL] Cleanup requested with return code 0  Fri Mar 6 03:34:03 2020 [DEBUG] 
Set autoupdate_status to 0  Fri Mar 6 03:34:03 2020 [DEBUG] Previous Value: 1  Fri Mar 6 03:34:03 2020 
[DEBUG] Updating DB  Fri Mar 6 03:34:03 2020 [DEBUG] Successfully Updated DB autoupdate_status to 0  
Fri Mar 6 03:34:03 2020 [DEVEL] Cleaning up scripts.

How to resolve SSL 500 proxy errors
A utility is available on IBM Fix Central to resolve manifest and connection issues. The AUProxyFP.tgz file on IBM Fix Central can be used to resolve proxy connection issues on all QRadar 7.3.x and 7.4.x versions.
  1. Download the Auto Update fix pack from IBM Fix Central to your laptop or workstation: AUProxyFP.tgz.
  2. Use SSH to log in to the QRadar Console as the root user.
  3. Copy the file to a directory of the QRadar Console, such as /root, /tmp, or /storetmp.
  4. Navigate to the directory where you copied the file.
  5. Type the following command to extract the file: gunzip -c AUProxyFP.tgz | tar zxvf -
  6. Navigate to the directory with the extracted file.
  7. Type the following command to install the proxy fix pack: ./install.sh
  8. After the installation completes, type the following command to verify the connection:
    /opt/qradar/bin/UpdateConfs.pl -testConnect 1 0

    - If successful, the following message is displayed and the administrator can continue to Step #8:
      [AUTOUPDATE] [TESTCONNECT] Test downloaded successfully!
    - If unsuccessful, the following message is displayed and the administrator need to verify their proxy configuration:
     [AUTOUPDATE] [TESTCONNECT] Could not download manifest list.
     
  9. Log in to the QRadar Console as an administrator.
  10. Click the Admin tab.
  11. Click Auto Update icon.
  12. Click Get New Updates button.
  13. Wait for the auto update to attempt the connection.
  14. Click View Log to verify the Last Update Status.
    image 5144

    Results
    If you continue to experience issues or error messages related to "Could not contact the update server: 500 SSL negotiation failed: Could not download manifest list", then contact QRadar Support.

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtDAAQ","label":"Auto Update"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
10 November 2021

UID

ibm16244622

Page Feedback