IBM Support

QRadar: Using the Cliniq script to perform system Health checks

Troubleshooting


Problem

What is Cliniq and how do you run it?

Cause

Run Cliniq health checks before major events, such as upgrades, to determine whether any issues need to be addressed first. You can also run Cliniq routinely to monitor the health of your system. This script is updated through the Auto updates process.

Resolving The Problem

Cliniq is a command-line interface health check tool that tests critical systems within QRadar. The cliniq script runs through a series of checks on various QRadar systems. It confirms whether these systems are running optimally or if there is a potential issue.
The Cliniq script provides checks on the following:
    Note: The ordering of tests might differ between systems.
  • Vault Installation and Config Check. Vault is a tool for controlling API keys, Passwords, and certificates.
  • Workload, service, container checks
  • Traefik Install and Config Check. Traefik is an Open Source Edger Router for the App framework.
  • Available Space Check-in /var/log/
  • High-Availability (HA) Recovery Token Check.
  • Deployment.xml In Global Config Check.
  • Log Rotate Check for extracted rolled files.
  • Docker Running Check.
  • S4 Folder Check. The /var/s4 directory is used to store the unseal script and root_token script, which are only accessible by the si-vault user.
Cliniq is designed to test for particular conditions and provide remediation steps for resolving issues found by the tests. Cliniq is version independent. These tools are released via Auto-Update.
To run the script
  1. Log in to the Console by using the root account via SSH connection.
  2. Type the command:  /opt/qradar/support/cliniq
Notes
  • Alternately you can save the output to a file for exporting. 
    For example:  /opt/qradar/support/cliniq > /store/ibm_support/cliniq_output
  • If a particular component is not installed, the test is [SKIPPED] for that component.
Options
# /opt/qradar/support/cliniq -h
  Usage of /opt/qradar/support/cliniq:
   -j    json output mode
   -m MODE
    define MODE: checkup, install, patch, upgrade (default "checkup")
   -t TAG
    set a TAG to run
   -v    display version information and exit
An example of the output
# /opt/qradar/support/cliniq
DrQ version 1.1.0 (mode: checkup, tag(s): <none>, verbosity: verbose)
Vault Install Checks
  Checks that vault-qrd is installed properly
   [SUCCESS]
    Root CA file /etc/pki/ca-trust/source/anchors/vault-qrd_ca.pem exists.
    Intermediate CA file /etc/pki/ca-trust/source/anchors/vault-qrd_ca_int.pem exists.
    Linux user si-vault exists.
    Linux group si-vault exists.
    Linux group s4 exists.
    Vault binary file /opt/qradar/bin/si-vault exists.
    Service vault-qrd systemd file /etc/systemd/system/vault-qrd.service exists.
Vault Config Checks
  Checks that vault is configured properly
   [SUCCESS]
    Vault root token script /var/s4/vault-qrd_root_token.sh exists.
    Vault root token script /var/s4/vault-qrd_root_token.sh has correct permissions.
    Vault root token script /var/s4/vault-qrd_root_token.sh has correct owner.
    Vault root token script /var/s4/vault-qrd_root_token.sh has correct group.
    Vault unseal script /var/s4/vault-qrd-unseal.sh exists.
    Vault unseal script /var/s4/vault-qrd-unseal.sh has correct permissions.
    Vault unseal script /var/s4/vault-qrd-unseal.sh has correct owner.
    Vault unseal script /var/s4/vault-qrd-unseal.sh has correct group.
Workload, Service and Container Checks
  Checks which workloads are available, what services are available, what
  containers should exist and that they have functionality.
   [SUCCESS]
    conman workloads retrieved
DeploymentXml In Global Config Check
  Ensures that deployment.xml does not exist in ConfigServices globalconfig directories
   [SUCCESS]
    The file /store/configservices/deployed/globalconfig/deployment.xml does not exist.
    The file /store/configservices/staging/globalconfig/deployment.xml does not exist.
Logrotate Checks
  Verifies logrotate is running properly
   [SUCCESS]
    No files found in /var/log/qradar.old with incorrect extension.
Docker Running Check
  Check if Docker is installed and running
   [SUCCESS]
    'docker.service' is active
    'docker.service' is running
Check S4 Folder
  Verify /var/s4 exists with the correct ownership and permissions
   [SUCCESS]
    /var/s4 directory exists
    /var/s4 has correct group ownership
    /var/s4 has correct user ownership
    /var/s4 has correct file permissions
Traefik Config Checks
  Validates that traefik is configured properly
   [SUCCESS]
    Traefik configuration file /etc/traefik/traefik.toml exists.
    Traefik configuration file /etc/traefik/traefik.toml has correct permissions.
    Traefik configuration file /etc/traefik/traefik.toml has correct owner.
    Traefik configuration file /etc/traefik/traefik.toml has correct group.
    Traefik server key /etc/traefik/tls/traefik.key exists.
    Traefik server key file /etc/traefik/tls/traefik.key has correct permissions.
    Traefik server key file /etc/traefik/tls/traefik.key has correct owner.
    Traefik server key file /etc/traefik/tls/traefik.key has correct group.
    Traefik certificate file /etc/traefik/tls/traefik.cert exists.
    Traefik certificate file /etc/traefik/tls/traefik.cert has correct permissions.
    Traefik certificate file /etc/traefik/tls/traefik.cert has correct owner.
    Traefik certificate file /etc/traefik/tls/traefik.cert has correct group.
    Traefik certificate authority file /etc/traefik/tls/traefik_ca.crt exists.
    Traefik certificate authority file /etc/traefik/tls/traefik_ca.crt has correct permissions.
    Traefik certificate authority file /etc/traefik/tls/traefik_ca.crt has correct owner.
    Traefik certificate authority file /etc/traefik/tls/traefik_ca.crt has correct group.
Available Space Checks
  Check if /var/log has enough space
   [SUCCESS]
    /var/log has sufficient space available (7603MB found).
HA Recovery Token Check
  Ensure /opt/qradar/ha/.ha_recovery token has been properly managed.
   [SUCCESS]
    Problem not detected.
Traefik Install Checks
  Validates that traefik is installed properly
   [SUCCESS]
    Linux user traefik exists.
    Linux group traefik exists.
    Traefik binary file /usr/bin/traefik exists.

HA Recovery Token Check For Patch
  Ensure /opt/qradar/ha/.ha_recovery token has been properly managed.
   [SKIPPED]
    skipped: mode of file: 'patch'. current run mode: 'checkup'
[SUMMARY] 11 successful checkups
[SUMMARY]  0 failed checkups
[SUMMARY]  0 invalid files
[SUMMARY]  1 skipped file
If tests fail, suggestions are provided for remediation of the issue.
 
Workload, Service and Container Checks
  Checks which workloads are available, what services are available, what
  containers should exist and that they have functionality.
     [FAILURE]
         Failed to decode workloads: failed endpoint call: error getting
         response for url https://QRadar732Base.ibm.com:9000/v1/api/workloads:
         Get https://QRadar732Base.ibm.com:9000/v1/api/workloads: dial tcp
         <IP address>:9000: connect: connection refused
        [REMEDIATION]
           <none provided>
Docker Running Check
  Check if Docker is installed and running
     [FAILURE]
         'docker.service' is not active.
        [REMEDIATION]
           Run 'systemctl status docker.service' to display any possible errors
           that may have occured during startup. Resolve these issues, if any,
           and run 'systemctl restart docker.service'.
[SUMMARY] 9 successful checkups
[SUMMARY] 2 failed checkups
[SUMMARY] 0 invalid files
[SUMMARY] 1 skipped file
Results
If the suggestions for remediation do not resolve the issue, open a case with IBM QRadar Support.

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Component":"Troubleshooting","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.2;7.3","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
11 January 2021

UID

ibm11088572