IBM Support

QRadar: Offboarding event hashes

Troubleshooting


Problem

For audit purposes, retention policies, and to protect data administrators might need to move file hashes to another system. Transferring the hash files to another system allow users to reduce disk space and move data to a location for long-term storage. The Linux utilities rsync and SSH do most of the work for us.

Resolving The Problem

Before you begin
This step-by-step guide is an example for administrators and you must modify the commands to suit your specific site requirements and policies. For more information, see the QRadar offboard storage guide.

Procedure to transfer event or flow hashes to another system

Step 1: Create the target system

  1. Choose a target system as a destination host for the hash files. This guide uses examples of a Linux system with IP address of 192.168.2.2 as the target system in these procedures.
    1. On the destination host, create or select a user to receive the hash files. In this example, the username is "hashes"
    2. Confirm that all QRadar hosts can connect with an SSH session to the destination host and login as user "QRadar_user".
    3. On the destination system, create a directory for each QRadar host.
      Tip: Create a directory that is the IP of the QRadar host sending hash files. The directory name makes it easy to identify the hash files from each host. For example, if your QRadar host is at 192.168.2.5, to create a directory with that IP you would type:
      mkdir ~/192.168.2.5

Step 2: On the QRadar host, as root, generate a key pair if keys do not exist

  1. To verify that an ssh key pair does not exist, type: 
    ls -l ~/.ssh/id_rsa*
  2. If no results are returned, the key does not exist. To create a key pair, type:
    ssh-keygen -b 4096
    1. Accept the default location
    2. Leave the passphrase empty

Step 3: Copy the public key for each host to the destination host

Manually copy the public key from the QRadar host to the destination host by using the command:

scp ~/.ssh/id_rsa.pub QRadar_user@192.168.2.2:

Step 4: On the destination host add public keys as QRadar_user

Add the public keys as the QRadar_user from the QRadar host to ~/.ssh/authorized_keys typing the commands:

cat ~/.id_rsa.pub >> ~/.ssh/authorized_keys

Note: You need to type the command from where the public keys reside.

Step 5: Confirm that the QRadar host can connect to the destination

From each QRadar host you want to transfer hash file from, use an SSH session to confirm you can log in to the remote system.
Example from QRadar host 192.168.2.5, type: 

ssh 192.168.2.2


Results
You must confirm you can successfully SSH to the remote host. Any errors or prompts displayed must be resolved before you continue to the next section.

Step 6: Add a crontab job to execute the rsync of hash files to the destination

 Assuming the system is configured for hashing with the HMACSHA1 algorithm:

  1. On the QRadar host, create the file /etc/cron.d/rsynchashes with the following command:
    touch /etc/cron.d/rsynchashes 
  2. Using vi editor, edit the rsynchashes file and add the following line:
    * * * * * rsync -Rrptv --ignore-existing --include="*.HMACSHA1" --exclude="*~[0-9]" /store/ariel/events/md/ hashes@192.168.2.2:192.168.2.5/
    1. Where HMACSHA1 is one of the following based on your event hashing configuration:
      1. MD5
      2. SHA1
      3. SHA512
      4. HMACMD5
      5. HMACSHA1
      6. HMACSHA256
      7. HMACSHA512
    2. The IP addresses are replaced with the destination host IP and the QRadar host IP_directory.
  3. This crontab entry:
    1. Runs every minute.
    2. Copies file hashes not on the destination.
    3. Does not overwrite a previously transferred file.

Step 7: Disk maintenance

On the destination host, the hashes accumulate forever regardless of the configured QRadar retention time. Over time, performance issues can occur due to the number of files to be compared on the destination or even inode exhaustion. It is recommended that administrators create a job that removes file hashes older than the longest configured QRadar event retention period. For example, the retention is 90 days, create a daily cron job on the destination that removes hashes older than 90 days.

For example,

find /home/hashes/192.168.2.5/ -type f -daystart -mtime +90 -name "*.HMACSHA1" -delete

Step 8: Optional. If the hashes are available online for integrity checking, ensure the files are read-only and accessible to QRadar

  1. Assuming NFS is acceptable, ensure the system packages are installed on the destination.
  2. On the destination host, add the following to /etc/exports:
    /home/hashes/192.168.2.5 192.168.2.5(ro)
  3. On the QRadar host, mount the read-only file system for the destination:
    mkdir  /mnt/md
    mount -t nfs -o ro 192.168.2.2:/home/QRadar_user/192.168.2.5 /mnt/md
  4. When you verify the integrity of the hashes, you can use the -r option to check file hashes on another host. For example,
    /opt/qradar/bin/check_ariel_integrity.sh -d 60 -a HMACSHA1 -r /mnt/md
     

[{"Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Version(s)"}]

Document Information

Modified date:
21 February 2023

UID

ibm10742795