IBM Support

QRadar: Offboarding event hashes

Troubleshooting


Problem

For audit purposes, retention policies, and to protect data it may be necessary for administrators to move file hashes to another system. Transferring the hash files to another system is fairly trivial in its basic form. The Linux utilities rsync and SSH do most of the work for us.

Resolving The Problem

Before you begin
This step-by-step guide is an example for administrators and should be modified to suit your specific site requirements and policies. NFS and SMB mounts should be used for backups and not for QRadar filesystems. For more information on NFS, refer to the Offboard storage guide.

Procedure to Transfer Hash Files to another System

Step 1: Create the target system

  1. Choose a target system as a destination host for the hash files. This guide uses examples of a Linux system with IP address of 192.168.2.2 as the target system in these procedures.
    1. On the destination host, create or select a user to receive the hash files. In this example,  the username is "hashes"
    2. Confirm that all QRadar hosts can connect via an SSH session to the destination host and login as user "QRadar_user".
    3. On the destination host, as the user QRadar_user, create a directory for each QRadar host that will be sending hash files.
      Tip: Create a directory that is the IP of the QRadar host sending hash files. This makes it easy to identify each host. Examples for QRadar host 192.168.2.5, to create a directory with that IP you would type: mkdir ~/192.168.2.5

Step 2: On the QRadar host, as root, generate a key pair if keys do not exist

  1. To verify that an ssh key pair does not already exist, type:  ls -l ~/.ssh/id_rsa*
  2. If no results are returned, the key does not exist. To create a key pair, type: ssh-keygen -b 4096
    1. Accept the default location
    2. Leave the passphrase empty

Step 3: Copy the public key for each host to the destination host

Manually copy the public key from the QRadar host to the destination host by using the command:
scp ~/.ssh/id_rsa.pub QRadar_user@192.168.2.2:

Step 4: On the destination host add public keys as QRadar_user

Add the public keys as the QRadar_user from the QRadar host to ~/.ssh/authorized_keys typing the commands cat ~/.id_rsa.pub >> ~/.ssh/authorized_keys
Note:
You need to type the command from where the public keys reside.

Step 5: Confirm that the QRadar host can connect to the destination

From each QRadar host you want to transfer hash file from, use an SSH session to confirm you can login to the remote system.
Example from QRadar host 192.168.2.5 type:  ssh 192.168.2.2

Results
You should be able to login with no errors.

Step 6: Add a crontab job to execute the rsync of hash files to the destination.

 Assuming the system has already been configured for hashing with the HMACSHA1 algorithm:

  1. On the QRadar host, create the file /etc/cron.d/rsynchashes by using the command
    touch /etc/cron.d/rsynchashes
  2. Using vi editor, edit the rsynchashes file and add the following line.
    * * * * * rsync -Rrptv --ignore-existing --include="*.HMACSHA1" --exclude="*~[0-9]" /store/ariel/events/md/ hashes@192.168.2.2:192.168.2.5/
    1. Where HMACSHA1 is one of the following based on your event hashing configuration:
      1. MD5
      2. SHA1
      3. SHA512
      4. HMACMD5
      5. HMACSHA1
      6. HMACSHA256
      7. HMACSHA512
    2. The IPs are replaced with the destination host IP and the QRadar host IP_directory, respectively.
  3. This crontab entry will:
    1. Run once a minute.
    2. Copy all hashes not yet copied to the destination.
    3. Will never overwrite a previously transferred file.

Step 7: Disk Maintenance

On the destination host, the hashes will accumulate forever regardless of the configured QRadar retention time. This could lead to endless performance slowdown due to the number of files to be compared on the destination or even inode exhaustion. It is recommended that administrators create a job that will remove the hashes older than the longest configured QRadar event retention period. If, for example, the retention is 90 days, create a daily cronjob on the destination that will remove hashes older than 90 days.

For example:
find /home/hashes/192.168.2.5/ -type f -daystart -mtime +90 -name "*.HMACSHA1" -delete

Step 8: Optionally, if the hashes are to be available online for integrity checking, make them available as a read only filesystem to the QRadar host

  1. Assuming NFS is acceptable, make sure the nfs system packages are installed on the destination.
  2. On the destination host, add the following to /etc/exports:
    Example: 
    /home/hashes/192.168.2.5 192.168.2.5(ro)
  3. On the QRadar host, to mount the read only filesystem from the destination host, type:
    mkdir  /mnt/md
    mount -t nfs -o ro 192.168.2.2:/home/QRadar_user/192.168.2.5 /mnt/md   
  4. When you verify the integrity of the hashes, you have the option to select the read only off-board hashes with the -r argument.
    For example:  /opt/qradar/bin/check_ariel_integrity.sh -d 60 -a HMACSHA1 -r /mnt/md

[{"Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Version(s)"}]

Document Information

Modified date:
01 April 2021

UID

ibm10742795