IBM Support

QRadar: Troubleshooting SSH when connections cannot be established

Troubleshooting


Problem

If you cannot SSH from the Console, it might be the result that SSH keys are corrupted or have permission issues. This article talks about how to diagnose and resolve these types of issues.

Resolving The Problem

If you cannot SSH from the Console, the examples below will provide Administrators an overview of what can prevent a tunneled connection in QRadar with potential workarounds.

Managed Tunnel Key directory (Console only)

If connections are being created properly between appliances, the Console should have one key for each encrypted host. Administrators can review their system to determine that each host has a key and that the ownership is as defined in the example.

/store/configservices/staging/globalconfig/ssh_public_keys/
[root@Console-1 ~]# ls -al /store/configservices/staging/globalconfig/ssh_public_keys/
drwxr-xr-x 2 nobody nobody 62 May 2 20:19 .
rwxrwxr-x 9 nobody nobody 24576 May 3 10:10 ..
-rw-r--r-- 1 nobody nobody 406 May 2 18:25 Console_key
-rw-r--r-- 1 nobody nobody 409 May 2 18:27 host_103_key
rw-r--r-- 1 nobody nobody 409 May 2 20:19 host_104_key

Review the permissions within the /root/.ssh directory (Console & managed hosts)

Permissions on the .ssh directory should be 700 and the files within the directory should be 600. For Console appliances and managed hosts, the files to review are in the /root/.ssh directory. Administrators check and fix permissions on directory /root/.ssh/ and SSH files.
 
[root@Console-1 .ssh]# ls -la .ssh/
total 24
drwx------ 2 root root 4096 May 2 18:35 .
dr-xr-x---. 4 root root 4096 May 2 18:38 ..
-rw------- 1 root nobody 426 May 2 18:35 authorized_keys
-rw------- 1 root nobody 1675 May 2 18:25 id_rsa
-rw------- 1 root nobody 406 May 2 18:25 id_rsa.pub
-rw------- 1 root root 788 May 2 18:25 known_hosts


Workaround
If permissions are incorrect, then Administrators should update permissions as instructed below:
 
[root@Console-1~]# chmod 700 /root/.ssh
[root@Console-1~]# chmod 600 /root/.ssh/*

Review the Console's public key file is present on the managed host

If the Console's id_rsa.pub is not in remote host's /root/.ssh/authorized_keys or there are no local public/private keys, then the SSH session will request a password. This password request due to a missing authorized key can prevent the tunnel from being created properly.
 
[root@Console-1 ~]# ssh 192.0.2.11
- SSH public key login from managed hosts to the Console is no longer automatic.
- SSH public key login from a QFlow to a Flow Processor is still automatic.


Workaround
  1. Log in to the Console using an SSH session as root user.
  2. Copy the keys using the command: ssh-copy-id user@host
    Example of command and output
      
      #ssh-copy-id root@192.168.0.84
        /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
        /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are
        already installed
        /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to  
         install the new key
         root@192.168.0.84's password:
        Number of key(s) added: 1
        Now try logging into the machine, with:  "ssh 'root@192.168.0.84'" and check to make sure  
        that only the key(s) you wanted were added.

Remote hosts SSH public key is wrong in local hosts /root/.ssh/known_hosts.

If the public key is incorrect on a host, this could be the cause of why an SSH connection cannot be established. When a standard SSH session cannot be established, then attempting to add a managed host with a tunnel would also fail to be added properly. The information below outlines how to review known SSH hosts issues.

In the example below, attempting to open a standard SSH session from the Console to a managed host cannot complete due to key not being found in the known_hosts list.

[root@QRadar-3100 .ssh]# ssh 192.168.0.77


Explanation
The error message 'Offending key in /root/.ssh/known_hosts: 2' indicates to the Administrator that line #2 in the known_hosts file is incorrect. Administrators can either compare the fingerprint.
The error message looks similar to:
 
Last login: Tue May 3 16:32:30 2016 from 192.168.0.75
This server was upgraded to QRadar 7.2.6.20151107134559 on Thu Apr 7 16:05:15 EDT 2016
with patch 7.2.6.20160405164932 applied on Mon Apr 11 14:00:17 EDT 2016

[root@Qradar726-1201 ~]# ssh 192.168.0.76
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @@@
>@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
bd:36:16:a8:00:2a:c9:56:6d:e2:26:eb:8d:66:3f:d5.
Please contact your system administrator.
Add correct host key in /root/.ssh/known_hosts to get rid of this message.
Offending key in /root/.ssh/known_hosts:2
RSA host key for 192.168.0.76 has changed and you have requested strict checking.
Host key verification failed.


Workaround
Administrators can use the command below to correct or remove line #2. After line #2 is removed or corrected, Administrators can attempt to connect by using SSH to get a prompt to update the known_hosts list or if the key was corrected, then the SSH session would be established without any prompts to the user. 
  1. Log into the QRadar Console using an SSH session.
  2. If the error message is not originating on the console SSH to the offending managed host.
  3. At the prompt type:
    ssh-keygen -R <IP of host>
    192.168.0.76
    ssh-keygen -R 192.168.0.76 
    Example where line 2 responds to 192.168.0.76
    ssh-keygen -R 192.168.0.76

Document Location

Worldwide

[{"Business Unit":{"code":"BU008","label":"Security"},"Product":{"code":"SSBQAC","label":"IBM QRadar SIEM"},"Component":"Deploy","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
08 January 2021

UID

ibm10960868