IBM Support

QRadar: Verifying SSH connectivity to the target Managed Host

Troubleshooting


Problem

When a Managed Host is suspected as the source of a problem, verifying SSH connectivity to that Managed Host is an important step.

Cause

Verifying that you are able to connect to a Managed Host from your Console by using SSH can give a wealth of information about the state of network connectivity to that host. Before you can start troubleshooting any issues on a Managed Host, you must be able to access it. While SSH is not intended to be a diagnostic tool, you cannot establish an SSH connection unless the networking layers 1 and 2 are operating correctly for the host in question. How SSH is behaving can give you an indication of what actual diagnostic tools need to be utilized next.

Diagnosing The Problem

Before you begin: This document intersects some of the same material as Technote 1981436: QRadar: Troubleshooting tunnels and SSH issues in QRadar 7.2.5 and later, which is focused on resolving encrypted host communications and SSH connectivity issues. The focus of which is on higher layer connectivity issues that are a part of QRadar configuration. This article in turn is the interpretation of various responses of the SSH protocol as part of a larger Layer 1, 2 and 3 troubleshooting task.

You can use an SSH connection to verify network connectivity on a host. You would first open an SSH connection to your console first, and from there open an SSH session to your targeted Managed Host.

Successful Connection: If you are able to connect without any issues, this shows that most (but not all) Layer 1 and Layer 2 networking issues can be eliminated as the possible cause of your symptoms.

If you are receiving error messages, or you are experiencing stability issues, responsiveness issues, this usually indicates problems in the network.


Layer 4 Problems / Authentication Errors: Various SSH authentication errors are considered Layer 4 SSH problems. Layer 4 problems with SSH are very much relevant as they interfere with the correct operation of QRadar. However, these layer 4 problems can be considered problems with the QRadar configuration or integration. They do not necessarily indicate anything that is wrong with the rest of your network environment. Some examples of these Layer 4 problems include receiving a Password Prompt or a Remote Host Identification Changed message.

Instability and Performance Problems: Instability and performance problems usually (bot not always) indicate the presence of more complex layer 1 issues such as insufficient bandwidth, link layer aggregation (bonding/NIC teaming) misconfiguration, MTU problems, and so on.

Other Error Messages:

A simplified way of thinking about an SSH connection, especially in regards to Layers 1, 2 and 3 can be picturing that your console is sending messages called packets and the Managed Host responding to these with packets of its own. Understanding where this simplified model is failing can help you pinpoint the problem. The three most common error messages encountered are the following:
  • Connection Timed Out: The connection that is timed out message means that the Console successfully sent packets onto the network but did not receive a response in a reasonable period. This can mean that the packets console sent were either not received or perhaps the response was lost. The presence of this error message proves that you are facing a communication issues in lower layers and most likely in layers 2 or 3. The typical Layer 3 problem that causes this type of an issue is a firewall blocking your connection and dropping your packets without any notification. A typical Layer 2 issue is an incorrect gateway either on your console or your Managed Host. Other types of gateway/router problems on your network can also cause these.
  • Connection Refused: The connection refused message indicates that the Console sent packets onto the network and received a response but this response was a rejection of the connection attempt. This can happen if the packet was received by an application that is configured to refuse your connection attempt such as a firewall. In other words, you are likely facing layer 3.
  • No Route to Host: This message indicates that the Console does not know how to reach the targeted Managed Host. This is usually an indication of a layer 2 or a layer 1 problem. Most likely causes include an incorrect gateway or subnet mask configurations, cabling, or other hardware problems on your Console.

Resolving The Problem

If you are able to access the host by using SSH, this can be used to work around potential Layer 4 and some Layer 3 issues: Non-encrypted communications in QRadar utilize various transport layer ports. On the other hand, when using the encryption option, all transport layer communications are done over SSH tunnels and thus over TCP port 22, which is verified to work correctly with SSH.

However, If you are facing issues with your SSH connection however, you need to take further actions to focus your troubleshooting. Since communication occurs between two endpoints, the QRadar Console and your targeted Managed Host, it is best to have access to both end points during troubleshooting.

For this purpose, it is best to utilize the Integrated Managed Modules (IMM) that QRadar appliances come equipped with. It allows for out of band management of devices even when the management network access is not possible. Technote 1974628: QRadar: Managing QRadar Appliances with IMM can provide further information if needed.

First, ensure that you have out of band access to the targeted Managed Host. Next, try an SSH connection from the Console to the Managed Host. This provides further hints about which networking layer has a problem. The results of this attempt can be interpreted in much the same way as discussed in the Diagnosing the Problem section. However, note, SSH connections can legitimately be blocked from the Managed Hosts to the Console and even if they are not, you are normally prompted for a password.

Example: When attempting to SSH into the Managed Host A from the Console C, you are receiving a Connection Timed Out error. This indicates to you that you are most likely facing a layer 2 or a layer 3 problem. Then, you connect to the Managed Host A using its IMM and attempt to connect to the Console C from managed host A and you receive a "No Route to Host" problem.

If you are unsure about what problem you are facing, even after the SSH connection attempt at the reverse direction, you should have a clear indication of your next steps:

Layer 4 Problems / Authentication Errors: Technote 1981436: QRadar: Troubleshooting tunnels and SSH issues in QRadar 7.2.5 and later provides information about troubleshooting these issues.


[{"Product":{"code":"SSBQAC","label":"IBM QRadar SIEM"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"General Information","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.2","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
21 October 2021

UID

swg22002165