QRadar SOAR: Invalid host name causes App Host/Edge Gateway problems

Troubleshooting

Problem

An App Host/Edge Gateway connected to IBM QRadar SOAR did not work properly after the virtual machine was cloned and a new virtual machine used the cloned image.

Symptom

The App Host/Edge Gateway would not create the cni0 interface successfully so the App Host/Edge Gateway did not show as on line and the pods did not start as expected. No applications worked.

Cause

After the cloned virtual machine was started, the IP address and host name were changed. The host name was changed to include an underscore. RFC 1123 does not allow the use of underscores in host names. Kubernetes did not like the invalid host name and would not start.

Environment

The client cloned a production App Host/Edge Gateway 1.13 server so they could test it in a non-production environment.

Diagnosing The Problem

The client tried to manually create the cni0 interface but that did not work.

curl -v -k https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/76/7675586df687972b960134ddaf042c570c895bf1fbdf9fc0ce0bf09c1e1c2811/data?verify=1628763245-LH6DFryld4q9saENHZjacT0yYvo%3D
sudo k3s ctr images pull -k docker.io/rancher/pause:3.1 --image-pull-policy Always
sudo systemctl restart k3s
sudo restartAppHost
sudo netstat -nr

The process to manually create the cni0 interface does not work if the proxy or network is not configured to allow access to all the external domains stipulated in the product documentation.

The client also deployed a new App Host/Edge Gateway OVA but the problem persisted.

Further investigation of App Host diagnostics by using https://www.ibm.com/support/pages/node/6338707, the file, journalctl.txt showed this error.

Dec 13 06:18:44 apphost_test k3s[4922]: E1213 06:18:44.929185    4922 kubelet_node_status.go:92] "Unable to register node with API server" err="Node \"apphost_test\" is invalid: metadata.name: Invalid value: \"apphost_test\": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')" node="apphost_test"
Dec 13 06:18:44 q0resilientapp_test k3s[4922]: E1213 06:18:44.988086    4922 kubelet.go:2424] "Error getting node" err="node \"apphost_test\" not found"

Checking RFC 1123 confirmed that the use of an underscore is invalid.

Kubernetes documentation confirms that the underscore is invalid -> https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names

RFC 1123 Label Names

Some resource types require their names to follow the DNS label standard as defined in RFC 1123. This means the name must:

Contain at most 63 characters
Contain only lowercase alphanumeric characters or '-'
Start with an alphanumeric character
End with an alphanumeric character

Resolving The Problem

The client changed the server host name, removing the underscore and restarted App Host/Edge Gateway and k3s.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSA230","label":"IBM Security QRadar SOAR"},"ARM Category":[{"code":"a8m0z0000001jTpAAI","label":"Integrations-\u003EAppHost"}],"ARM Case Number":"TS014841346","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEGM63","label":"IBM Security QRadar SOAR on Cloud"},"ARM Category":[{"code":"a8m0z0000001jTpAAI","label":"Integrations-\u003EAppHost"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSTDPP","label":"IBM Cloud Pak for Security"},"ARM Category":[{"code":"a8m0z0000001h8uAAA","label":"Cloud Pak for Security (CP4S)-\u003EInstall or Upgrade"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Tips