IBM Support

QRadar: Troubleshooting iptables issues

Troubleshooting


Problem

Errors in the iptables and ip6tables service might lead to issues such as adding managed hosts, applications not starting, or working as expected, deploy changes timing out, and patches failing after the pretests run.
This article guides administrators through identifying and resolving common issues in the iptables service in QRadar®. 

Symptom

The symptoms vary depending on the process where the iptables service is called. Common symptoms are:
  • The xtables.lock is present.
    Note: However, this file can interfere with other services, but the message is the same.
     
iptables.init[28455]: Another app is currently holding the xtables lock; still 1s 0us time ahead to have a chance to grab the lock...
iptables.init[28455]: Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
iptables.init[28455]: [FAILED]
systemd[1]: iptables.service: main process exited, code=exited, status=1/FAILURE
  • Bad syntax in the iptables files.
Job for iptables.service failed because the control process exited with error code. See "systemctl status iptables.service" and "journalctl -xe" for details.

iptables.init[17589]: iptables: Applying firewall rules: iptables-restore: line 114 failed
iptables.init[17589]: [FAILED]
  • QRadar® patch reports the error:
 [INFO](testmode) Running pretest 10/11: Check for invalid iptables rules
Unexpected error occurred while running /opt/qradar/bin/iptables_update.pl.
[ERROR](testmode) Patch pretest 'Check for invalid iptables rules' failed. (check_iptables_rules.sh)
[ERROR](testmode) Failed 1/11 pretests. Aborting the patch.
[ERROR](testmode) Failed pretests
[ERROR](testmode) Pre Patch Testing shows a configuration issue. Patching this host cannot continue.

Cause

The iptables service is in a failed state and cannot start. The most common causes for iptables not to start are:
  1. The xtables.lock was not released by another application or service. 
  2. Bad syntax in any of the iptables files.
  3. typographical errors in any of the iptables files.
  4. Bad characters such as empty spaces in any of the iptables files.

Environment

QRadar® all 7.3 and 7.4 versions.

Diagnosing The Problem

The xtables.lock was not released by another application or service. 
To diagnose this cause, the administrator must check the date reported in the xtables.lock file. If the date differs from the actual time of the Operating System, then it is considered a leftover file and must be removed.
  1. Review the date of the xtables.lock.
     
    ls -l /run/xtables.lock
    -rw-------  1 root     root         0 Feb 19 02:49 xtables.lock
  2. Validate the time against the OS time.
     
    date
    Tue May 18 18:47:58 EDT 2021
Bad syntax, typographical errors, and empty characters in any of the iptables files.
To diagnose this cause, the administrator must check the line reported to have the error to determine whether a bad syntax, a bad typographical error, or empty characters are causing the issue.
  1. Attempt to update the services and make note of the service.
     
    /opt/qradar/bin/iptables_update.pl
    
    ERROR: There was an issue reloading iptables. Check /var/log/qradar-iptables.log for details.
  2. Review the journal logs of the iptables or ip6tables service and make note of the conflicting line.
     
    For IPv4 iptables service:
    journalctl -xu iptables -n 10 --no-pager
    
    For IPv6 iptables service
    journalctl -xu ip6tables -n 10 --no-pager
    
    Error message:
    iptables: Trying to reload firewall rules: iptables-restore: line 114 failed
    
  3. Identify the line and the file where it's configured.
     
    For IPv4 iptables service:
    cat -An /etc/sysconfig/iptables | grep -E 'From file|End file|<line number>' | grep <line number> -C 1
    
    For IPv6 iptables service:
    cat -An /etc/sysconfig/ip6tables | grep -E 'From file|End file|<line number>' | grep <line number> -C 1
Examples:
For readability purposes, the following examples are run against /etc/sysconfig/iptables. The administrator might see similar output when the issue lays in the /etc/sysconfig/ip6tables file.
 
typographical error (IMPUT instead of INPUT) in line 114 evidencing the conflicting line is configured in the /opt/qradar/conf/iptables.post file.
 cat -An /etc/sysconfig/iptables | grep -E 'From file|End file|114' | grep 114 -C 1
   105  ### From file /opt/qradar/conf/iptables.post$
   114  -A IMPUT -p udp --dport 8001 -j REJECT$
   117  ### End file /opt/qradar/conf/iptables.post$
Bad character (^I ) in line 117 evidencing the conflicting line is configured in the /opt/qradar/conf/iptables.post file.
 
cat -An /etc/sysconfig/iptables | grep -E 'From file|End file|117' | grep 117 -C 1
   105  ### From file /opt/qradar/conf/iptables.post$
   117  ^I $
   119  ### End file /opt/qradar/conf/iptables.post$
Bad format (termination of file misplaced) in line 116 evidencing the conflicting line is configured in the /opt/qradar/conf/iptables.post file.
cat -An /etc/sysconfig/ip6tables | grep -E 'From file|End file|116' | grep 116 -C 1 
   105  ### From file /opt/qradar/conf/iptables.post$
   116  -A OUTPUT -p udp --dport 8001 -j REJECT# END /opt/qradar/conf/iptables.post FILE$
   117  ### End file /opt/qradar/conf/iptables.post$

Resolving The Problem

Depending on the cause, the administrators must perform the respective corrective actions:

  1. The xtables.lock was not released by another application or service. 
    1. If the xtables.lock date differs from the actual time, the file can be removed without further impact.
       
      rm -fv /run/xtables.lock
    2. Run the /opt/qradar/bin/iptables_update.pl script to reload the iptables and ip6tables services.
       
      /opt/qradar/bin/iptables_update.pl

       
  2. Bad syntax and typographical errors in any of the iptables files.

    Note:
     In the following steps, line 1 of the file /opt/qradar/conf/iptables.pre is used as an example. The administrator must take a backup that matches their environment.
    1. The administrator must refer to the Linux man page for the appropriate syntax.
      1. IPv4 iptables service
      2. IPv6 iptables service.
         
    2. Back up the existing file.
       
      mkdir -p /store/IBM_Support/
      cp -fv /path/to/conflicting/file /store/IBM_Support/
      
      Example:
      cp -fv /opt/qradar/conf/iptables.pre /store/IBM_Support/
    3. Fix the typographical error by editing the file.
       
      vi /opt/qradar/conf/iptables.pre
    4. Run the /opt/qradar/bin/iptables_update.pl script to reload the iptables and ip6tables services.
       
      /opt/qradar/bin/iptables_update.pl
  3. Bad characters in any of the iptables files.

    The conflicting line from the file must be removed.

    Note: In the following steps, line 1 of the file /opt/qradar/conf/iptables.pre is used as an example. The administrator must take a backup that matches their environment.
    1. Back up the existing file.
       
      mkdir -p /store/IBM_Support/
      cp -fv /path/to/conflicting/file /store/IBM_Support/
      
      Example:
      cp -fv /opt/qradar/conf/iptables.pre /store/IBM_Support/
    2. Verify the line in the conflicting file. 
       
      cat -An /opt/qradar/conf/iptables.pre
           1  ^I $
      
    3. Remove the line in the file.
       
      sed -i '1d' /opt/qradar/conf/iptables.pre
    4. Run the /opt/qradar/bin/iptables_update.pl script to reload the iptables and ip6tables services.
       
      /opt/qradar/bin/iptables_update.pl
Result
The iptables and ip6tables service must start without any error reported after running the iptables_update.pl script.
 
>>> Shutting down existing firewall...
/tmp/iptables.3898
/tmp/ip6tables.3898
>>> Beginning update...
 Writing out rules for web access...
>>> IPTables update complete. Reloading firewall...
>>> Done!
>>> IP6Tables update complete. Reloading firewall...
>>> Done!
Finished starting ipv6
Done iptables update.
If administrators continue to experience issues with the iptables and ip6tables service not starting, contact QRadar Support.

Document Location

Worldwide

[{"Type":"SW","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtNAAQ","label":"Deployment"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Version(s)"}]

Document Information

Modified date:
24 May 2021

UID

ibm16453981