IBM Support

IBM AIX How-to: Resolving multiple default routes in the network routing table

Question & Answer


Question

I have multiple default routes in my AIX routing table. What does this additional route indicate? How do I keep the main default route?

Cause

Routing in AIX determines the set of rules that govern when connections are made from AIX to other systems. Whenever a connection is made from AIX to another system, AIX routes to that system. Since that connection doesn't go directly to the other system, it might go through other systems and nodes in between. These nodes are most likely a router, or a series of routers to make it possible for AIX to get to the other host.

To view the routing table on AIX, the following two commands can be used:

# netstat -rn
Or

# netstat -r
*Tip: #netstat -r can be used with the -n in name service debugging. If there is an issue with name service and, as a result, this translation from a hostname to IP address doesn't work, the #netstat -r command never returns an output. The -n flag suppresses the IP address to hostname mapping and shows the result in numerical form.

Running #netstat -rn shows the following output:
# netstat -rn
Routing tables
Destination        Gateway           Flags   Refs     Use  If   Exp  Groups
Route Tree for Protocol Family 2 (Internet):
default            9.40.205.1        UG        3    673539 en4      -      -   
9.40.205.0         9.40.205.53       UHSb      0         0 en4      -      -   =>
9.40.205/24        9.40.205.53       U         0     69028 en4      -      -   
9.40.205.53        127.0.0.1         UGHS      2     11173 lo0      -      -   
9.40.205.255       9.40.205.53       UHSb      0         4 en4      -      -   
10.99/16           9.3.4.200         UG        0     38975 en4      -      -   
127/8              127.0.0.1         U         6    329732 lo0      -      -   
Normally, only one 'default' route can be observed.

However, in some scenarios, several default routes can be seen as the following example of running #netstat -rn shows:
 
# netstat -rn
Routing tables
Destination        Gateway           Flags   Refs     Use  If   Exp  Groups
Route Tree for Protocol Family 2 (Internet):
default            9.40.205.1        UG        3    673624 en4      -      -   =>
default            9.40.205.2        UG        0         0 en4      -      -   
9.40.205.0         9.40.205.53       UHSb      0         0 en4      -      -   =>
9.40.205/24        9.40.205.53       U         1     69028 en4      -      -   
9.40.205.53        127.0.0.1         UGHS      2     11173 lo0      -      -   
9.40.205.255       9.40.205.53       UHSb      0         4 en4      -      -   
10.99/16           9.3.4.200         UG        0     38975 en4      -      -   
127/8              127.0.0.1         U         6    329740 lo0      -      -   

The example indicates that 2 default routes are used by en4. That means whenever AIX routes to route packets to the external network it is going to use both the 9.40.205.1 and 9.40.205.2 gateways.

The solution to that problem is to remove the second default gateway from the routing table.

1- A check is made to identify whether the other default gateway is stored in the AIX settings (the ODM). This check is done through running the following command:

# lsattr -El inet0

The output looks similar to the following:

# lsattr -El inet0
authm         65536                         Authentication Methods              True
bootup_option no                            Use BSD-style Network Configuration True
gateway                                     Gateway                             True
hostname      tcp53                         Host Name                           True
rout6                                       IPv6 Route                          True
route         net,-hopcount,0,,0,9.40.205.1 Route                               True

From the output, it is noticeable that AIX loads only one default route. That route is the first default route 9.40.205.1.
Where does the 9.40.205.2 route come from?

Another command that is used to add routes temporarily until system restart is the route command. The route command allows users to make manual entries into the network routing tables. If the system gets restarted, the routing table gets built from the default settings.

In case that the second route gets added to the routing table consistently after restart and doesn't show in the output of lsattr -El inet0:
2- A check can be done to review the execution of the route command in the rc.net file.

During system boot, specifically at phase 2, the 'init' process run the process defined by records in /etc/inittab files. It invokes the configuration manager to configure all of the remaining devices. /etc/inittab is where the /etc/rc.net runs where the network interface devices are initialized and configured. The script invoked is defined in the configuration rules stored in the ODM.

A snippet from the rc.net shows that:

##################################################################
# rc.net - called by cfgmgr during 2nd boot phase.
#
# Configures and starts TCP/IP interfaces.
# Sets hostname, default gateway and static routes.
# Note: all the stdout should be redirected to a file (e.g. /dev/null),
#       because stdout is used to pass logical name(s) back to the cfgmgr
#       to be configured.  The LOGFILE variable specifies the output file.
#
# The first section of rc.net configures the network via the new
#       configuration methods.  These configuration methods require that
#       the interface and protocol information be entered in the ODM
#       database (with either SMIT or the high level configuration commands
#       (mkdev, chdev).
# The second section (commented out) is an example of the equivalent
#       traditional commands used to perform the same function.  You may
#       use the traditional commands instead of the configuration methods
#       if you prefer.  These commands do NOT use the ODM database.
# The third section performs miscellaneous commands which are
#       compatible with either of the previous two sections.
##################################################################

As shown from the description of the script, it configures and starts the TCP/IP interfaces by setting the hostname, default gateway, and static routes. In some situations the following entry is shown in the rc.net:
/usr/sbin/route add default 9.40.205.2

That entry dictates that whenever AIX boots up, it is going to run that route add command to add the 9.40.205.2 default route.

Answer

1- Remove the hidden route by using the route delete command:
#route delete default <Router IP>

In the case that the route still gets added after system restarts:
2- Remove the possible entry in rc.net by either commenting adding a '#' in front of the line that's calling the route command or completely removing the line.
# /usr/sbin/route add default x.x.x.x

Additional Information

SUPPORT

If you require more assistance, use the following step-by-step instructions to contact IBM to open a case for software with an active and valid support contract.  

1. Document (or collect screen captures of) all symptoms, errors, and messages related to your issue.

2. Capture any logs or data relevant to the situation.

3. Contact IBM to open a case:

   -For electronic support, see the IBM Support Community:
     https://www.ibm.com/mysupport
   -If you require telephone support, see the web page:
      https://www.ibm.com/planetwide/

4. Provide a clear, concise description of the issue.

 - For more information, see: Working with IBM AIX Support: Describing the problem.

5. If the system is accessible, collect a system snap, and upload all of the details and data for your case.

 - For more information, see: Working with IBM AIX Support: Collecting snap data

Click here to submit feedback for this document.

Author:
Mustafa Atallah
Reviewed by:
Darshan Patel - Roger Leuckie


[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z000000cvzIAAQ","label":"Networking"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Product Synonym

AIX

Document Information

More support for:
AIX

Component:
Networking

Software version:
All Versions

Document number:
6832258

Modified date:
12 November 2022

UID

ibm16832258

Manage My Notification Subscriptions