IBM Support

Flex p-node: Linux ports eth0-eth7 change to eth8-eth15 when IOFM is enabled - IBM Flex System

Troubleshooting


Problem

[This abstract has been truncated due to length constraints] The Ethernet ports output from 'ifconfig -a' will change values if IBM Open Fabric Manager (IOFM) is enabled/disabled for a power compute node. With an IOFM configuration disabled the ethernet ports are eth0-eth7. For example: [root@localhost approximately]# ifconfig -a eth0 Link encap:Ethernet HWaddr 00:90:FA:07:F5:EE BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) eth1 Link encap:Ethernet HWaddr 00:90:FA:07:F5:EF BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (

Resolving The Problem

Source

RETAIN tip: H211647

Symptom

The rthernet ports output from 'ifconfig -a' will change values if IBM Open Fabric Manager (IOFM) is enabled/disabled for a power compute node.

With an IOFM configuration disabled the ethernet ports are eth0-eth7. For example: [root@localhost approximately]# ifconfig -a

eth0 Link encap:Ethernet HWaddr 00:90:FA:07:F5:E
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth1 Link encap:Ethernet HWaddr 00:90:FA:07:F5:EF
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth2 Link encap:Ethernet HWaddr 00:90:FA:07:F5:F0
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth3 Link encap:Ethernet HWaddr 00:90:FA:07:F5:F1
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth4 Link encap:Ethernet HWaddr 00:90:FA:07:F5:F7
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth5 Link encap:Ethernet HWaddr 00:90:FA:07:F5:F8
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth6 Link encap:Ethernet HWaddr 00:90:FA:07:F5:F9
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth7 Link encap:Ethernet HWaddr 00:90:FA:07:F5:FA
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2 errors:0 dropped:0 overruns:0 frame:0
TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:100 (100.0 b) TX bytes:100 (100.0 b)
 

With an IOFM configuration enabled the ethernet ports change to eth8-eth15. For example: [root@localhost approximately]# ifconfig -a

eth8 Link encap:Ethernet HWaddr 00:00:00:00:00:04
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth9 Link encap:Ethernet HWaddr 00:00:00:00:00:05
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth10 Link encap:Ethernet HWaddr 00:00:00:00:00:02
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth11 Link encap:Ethernet HWaddr 00:00:00:00:00:06
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth12 Link encap:Ethernet HWaddr 00:00:00:00:00:07
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth13 Link encap:Ethernet HWaddr 00:00:00:00:00:08
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth14 Link encap:Ethernet HWaddr 00:00:00:00:00:03
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth15 Link encap:Ethernet HWaddr 00:00:00:00:00:01
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2 errors:0 dropped:0 overruns:0 frame:0
TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:100 (100.0 b) TX bytes:100 (100.0 b)

Affected configurations

The system may be any of the following IBM servers:

  • Flex System p24L Compute Node, type 1457, any model
  • Flex System p260 Compute Node, type 7895, any model
  • Flex System p270 Compute Node, type 7954, any model
  • Flex System p460 Compute Node, type 7895, any model

The system is configured with at least one of the following:

  • Red Hat Enterprise Linux 6, any update
  • SUSE Linux Enterprise Server 11, any service pack

This tip is not option specific.

Note: This does not imply that the network operating system will work under all combinations of hardware and software.

Please see the compatibility page for more information: http://www.ibm.com/systems/info/x86servers/serverproven/compat/us/

Workaround

Edit '/etc/udev/rules.d/70-persistent-net.rules' and have two (2) entries that specify the same network device name. One (1) entry will be to match the factory assigned Media Access Control (MAC) address when IOFM is disabled and the other to match the machine unique MAC address when IOFM is enabled. Similar entries will exist for each of the ports.

For example, when the issue occurs, the '/etc/udev/rules.d/70-persistent-net.rules' file will have two (2) entries such as the ones below, which correspond to the same physical ethernet port. One (1) calls the port eth0 and the other calls it eth15.

  # PCI device 0x10df:0xe220 (be2net)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:90:fa:07:f5:ee", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x10df:0xe220 (be2net)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:00:00:00:00:01", ATTR{type}=="1", KERNEL=="eth*", NAME="eth15"

In order to work around this issue, these entries should be changed to the following:

  # PCI device 0x10df:0xe220 (be2net)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:90:fa:07:f5:ee", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x10df:0xe220 (be2net)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:00:00:00:00:01", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

Additional information

When IOFM is enabled, the MAC Address changes from its factory assigned value to a new, machine specific one. The udev rules file '/etc/udev/rules.d/70-persistent-net.rules' that reserves and assigns a persistent device name, for example, eth0, did not match the new MAC address and so there was a new device name assigned instead.

The workaround alleviates the symptom by ensuring both the factory assigned MAC Address and the new, machine specific MAC Address are associated with the same ethernet device name.

Document Location

Worldwide

Operating System

PureFlex System and Flex System:SUSE Linux Enterprise Server 11 x86-64

PureFlex System and Flex System:Red Hat Enterprise Linux 6 x86-64

[{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU02BWP","label":"PureFlex System and Flex System->p270 Compute Node->7954"},"Platform":[{"code":"PF042","label":"Caldera"},{"code":"PF047","label":"SurePOS"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU91NSG","label":"PureFlex System and Flex System->p24L Compute Node->1457"},"Platform":[{"code":"PF042","label":"Caldera"},{"code":"PF047","label":"SurePOS"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU91NSI","label":"PureFlex System and Flex System->p260 Compute Node->7895"},"Platform":[{"code":"PF042","label":"Caldera"},{"code":"PF047","label":"SurePOS"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU91NSK","label":"PureFlex System and Flex System->p460 Compute Node->7895"},"Platform":[{"code":"PF042","label":"Caldera"},{"code":"PF047","label":"SurePOS"}],"Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
30 January 2019

UID

ibm1MIGR-5094355

Page Feedback