Topic
  • 2 replies
  • Latest Post - ‏2013-02-07T18:53:42Z by SystemAdmin
SystemAdmin
SystemAdmin
3234 Posts

Pinned topic Nortel Switch Module / Linux Configuration Question

‏2013-02-01T17:49:39Z |
I have a BladeCenter with two Nortel Networks Layer2-3 GbE Switch Modules (Copper). The switch in bay 2 is working fine with two trunk groups (EXT1-4 and EXT5-6) using all 6 ports. My issue is with bay 1 configuration (all notes below relate only to bay 1, not bay 2). I want to use EXT1 and EXT2 on the switch in bay 1, each on a different subnet:

EXT1: 172.23.111.0/24
EXT2: 172.23.110.0/24

Both are connected to a Cisco router; vlan 111 for EXT1 and vlan 110 for EXT2. On the GbE switch, I have:

Vlan 2111: INT1-14, EXT1, EXT3-6
Vlan 2110: INT1-14, EXT2

The blades are all running linux. The eth1 device maps to one of the two trunk groups for bay 2, and works just fine. The eth0 device maps to EXT1, and works just fine for the 172.23.111.0 subnet. My assumption on the linux side is to configure the second IP as an alias for eth0. The problem is that I cannot get to the 172.23.110.0 subnet outside of the switch module; pings to the gateway IP fail (Destination Host Unreachable), yet pings to another blade that has a 172.23.110.0 subnet IP work fine.

I'm wondering if there is an issue with my switch module config or with my linux config. The attached txt file has the configuration for both the switch module and ethernet for linux (note that same behavior results whether or not I include the VLAN=yes option).

Any thoughts?

Attachments

Updated on 2013-02-07T18:53:42Z at 2013-02-07T18:53:42Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    3234 Posts

    Re: Nortel Switch Module / Linux Configuration Question

    ‏2013-02-05T18:06:21Z  
    Figured it out; I was on the right track. VLAN tagging has to be enabled on the linux blades needing the second subnet. Note: VLAN tagging has to be enabled on any INT port on the switch module for each blade that needs access to the VLAN; tagging is not needed on the EXT ports, unless the EXT port is part of a trunk that allows multiple VLAN traffic (this would be configured on the external router). In addition, the VLAN id has to match that of the router; I had mine incorrectly set to 2110 and 2111 (should have been 110 and 111). On the linux (Centos 6) side, there is some additional ip route/rule work that had to take place. See the following for details.

    Linux config:

    /etc/sysconfig/network

    ...
    GATEWAYDEV=eth0.111
    ...

    /etc/sysconfig/network-scripts/ifcfg-eth0

    DEVICE=eth0
    BOOTPROTO=none
    HWADDR=00:14:5E:BD:66:94
    IPV6INIT=no
    IPV6_AUTOCONF=no
    ONBOOT=yes
    TYPE=Ethernet

    /etc/sysconfig/network-scripts/ifcfg-eth0.110

    DEVICE=eth0.110
    BOOTPROTO=static
    IPADDR=172.23.110.32
    NETMASK=255.255.255.0
    NETWORK=172.23.110.0
    GATEWAY=172.23.110.254
    BROADCAST=172.23.110.255
    ONBOOT=yes
    TYPE=Ethernet
    VLAN=yes

    /etc/sysconfig/network-scripts/ifcfg-eth0.111

    DEVICE=eth0.111
    BOOTPROTO=static
    IPADDR=172.23.111.32
    NETMASK=255.255.255.0
    NETWORK=172.23.111.0
    GATEWAY=172.23.111.254
    BROADCAST=172.23.111.255
    ONBOOT=yes
    TYPE=Ethernet
    VLAN=yes

    /etc/sysconfig/network-scripts/route-eth0.110

    default via 172.23.110.254 dev eth0.110 table vlan110

    /etc/sysconfig/network-scripts/route-eth0.111

    default via 172.23.111.254 dev eth0.111 table vlan111

    /etc/sysconfig/network-scripts/rule-eth0.110

    from 172.23.110.32 tab vlan110 priority 500

    /etc/sysconfig/network-scripts/rule-eth0.111

    from 172.23.111.32 tab vlan111 priority 500

    /etc/iproute2/rt_tables

    ... (just added these at the end)
    110 vlan110
    111 vlan111
  • SystemAdmin
    SystemAdmin
    3234 Posts

    Re: Nortel Switch Module / Linux Configuration Question

    ‏2013-02-07T18:53:42Z  
    Figured it out; I was on the right track. VLAN tagging has to be enabled on the linux blades needing the second subnet. Note: VLAN tagging has to be enabled on any INT port on the switch module for each blade that needs access to the VLAN; tagging is not needed on the EXT ports, unless the EXT port is part of a trunk that allows multiple VLAN traffic (this would be configured on the external router). In addition, the VLAN id has to match that of the router; I had mine incorrectly set to 2110 and 2111 (should have been 110 and 111). On the linux (Centos 6) side, there is some additional ip route/rule work that had to take place. See the following for details.

    Linux config:

    /etc/sysconfig/network

    ...
    GATEWAYDEV=eth0.111
    ...

    /etc/sysconfig/network-scripts/ifcfg-eth0

    DEVICE=eth0
    BOOTPROTO=none
    HWADDR=00:14:5E:BD:66:94
    IPV6INIT=no
    IPV6_AUTOCONF=no
    ONBOOT=yes
    TYPE=Ethernet

    /etc/sysconfig/network-scripts/ifcfg-eth0.110

    DEVICE=eth0.110
    BOOTPROTO=static
    IPADDR=172.23.110.32
    NETMASK=255.255.255.0
    NETWORK=172.23.110.0
    GATEWAY=172.23.110.254
    BROADCAST=172.23.110.255
    ONBOOT=yes
    TYPE=Ethernet
    VLAN=yes

    /etc/sysconfig/network-scripts/ifcfg-eth0.111

    DEVICE=eth0.111
    BOOTPROTO=static
    IPADDR=172.23.111.32
    NETMASK=255.255.255.0
    NETWORK=172.23.111.0
    GATEWAY=172.23.111.254
    BROADCAST=172.23.111.255
    ONBOOT=yes
    TYPE=Ethernet
    VLAN=yes

    /etc/sysconfig/network-scripts/route-eth0.110

    default via 172.23.110.254 dev eth0.110 table vlan110

    /etc/sysconfig/network-scripts/route-eth0.111

    default via 172.23.111.254 dev eth0.111 table vlan111

    /etc/sysconfig/network-scripts/rule-eth0.110

    from 172.23.110.32 tab vlan110 priority 500

    /etc/sysconfig/network-scripts/rule-eth0.111

    from 172.23.111.32 tab vlan111 priority 500

    /etc/iproute2/rt_tables

    ... (just added these at the end)
    110 vlan110
    111 vlan111
    A final note... with firmware 1.0.1.1, it stopped working when I reset the IO modules. I installed 1.5.4.0. The issue was corrected by enabling "PVID tagging" for the internal port. Note that a couple blades would not work with this enabled, different linux version I guess; not all of my blades were configured with multiple vlans.