Setting up the IP interfaces on the switch on a RoCE network
(Linux®)
On a remote direct memory access (RDMA) over Converged Ethernet
(RoCE) network, you must manually set up the IP interfaces on the switch.
Starting from V11.1.4.4,
the procedures documented in this page are no longer required as adapter port liveliness test has
been enhanced and automated. Some restrictions apply. Refer to technote#0733765 for
restrictions.
Before you begin
Ensure that you completed the following tasks:
The switch to be configured must be turned on.
You must be able to administrate the switch through a console.
For example, a host that is connected to the switch's management port.
You must have administrative access to the switch.
The switch must be setup with an IP for its management port and
it must be running.
About this task
The number of IP addresses to be setup on a switch is
the same as the number of distinct IP subnets directly connected to
it from the CFor member server.
Procedure
To set up the IP interfaces:
Log on to the command-line interface of the switch with
the admin user ID and password.
Create the interface and setup its IP address using these
guidelines:
Create the same number of IP interfaces on a switch as the number
of distinct IP subnets connected from a CF
or member host.
(For example, in the figure below, each switch has two distinct IP
subnets connected from the CF host.)
Each IP interface must be assigned an IP address that is on one
of the distinct IP subnets from the CF host.
No IP subnet can be used more than once on a switch.
For a single switch cluster, the setup is now completed.
If the cluster in Figure 1 only has one switch, after completing steps
1 to 2, the switch will have the following IP addresses assigned to
their IP interfaces :
192.168.1.2, 192.168.2.2, 192.168.3.2 and 192.168.4.2.
The
final step is to verify that the newly created IP interfaces are pingable
from each other and each adapter in members and CFs can
ping the external pingable IP address(es) specified for that adapter
in their netmon.cf.
For a two-switch cluster where at least 2 adapter ports are used in each CF and member, repeat
steps 1 and 2 on the other switch. At the end of this step, there should be an equal number of IP
interfaces on each switch with each IP address on a different IP subnet. The total number of
distinct IP subnets on both switches is the same as the maximum of the total number of distinct IP
subnets on a CF or member host.
Using Figure 1 as an example, after completing steps 1 to 4, each switch will have the following
IP addresses assigned to their IP interfaces:
Switch 1 - 192.168.1.2 and 192.168.2.2
Switch 2 - 192.168.3.2 and 192.168.4.2
For a two-switch cluster where at least 2 adapter ports are used in the CF but only a single
adapter port is used in each member, create an additional IP interface on the switch where its newly
created IP interfaces do not have the same IP subnet as the member's adapter. Assign this new IP
interface an IP address that is on the same IP subnet as the member. After this step, one switch
should have one extra IP interface than the other. In addition, both switches will have exactly one
IP interface where its IP address is on the same IP subnet as all the members.
Note: The final step is to verify the newly created IP interfaces are pingable from each other and
each adapter in members and CFs can ping the external
pingable IP addresses specified for that adapter in their netmon.cf.
Example
Below is the step-by-step instructions on how to create the IP interfaces on a
cluster using two BNT switches :
Log in with the admin user ID.
Create the interface and setup its IP address and IP subnet mask.
If you are creating two interfaces on the switch:
To create interface 1, setup the IP address and IP subnet mask. For
example:
