Cabling the InfiniBand network

Use this procedure to cable your InfiniBand network.

It is possible to perform some of the tasks in this procedure by using a method other than that which is described. If you have other methods for cabling the InfiniBand network, you still must review a few key points in the installation process regarding order and coordination of tasks and configuration settings that are required in a cluster environment.

Note: IBM® is responsible for replacing faulty or damaged cables that have IBM part numbers.

To cable your switch network, complete the following steps:

Note: The switch cabling encompasses major tasks C1 and C4, which are shown in Figure 1.

Obtain and review a copy of the cable plan for the InfiniBand network.
Label the cable ends before routing the cable.
Power on the switches before attaching cables to them.
C1 - Route the InfiniBand cables according to the cable plan and attach them to only the switch ports. Refer to the switch vendor documentation for more information about how to plug cables.
C4 - Connect the InfiniBand cables to the host channel adapter (HCA) ports according to the planning documentation.
If both servers and switches have power applied as you complete cable connections, you should check the port LEDs while you plug in the cables. See the switch vendor's Switch Users Guide to understand the correct LED states. Fabric management can now be started.
Note: Depending on assigned installation responsibilities, it is possible that someone else might need to perform these actions. Coordinate this with the appropriate people.
- For QLogic embedded subnet managers, use the smControl start command, the smPmBmStart enable command, and the smConfig startAtBoot yes command. The latter command can be issued at the switch command line, or by using the Fast Fabric cmdall command. QLogic host-based Fabric Manager uses the iview_fm start command as instructed in Installing the fabric management server. Contact the person installing the fabric management server and indicate that the Fabric Manager should not be started on the fabric management server.
Note: If you are responsible for verifying the InfiniBand network topology and operation, you can proceed to that procedure.