Clustering systems using InfiniBand hardware

IBM® Systems server hardware supports clustering through Host Channel Adapters (HCAs) and InfiniBand switches. The guide to Clustering systems using InfiniBand hardware, provides planning and installing information to help guide you through the process of installing a cluster fabric that incorporates these switches. Also in this guide, there is information about how to manage and service a cluster that uses InfiniBand technology.

The following illustration shows servers that are connected in a cluster configuration with InfiniBand switch networks (fabric). The servers in these networks can be connected through switches with either Peripheral Computer Interconnect (PCI) HCAs or GX Remote Input Output (RIO) adapter HCAs. Not all configurations support the following network configuration; refer to your IBM sales information for supported configurations.

Figure 1. An HMC managed InfiniBand network with four switches (fabric) and four servers connected.

The following switch models are available through IBM:

7048-120 (Topspin 120 Server Switch)

The following switch models are available through Cisco:

7048-270 (Topspin 270 Server Switch)
Cisco SFS 7000 Server Switch
Cisco SFS 7008 Server Switch

If you are planning to use an InfiniBand switch in your clustered system configuration, the following table provides a list of InfiniBand hardware and software components available to help you.

Table 1. InfiniBand network hardware solutions
Solution components	GX Host channel adapter
Adapter	GX Dual-port 12x HCA
Systems	IBM System p5®, mid range, and high-end
Switches	Topspin 120, Cisco 7000, Topspin 270, Cisco 7008
Cables	IBM certified cables
Fabric Management	IBM Network Manager
AIX version	AIX 5L™ Version 5.3 with the 5300-04 Technology Level plus APAR IY84727.
Linux version	SUSE Linux® Enterprise Server 9 (SP3) with IBM GX HCA driver and Open Fabrics Enterprise Distribution

Note: See the Facts and Features Web site for the most recent information regarding cluster offerings.

Information resources

If you are planning to set up a clustered server configuration with InfiniBand switches, the information in this topic helps guide you through the process. This section also contains a framework of information to help you or your service representative to diagnose and resolve InfiniBand switch-network (fabric) problems. The following documentation is available here for your convenience:

Planning for InfiniBand networks

This documentation, Guide to Clustering systems using InfiniBand hardware, provides planning and installing information to help guide you through installing a cluster fabric using InfiniBand switches
Managing your server using the Hardware Management Console (HMC)
Topspin 120/Cisco SFS 7000 Hardware Guide
Topspin 270/Cisco SFS 7008 Hardware Guide
Worldwide Customized Installation Instructions (WCII), service representative install instructions: http://w3.rchland.ibm.com/projects/WCII
AIX Information Center documentation

Installing InfiniBand networks

The following guides are available to help you set up your network:

This documentation, Clustering systems using InfiniBand hardware, provides planning and installing information to guide you through the process of installing a cluster fabric using InfiniBand switches.
InfiniBand Hardware Installation and Cabling Guide Web Release, Topspin number: 10-00122-WEB
Topspin 120/Cisco SFS 7000 Quick Start Guide, order number: 10-00033-04-A0
Topspin 270/Cisco SFS 7008 Quick Start Guide , order number: 10-00045-04-A0

Administering and servicing InfiniBand networks

If you need to do administrative tasks on your InfiniBand cluster, the following guides are available to help you. This documentation is also intended for use by service personnel:

This documentation, Guide to Clustering systems using InfiniBand hardware, provides a guide to help administer and service a cluster fabric using InfiniBand switches.
Element Manager User Guide, order number: 10-00116-02-A0
Chassis Manager User Guide, order number: 10-00029-05-A0
Command Line Interface Reference Guide, order number: 10-00012-07-A0
Host-side Drivers User Guide for Linux, order number: 10-00125-02-A0