Hardware overview of a single rack

Review the overall system layout and hardware configuration.

  • Ensure that IBM Storage Fusion is installed in a restricted access location, such that the area is accessible only to skilled and instructed persons with proper authorization.

System layout

The appliance rack details are as follows:

graphics/hcihardware.png

Rack (Model R42)

42U rack with 2 to 6 PDUs, cabling, and components.

Compute-only server (Model C00 and C04) / Compute-storage server (Model C01 and C05)

The compute-storage servers are the basic building blocks of IBM Storage Fusion HCI System. Each system has a minimum of six Model C01 servers or six Model C05 servers, or three of each.Each of the Model C01 servers have a minimum of two storage drives that can be increased up to a maximum of ten storage drives on each server. It is possible to expand IBM Storage Fusion HCI System beyond the minimum six servers with combinations of Models C00, C01, C04, and C05. Servers can be added to a maximum of 20. (Unless the GPU option is selected, in which case, there can be a maximum of 18 servers.) Each of the servers added to IBM Storage Fusion HCI System is also added to the storage cluster, increasing the total storage capacity. The servers are also combined into a Red Hat® OpenShift® Container Platform. All applications that run on IBM Storage Fusion HCI System are deployed and run within the OpenShift cluster.

RU2 to RU7 rack units
These are control and compute nodes that are available by default. A minimum of three control nodes and three compute nodes with CPU of 32 cores and memory of 256 GB.
Note: The server located in rack position RU7 is used as the provisioning node during the network step up installation stage.
RU8 to RU17
You can purchase and up size storage or compute nodes.
RU29 to RU32
You can purchase and up size storage or compute nodes.
Hardware configuration of compute-storage node (9155-C01) is as follows:
  • Lenovo SR645 server
  • 2x AMD EPYC 7302 16 Cores (32 Cores total), 32 Threads (64 Threads total), 3.0 GHz or 3.3 GHz CPU
  • 2x960 GB M.2 OS drives (RAID 1)
  • 1x NVIDIA ConnectX-6 dual-port 100 GbE network adapter
  • 1x NVIDIA ConnectX-4 dual-port 25 GbE network adapter
  • 1x 1 GbE RJ45 4-port OpenShift Container Platform adapter
  • 2-10x Samsung PM1733 7.68 TB NVMe PCIe 4.0 disks
  • NVMe disks are added in pairs
  • All compute-storage servers must have the same number of NVMe drives
  • The maximum number of compute-storage servers is 20, reduced by the number of GPU servers installed
  • 1U height
Hardware configuration of compute-storage node (9155-C05) is as follows:
  • Lenovo SR645 higher density
  • 2x AMD EPYC 7543 32C (64C total) 256MB L3, 2.8 GHz / 3.7 GHz, CPU 225W
  • Base RAM: 16x 64GB DIMMs (16 GB/core, 1024GB total)
  • Upgraded RAM: 32x 64GB DIMMs (32 GB/core, 2048GB total)
  • 2x 480GB M.2 OS drives (RAID 1)
  • 1x NVIDIA ConnectX-6 dual-port 100GbE network adapter
  • 1x NVIDIA ConnectX-4 dual-port 25GbE network adapter
  • 1x 1GbE RJ45 4-port OCP adapter
  • 2-10x Samsung PM1733 7.68TB NVMe PCIe 4.0 drives
  • 1U height
Hardware configuration of compute-only node (9155-C00) is as follows:
  • Lenovo SR645 server
  • 2x AMD EPYC 7302 16 Cores (32 Cores total), 32 Threads (64 Threads total), 3.0 GHz or 3.3 GHz CPU
  • 256 GB RAM (16x 16 GB DIMMs)
  • 2x 960 GB M.2 OS drives (RAID 1)
  • 1x NVIDIA ConnectX-6 dual-port 100 GbE network adapter
  • 1x NVIDIA ConnectX-4 dual-port 25 GbE network adapter
  • 1x 1 GbE RJ45 4-port OpenShift adapter
  • Same specifications as the compute-storage server but with zero NVMe disks
  • 1U height
Hardware configuration of compute-only node (9155-C04) is as follows:
  • Lenovo SR645 higher density
  • 2x AMD EPYC 7543 32C (64C total) 256MB L3, 2.8 GHz / 3.7 GHz, CPU 225W
  • Base RAM: 16x 64GB DIMMs (16 GB/core, 1024GB total)
  • Upgraded RAM: 32x 64GB DIMMs (32 GB/core, 2048GB total)
  • 2x 480GB M.2 OS drives (RAID 1)
  • 1x NVIDIA ConnectX-6 dual-port 100GbE network adapter
  • 1x NVIDIA ConnectX-4 dual-port 25GbE network adapter
  • 1x 1GbE RJ45 4-port OCP adapter
  • Zero NVMe drives
  • 1U height
Hardware configuration AFM Node (9155-F01):
  • Lenovo SR630 AFM Node (9155-F01)
  • 2x Intel 6242 16C (32C total) 150W 2.8 GHz / 3.9 GHz CPU
  • 192GB RAM (12x 16GB DIMMs)
  • 2x 960GB SATA Hot Swap SSD OS drives (RAID 1)
  • 2x NVIDIA ConnectX-4 dual-port 25GbE network adapter
  • 1x 1GbE RJ45 2-port LOM
  • 1U height
Hardware configuration SR665 GPU Node (9155-G01):
  • Lenovo SR665 GPU Node (9155-G01)
  • 2x AMD EPYC 7F72 24C (48 total) 240W 3.2 GHz / 3.7 GHz CPU
  • 512GB RAM (16x 32GB DIMMs)
  • 2x 960GB M.2 OS drives (RAID 1)
  • 1x NVIDIA ConnectX-6 dual-port 100GbE network adapter
  • 1x NVIDIA ConnectX-4 dual-port 25GbE network adapter
  • 1x 10GBase-T 2-port + 1GbE 2-port RJ45 OCP adapter
  • 2x 2.5" PM1645a 3.2TB Mainstream SAS 12Gb Hot Swap SSD
  • 3x NVIDIA A100 40GB PCIe 4 GPUs
  • 2U height
Networks
IBM Storage Fusion HCI System has two physical networks defined within it: a high-speed network for use by the storage cluster and applications, and a management network that is used for controlling the servers and monitoring the health of the servers.
100GbE High Speed Network Switch (Model S01)

The high-speed network is built around a pair of 32-port, 100Gb Ethernet switches. The switches are configured together using MLAG to create a redundant pair. All of the compute-storage servers and the GPU servers have a 2-port, 100Gb Ethernet adapter. One port on the adapter is connected to the first high-speed switch and the second port is connected to the second high-speed switch. This 100GbE connections are reserved for use by the storage cluster. All of the compute-storage servers and the GPU servers also have a 2-port, 25Gb Ethernet adapter. Using breakout cables that split the 100GbE ports on the switch into four 25GbE ports, one port on the server’s 25 GbE network adapter is connected to the first high-speed switch and the second port is connected to the second high-speed switch. The AFM servers do not have a 2-port, 100 GbE network adapter. Instead, these servers have two of the 2-port 25GbE network adapters. Again, using breakout cables, each of the adapters has one port connected to the first high-speed switch and the other port connected to the other high-speed switch. The 25GbE network connected are intended for use by the Red Hat OpenShift cluster and the applications that are deployed within that cluster.

  • RU20 to RU21
1GbE Management Network Switch (Model S02)
The management network is built around a pair of 48-port, 1Gb Ethernet switches. The IMM port of every IBM Storage Fusion HCI System server is connected to the first of the management switches using CAT5e cables with RJ45 connectors. The alternate IMM port, on either the LOM or an OCP adapter, is configured for all IBM Storage Fusion HCI System servers and it is connected to the second management switch. These connections are also made using CAT5e cables with RJ45 connectors. This is all done so that there is redundancy to support management functions even if one of the management switches fail or one of the cables become disconnected. Each management port from the high speed switches (S01s) as well as each of the 6 rack PDUs are connected to the management network switches.
  • RU18 to RU19
(Optional) Console Kit (KVM) (Model TF5)
Service console used for initial system setup.
  • RU22
AFM Server (Model F01)
The optional Active File Manager (AFM) feature of IBM Spectrum® Spectrum Erasure Code Edition (ECE) is supported by adding an optional one or two servers to IBM Storage Fusion HCI System that function as AFM Gateway Nodes. AFM gateway nodes may generate a large amount of network traffic between themselves and the home system to fetch and to synchronize files. To ensure the best performance and cluster stability, these separate nodes are used so that AFM traffic has its own physical adapter that is separate from the IBM Spectrum Scale cluster network and from servers used for other application workloads.
  • RU23 and RU24
GPU Server (Model G01)
An optional one or two GPU accelerated nodes for AI workloads that leverage the NVIDIA A100 GPUs (3 per server). A minimum quantity of one GPU server can be ordered up to a maximum of two in total.
  • RU25 to RU28

Hardware configuration

You can purchase a rack from IBM or provide one yourself.
Note: The rack can be purchased from IBM or provided by the client.
Base configuration:
  • 42U rack

    2x Ethernet high-speed switches

    2x Ethernet management switches

    6x compute-storage servers with 2 NVMe drives or server:
    • The server in RU7 is connected to the service console tray
    • Servers in RU2, RU3, and RU4 become the OpenShift control plane servers
    • 6x 32 core servers, or 6x 64 core servers, or 3x 32 core servers + 3x 64 core servers
    • 2x 7.68 TB NVMe PCle drives
Available options:
  • Additional compute-storage servers to a maximum of 20 (minus any GPU servers)
  • One or two optional GPU servers, each with 3x NVIDIA A100 GPUs
  • Increased storage by adding drives to compute-storage servers: 7.68 TB NVMe PCle drives or server to a maximum of 10 drives or server
  • Increased compute power by adding compute-only servers
  • A minimum quantity of one AFM (Active File Management) server can be ordered up to a maximum of two in total

Physical configuration

For the physical configuration, limitations, and models, see Family 9155+01 IBM Storage Fusion HCI.

Power Distribution Unit (PDU) positions in IBM Storage Fusion appliance

Use this section as a guidance for PDU positions in IBM Storage Fusion appliance.

Figure 1. IBM Storage Fusion PDU positions
IBM Spectrum Fusion PDU positions
Apply the following rules:
  • All configurations require PDUs 1 and 2 to be connected to power.
  • If more than six compute-storage servers exist, PDUs 3 and 4 must be connected to power.
  • If AFM servers exist, PDUs 3 and 4 must be connected to power.
  • If more than 14 compute-storage servers exist, PDUs 5 and 6 must be connected to power.
  • If GPU servers exist, then PDUs 5 and 6 must be connected to power.
Note: For independent or redundant power feeds, the two power sources must be split between the left and lower (odd numbered) PDUs and the right and upper (even numbered) PDUs.
There are two possible models of the PDU and the one you get depends on your power connection:
  • For all single-phase power and for wye-wired three-phase power there is one PDU feature code. See Supported PDU power cords are ECJN with Souriau inlet table in Supported PDU power cords
  • For delta-wired three-phase power (typically used only in N America) there is a different PDU feature code. See Supported PDU power cords for PDU feature codes ECJQ with Amphenol inlet table in Supported PDU power cords.
For more information about the line cords for power connections, see Supported PDU power cords. For more information about power prerequisites for IBM Storage Fusion, see General power information.

Drives and usable storage capacities

To calculate the usable storage capacities for drives, use the IBM Storage Modeler (StorM) tool.

Weight of miscellaneous rack parts

The following table lists the weight of the rack configs that include miscellaneous parts beyond individual components like cables, Constellation rack and so on:
Table 1. Weight of miscellaneous rack parts
Component Weight (lbs) Weight (kg) Quantity Combined Weight (lbs) Combined Weight (kg) Model
SN3700C switch 27.5 12.5 2 54.9 25.0 S01
7316-TF5 console 12.0 5.5 1 12.0 5.5 TF5
AS4610 switch 11.8 5.4 2 23.7 10.8 S02
SR665 GPU server 85.5 38.9 0 0.0 0.0 G01
SR630 AFM server 41.9 19.0 0 0.0 0.0 F01
SR645-0 32-core 256GB server 39.1 17.8 0 0.0 0.0 C00
SR645-2 32-core 256GB server 40.2 18.3 6 241.2 109.6 C01 with 2 drives
SR645-10 32-core 256GB server 44.6 20.3 0 0.0 0.0 C01 with 10 drives
SR645-0 64-core 1024GB server 39.1 17.8 0 0.0 0.0 C04
SR645-2 64-core 1024GB server 40.2 18.3 0 0.0 0.0 C05 with 2 drives
SR645-10 64-core 1024GB server 40.2 18.3 0 0.0 0.0 C05 with 10 drives
Constellation rack 398.2 181.0 1 398.2 181.0 R42
Intelligent switched PDU+ 9.5 4.3 6 57.0 25.9 n/a
All cables, rails, etc. 395.5 179.8 1 395.5 179.8 n/a
      Total Weight 1182.5 537.5