System overview

An IBM Storage Scale System 6000 (MTM 5149-F48) enclosure contains Non-Volatile Memory Express (NVMe) attached drives and a pair of server canisters.

IBM Storage Scale System 6000 is an all-flash array platform. This storage platform uses NVMe-attached drives to provide significant performance improvements as compared to SAS-attached flash drives.

IBM Storage Scale System 6000 can contain up to 48 NVMe-attached drives. You can either use 24 drives in a half-populated configuration or 48 drives in a fully populated configuration. These drives are accessible from the front of IBM Storage Scale System 6000, as shown in the following figure.
Note: The minimum number of drives is 24. In a 24-drives configuration, drives must be plugged only into slots with even numbers (2-24) in the canister 1 (top); and only into slots with odd numbers (25-47) in the canister 2 (bottom).
Figure 1. Front view of the IBM Storage Scale System 6000
Front view of IBM Storage Scale System 6000
Each IBM Storage Scale System 6000 contains two identical I/O server canisters. The following figure shows the rear view of the MTM 5149-F48 module canisters.
Figure 2. Rear view of IBM Storage Scale System 6000
Rear view of IBM Storage Scale System 6000

The IBM Storage Scale System 6000 uses two AMD EPYC 9454 SP5 per server canister. The following table provides an overview of IBM Storage Scale System 6000.

Table 1. Overview of the IBM Storage Scale System 6000 system
Product Number of DIMMs per server canister Total memory per server canister Total storage (raw capacity) per IBM Storage Scale System 6000 unit Server canister features
IBM Storage Scale System 6000 24 (32 GiB DIMM) 768 GiB Up to 1440 TB NVMe

Dual socket AMD EPYC 9454 SP5, 48-core processors.

Dual 960 GB NVMe boot drives.
24 (64 GiB DIMM) 1536 GiB
The major key characteristics of the IBM Storage Scale System 6000 system are as follows:
  • IBM Storage Scale software with enclosure-based, all-inclusive software feature licensing.
  • Support for PCIe Gen 5.
    • FC: AJQS. 1 to 3 x16 G5 CX-7 (VPI, dual port 200 Gb)
    • FC: AJQQ. 1 to 3 x16 G5 CX-7 (single port 400 Gb, no auxiliary card)
  • Support for dual AMD EPYC 9454 SP5, 48 cores (in the performance configuration).
  • NVMe transport protocol for high performance of 2.5 inches (SFF) NVMe-attached drives and flash drive:
    • Support for industry-standard 2.5 inches, NVMe-attached drive options with the following storage capacities: 3.84 TB, 7.68 TB, 15.36 TB, and 30.72 TB (mentioned in Table 2).
    Table 2. Storage capacities and feature codes of NVMe-attached drives
    Feature code Description FRU part number
    AK0K 3.84 TB 2.5 in. PCIe Gen 4 NVMe-attached drive 03JN944
    AK11 7.68 TB 2.5 in. PCIe Gen 4 NVMe-attached drive 03JN945
    AK0L 15.36 TB 2.5 in. PCIe Gen 4 NVMe-attached drive 03JN946
    AK0M 30.72 TB 2.5 in. PCIe Gen 4 NVMe-attached drive 03JN947
  • 24 or 48 drives per enclosure configurations are available.
  • Onboard ports per canister:
    • Two RJ45 of 10 GbE for management
    • One RJ45 of 1 GbE for BMC inter-canister communication
    • One mini HDMI port for console display
    • One USB 3.0 type A port for crash-cart keyboard
    • One USB Mini-C for service
    • Various I/Os depending on the add-in cards
  • Up to eight Gen 5 x16 PCIe slots per canister. These slots support the following adapter options:
    • PN: 01LL984 - CX-7 VPI, dual port, 200 Gb.
    • PN: 01LL983 - CX-7 InfiniBand, single port, 400 Gb.

The ports and LEDs details of IBM Storage Scale System 6000 are shown in the following figure.

Figure 3. IBM Storage Scale System 6000 ports and LEDs
IBM Storage Scale System 6000 ports and LEDs
Item Usage
1 Dual boot drives (mirrored)
2 2x 10 GbE management network
3-10 NVIDIA CX7 network x16 G5
11 1 GbE for BMC or host
12 Push button
13 USB mini-C
14 Mini HDMI
15 Dual CXL (not used)
16 USB A

Networking details

The networking details of IBM Storage Scale System 6000 are shown in the following figure.
Figure 4. IBM Storage Scale System 6000 networking
IBM Storage Scale System 6000 networking
  • Intel Ethernet Converged Network Adapter X550-T2
    • System interface: PCIe Gen 3
    • Ethernet interface: 10 GbE / 1 GbE / 100 Mb
  • FC: AJQS - CX-7 InfiniBand or VPI dual port HPC adapter
    • InfiniBand - HDR200 200 Gb / HDR100 100 Gb / EDR 100 Gb / NDR 200 Gb
    • Ethernet - 100 GbE / 200 GbE
  • FC: AJQQ - CX-7 InfiniBand, single port, 400 Gb

NVMe transport protocol in IBM Storage Scale System 6000

The IBM Storage Scale System 6000 systems use the Non-Volatile Memory express (NVMe) drive transport protocol.

  • NVMe is designed specifically for flash technologies. It is a faster and less complicated storage drive transport protocol than SAS.
  • The NVMe-attached drives support multiple queues so that each CPU core can communicate directly with the drive. This feature avoids the latency and overhead of core-to-core communication to give the best performance.
  • NVMe offers better performance and lower latencies exclusively for solid-state drives through multiple I/O queues and other enhancements.
  • High-performance IBM Storage Scale RAID supports the following RAID code options:
    • 3WayReplication
    • 4WayReplication
    • 8+2p
    • 8+3p
    Note: For information about RAID code options, see IBM Storage Scale RAID Administration guide.
  • The NVMe transport protocol supports industry-standard NVMe flash drives.

Enterprise-level software and support

IBM Storage Scale System 6000 consists of two server canisters that run the IBM Storage Scale software, which is a part of the IBM Storage family. For more information about the IBM Storage Scale capabilities, see IBM Storage Scale.

Enterprise-level software and support are available, based on the IBM Storage Scale software stack.

System topology

The following figure shows high-level architecture and topology.
Figure 5. IBM Storage Scale System 6000 high-level architecture and topology
Hardware high-level architecture and topology

EMS node

The IBM® Storage Scale System Utility Node is an enterprise management server (EMS). Each IBM Storage Scale System of one or more building blocks requires an EMS. You can order an IBM Storage Scale System Utility Node with IBM Storage Scale System 6000, but this configuration mostly applies to new clusters. Conversion from Power9 EMS to IBM Storage Scale System Utility Node is not yet supported. Mixing EMS types in the same environment is not supported either.

You need a minimum of one EMS node as a part of your IBM Storage Scale System cluster. The EMS node can be ordered as a part of IBM Storage Scale System 6000.

IBM Storage Scale System 6000 is supported on the IBM Storage Scale System Utility Node. The minimum release level on the IBM Storage Scale System 6000 is IBM Storage Scale System6.1.9 for IBM Storage Scale System Utility Node.

For more information about the IBM Storage Scale System Utility Node EMS, see Model 5149-23E server specifications.

The EMS node also serves as a third IBM Storage Scale quorum node in a configuration with one building block.

System management

An EMS node in an IBM Storage Scale System 6000 cluster provides system management functions. IBM Storage Scale System 6000 GUI and call home run on the EMS nodes and provide management and health monitoring capabilities. The EMS node also runs a container with Ansible® playbook that can provide orchestration of complex tasks, such as cluster configuration, file system creation, and code update. Figure 5 shows high-level view of system management topology.