Adapter placement rules and slot priorities for the 9040-MR9
Find information about the placement rules and slot priorities for the adapters that are supported for the 9040-MR9 system.
Slot descriptions for the 9040-MR9
The 9040-MR9 systems provide PCIe3 and PCIe4 slots. The 9040-MR9 systems offer two or four POWER9™ processor modules. The PCIe slots are enabled to support the PCIe3 cable adapter (FC EJ08 or FC EJ20) that is used to attach the EMX0 PCIe Gen3 I/O expansion drawer.
Table 1 provides information about the PCIe
switches in the 9040-MR9 systems.
Two PCIe3
switches (24-port PLX PEX 8725) in the system backplane provide PCIe3 buses from the system
processor modules that provide connectivity to the following features:- PCIe slots
- PCIe local area network (LAN) controller
- PCIe3 internal SAS controller
| Features provided | Switch 1 and Switch 2 |
|---|---|
| Lanes and ports | 24-lane, 10-ports, PCIe3 |
| With integrated 8.0 gigatransfers per second (GT/s) Serializer/Deserializer (SerDes) speed negotiation for each port | |
| Lane and polarity reversal | Supported |
| All ports support concurrent maintenance through I2C bus | Yes |
| End-to-end cyclic redundancy check (CRC) and poison bit error checking | Supported |
| Data path parity | Supported |
| Memory error correction | Supported |
| Advanced error reporting | Supported |
| Designate any port as the upstream port | Yes |
| 19x19 mm, 324-pin FCBGA package | Yes |
| Power consumption | Typical power: 5.4 watts |
| Location code | Description | SCM / PHB | Slot capabilities | Slot availability | |||||
|---|---|---|---|---|---|---|---|---|---|
| OpenCAPI | PCIe3 cable adapter | Dynamic direct memory access (DMA) window | Enlarged capacity assign order1 | 2 processors | 3 processors | 4 processors | |||
| P1-C2 | PCIe4 x16 | 3/3 | No | Yes | Yes | 8 | No | No | Yes |
| P1-C3 | PCIe4 x16 | 3/0 | Yes | Yes | Yes | 4 | No | No | Yes |
| P1-C4 | PCIe4 x16 | 2/3 | No | Yes | Yes | 7 | No | Yes | Yes |
| P1-C5 | PCIe4 x16 | 2/0 | Yes | Yes | Yes | 3 | No | Yes | Yes |
| P1-C6 | PCIe3 x8 (Base Ethernet adapter use only | 1/2 | No | No | Yes | 9 | Yes | Yes | Yes |
| P1-C7 | PCIe4 x16 | 1/3 | No | Yes | Yes | 6 | N/A | N/A | N/A |
| P1-C8 | PCIe4 x16 | 1/0 | Yes | Yes | Yes | 2 | Yes | Yes | Yes |
| P1-C9 | PCIe4 x 8 (SAS controller slot for controlling internal disk bays) | 1/1 | No | No | Yes | 11 | Yes | Yes | Yes |
| P1-C10 | PCIe4 x16 | 0/3 | Yes | Yes | Yes | 5 | Yes | Yes | Yes |
| P1-C11 | PCIe4 x16 | 0/0 | No | Yes | Yes | 1 | Yes | Yes | Yes |
| P1-C12 | PCIe4 x8 (SAS controller slot for controlling internal disk bays) | 0/1 | No | No | Yes | 10 | Yes | Yes | Yes |
|
1Assigned PCIe slot order when the I/O Adapter Enlarged Capacity option is enabled.
For example, if the option was enabled with a value of 5, five slots are enabled with the I/O
enlarged capacity.
Note: Enabling the I/O Adapter Enlarged Capacity option will only affect Linux® partitions. If your system does not have Linux partitions, the I/O Adapter Enlarged Capacity setting should be
disabled.
Note the following:
|
|||||||||
Firmware slot capabilities
System PCIe slots are allocated direct memory access (DMA) space using the following algorithm:
- All slots are allocated a 2 GB default DMA window.
- All I/O adapter slots (except the embedded USB) are allocated Dynamic DMA Window (DDW)
capability based on installed platform memory. DDW capability is calculated assuming 4K I/O mappings:
- For systems with less than 64 GB of memory, slots are allocated 16 GB of DDW capability.
- For systems with at least 64 GB of memory, but less than 128 GB of memory, slots are allocated 32 GB of DDW capability.
- For systems with 128 GB or more of memory, slots are allocated 64 GB of DDW capability.
- Slots can be enabled with Huge Dynamic DMA Window capability (HDDW) using the I/O Adapter Enlarged Capacity setting in ASMI.
- HDDW enabled slots are allocated enough DDW capability to map all of installed platform memory using 64 K I/O mappings.
- Minimum DMA window size for HDDW enabled slots is 32 GB.
- Slots that are HDDW enabled are allocated the larger of the calculated DDW capability and HDDW capability.
Adapter placement rules
Use this information while selecting slots for installing adapters in the 9040-MR9 system. Use Table 3 to identify adapter slot placement priorities and
maximums for the system.
- P1-C9 and P1-C12 are general-purpose slots that are also designated as serial-attached SCSI (SAS) controller slots for controlling the internal disk bays.
- All of the x16 PCIe slots are coherent accelerator processor interface (CAPI) enabled.
- Four of the x16 PCIe slots support NVlink or OpenCAPI 25 Gbs cable cards. The OpenCAPI cable cards cannot be serviced with the system power turned on.
- P1-C1 is for the service processor card assembly. The P1-C1 slot cannot be serviced with the system power turned on.
- P1-C6 is designated for the Base Ethernet card.
- The four x4 PCIe3 buses from the two PLX 24-port PCIe3 switches go to the four Non-volatile Memory Express (NVMe) drives on the direct-access storage device (DASD) backplane at the front of the system.
Verify whether the adapter is supported for your system. The feature code (FC) column in the following table lists all of the supported adapters for systems. You can select the FC for more details. To view a list of adapters supported in the POWER9 processor-based systems and the EMX0 PCIe3 expansion drawer, see Adapter information by feature code for the 9008-22L, 9009-22A, 9009-22G, 9009-41A, 9009-41G, 9009-42A, 9009-42G, 9040-MR9, 9080-M9S, 9223-22H, 9223-22S, 9223-42H, 9223-42S system and EMX0 PCIe3 expansion drawers.
The 9040-MR9 systems can have two, three, or four system processor modules.
Note: In the following table, the slot priority numbers 2 - 12 correspond to slot locations P1-C2
through P1-C12.
| Feature code | Description | Slot priority | Maximum number of adapters supported | ||
|---|---|---|---|---|---|
| 2 processors | 3 processors | 4 processors | |||
| 5729 | PCIe2 FH 4-port 8 Gb Fibre Channel adapter (FC 5729; CCIN 5729) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| 5735 | 8 Gb PCI Express dual-port Fibre Channel adapter (FC 5735 and EL58); CCIN 577D) | 12, 9,11,8, 10, 7 | 12, 9, 11, 8, 5, 10, 7, 4 | 12, 9, 11, 8, 5, 3, 10, 7, 4, 2 | 6/8/10 |
| 5748 | POWER® GXT145 PCI Express Graphics Accelerator (FC 5748; CCIN 5269) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| 5785 | 4-port Async EIA-232 PCIe 1X adapter (FC 5277 and 5785; CCIN 57D2) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| 5899 | PCIe2 4-port 1 GbE adapter (FC 5260, FC 5899, FC EL4L, and FC EL4M; CCIN 576F) | 6, 12, 9, 11, 8, 10, 7 | 6, 12, 9, 11, 8, 5, 10, 7, 4 | 6, 12, 9, 11, 8, 5, 3, 10, 7, 4, 2 | 7/9/11 |
| EC2S | PCIe3 2-port 10 Gb NIC & RoCE SR/Cu adapter (FC EC2R and EC2S; CCIN 58FA) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EC2U | PCIe3 2-port 25/10 Gb NIC & RoCE SFP28 Adapter (FC EC2U; CCIN 58FB) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EC3B | PCIe3 2-Port 40 GbE NIC RoCE QSFP+ adapter (FC EC3B; CCIN 57BD); Adapter part number: 00FW105 | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC3M | PCIe3 2-port 100 GbE NIC & RoCE QSFP28 adapter (FC EC3M; CCIN 2CEC); Adapter FRU number: 00WT078 | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EC38 | PCIe3 2-port 10 GbE NIC & RoCE SFP+ Copper adapter (FC EC37, EC38, EL3X, and EL53; CCIN 57BC) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 10, 7, 12, 9 | 7/9/11 |
| EC46 | PCIe2 4-port USB 3.0 adapter (FC EC45 and EC46; CCIN 58F9) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC5B | PCIe3 x8 Non-Volatile Memory 1.6 TB SSD NVMe adapter (FC EC5A, EC5B, EC5G, EC6U, and EC6V; CCIN 58FC) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC5D | PCIe3 x8 Non-Volatile Memory 3.2 TB SSD NVMe adapter (FC EC5C and EC5D; CCIN 58FD) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC5E | PCIe3 x8 Non-Volatile Memory 6.4 TB SSD NVMe adapter (FC EC5E, EC5F, EC6Y, and EC6Z; CCIN 58FE) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC63 | PCIe4 x16 1-Port EDR 100 GB InfiniBand ConnectX-5 CAPI-capable adapter (FC EC62 and EC63; CCIN 2CF1) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EC65 | PCIe4 x16 2-Port EDR 100 GB IB ConnectX-5 CAPI Capable adapter (FC EC65; CCIN 2CF2) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EC66 | 2-port 100 GB RoCE En Connectx-5 PCIe4 x16 adapter (FC EC66; CCIN 2CF3) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EC6J | PCIe2 LP 2-Port USB 3.0 Adapter (FC EC6J and FC EC6K; CCIN 590F); Adapter part number: 02JD518 | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC76 | PCIe4 2-port 100 GbE RoCE x16 adapter (FC EC75 and FC EC76; CCIN 2CFB); Adapter part number: 02CM921 | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EC7B | PCIe4 x8 NVMe 1.6 TB SSD NVMe Adapter (FC EC7A, EC7B, EC7J, and EC7K; CCIN 594A) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC7D | PCIe4 x8 NVMe 3.2 TB SSD NVMe Adapter (FC EC7C, EC7D, EC7L, and EC7M; CCIN 594B) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EC7F | PCIe4 x8 NVMe 3.2 TB SSD NVMe Adapter (FC EC7E, EC7F, EC7N, and EC7P; CCIN 594C) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EJ08 | PCIe3 cable adapter for the EMX0 PCIe3 expansion drawer (FC EJ08; CCIN 2CE2) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EJ0J | PCIe3 SAS RAID quad-port 6 Gb adapter (FC EJ0J and EL59); CCIN 57B4) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EJ0K | PCIe3 SAS RAID quad port 6 Gb x8, low-profile capable adapter (FC EJ0K; CCIN 57B4) | 12, 9, 11, 8, 10, 7 | 12, 9, 11, 8, 5, 10, 7, 4 | 12, 9, 11, 8, 5, 3, 10, 7, 4, 2 | 6/8/10 |
| EJ0L | PCIe3 12 GB Cache RAID SAS quad-port 6 Gb adapter (FC EJ0L; CCIN 57CE) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EJ10 | PCIe3 4 x8 SAS Port adapter (FC EJ10; CCIN 57B4) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EJ14 | PCIe3 12 GB Cache RAID PLUS SAS adapter quad-port 6 Gb x8 (FC EJ14; CCIN 57B1) | 12, 9, 11, 8, 10, 7 | 12, 9, 11, 8, 5, 10, 7, 4 | 12, 9, 11, 8, 5, 3, 10, 7, 4, 2 | 6/8/10 |
| EJ1N and EJ1P | PCIe1 SAS Tape/DVD dual-port 3 Gb x8 adapter (FC EJ1N and EJ1P; CCIN 57B3) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EJ20 | PCIe3 cable adapter for the EMX0 PCIe3 expansion drawer (FC EJ20; CCIN 2CF5) | 11, 8, 10, 7 | 11, 8, 5, 10, 7, 4 | 11, 8, 5, 3, 10, 7, 4, 2 | 4/6/8 |
| EJ27 | PCIe Crypto- graphic Coprocessor (FC EJ27 and EJ28; CCIN 476A) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EJ32 | 4767-001 Cryptographic Coprocessor (FC EJ32; CCIN 4767) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EJ37 | 4769-001 Cryptographic Coprocessor (FC EJ35 and EJ37 for BSC; CCIN C0AF) | 6 | 6 | 6 | 1/1/1 |
| EN0A | PCIe3 16 Gb 2-port Fibre Channel adapter (FC EL5B and FC EN0A; CCIN 577F) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN0G | PCIe2 8 Gb 2-port Fibre Channel adapter (FC EN0F and EN0G; CCIN 578D) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN0H | PCIe3 4-port (10 Gb FCoE and 1 GbE) (FCEL38, FC EL56, FC EN0H, and FC EN0J; CCIN 2B93) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EN0K | PCIe3 4-port (10 Gb FCoE and 1 GbE) copper and RJ45 adapter (FC EL57 and EN0K; CCIN 2CC1) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EN0M | PCIe3 4-port (10 Gb FCoE and 1 GbE) LR and RJ45 adapter (FC EN0M and EN0N; CCIN 2CC0) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN0S | PCIe2 4-port (10 Gb + 1 GbE) SR+RJ45 adapter (FC EN0S; CCIN 2CC3) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EN0U | PCIe2 4-port (10 Gb + 1 GbE) Copper SFP+RJ45 adapter (FC EN0U; CCIN 2CC3) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EN0W | PCIe2 2-port 10 GbE BaseT RJ45 adapter (FC EN0W; CCIN 2CC4) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EN12 | PCIe2 FH 4-port 8 Gb Fibre Channel adapter (FC EN12; CCIN EN0Y) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN15 | PCIe3 4-port 10 GbE SR adapter (FC EN15 and EN16; CCIN 2CE3) | 6, 11, 8, 10, 7, 12, 9 | 6, 11, 8, 5, 10, 7, 4, 12, 9 | 6, 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 7/9/11 |
| EN1A | PCIe3 8x 2-port Fibre Channel (32 Gb/s); (FC EL5U, EL5V, EN1A, and EN1B); CCIN 578F) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN1C | PCIe3 8x 4-port Fibre Channel (16 Gb/s); (FCs EL5W, EL5X, EN1C, and EN1D; CCIN 578E) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN1E | PCIe3 x8 4-port Fibre Channel (16 Gb/s); (FC EN1E and EN1F; CCIN 579A); Adapter part number: 02JD586 | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN1G | PCIe3 8x 2-port Fibre Channel (16 Gb/s) (EN1G and EN1H; CCIN 579B); Adapter part number: 02CM900 and 02CM903 | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN1J | PCIe4 x8 2-port Fibre Channel (32 Gb/s); (FC EN1J and EN1K; CCIN 579C); Adapter part number: 02CM909 | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |
| EN2A | PCIe3 16 Gb 2-port Fibre Channel adapter (FC EN2A and FC EN2B; CCIN 579D) | 11, 8, 10, 7, 12, 9 | 11, 8, 5, 10, 7, 4, 12, 9 | 11, 8, 5, 3, 10, 7, 4, 2, 12, 9 | 6/8/10 |