PCIe3 1.92 TB CAPI NVMe Flash Accelerator Adapter (FC EJ1K; CCIN 58CD)
Learn about the specifications and operating system requirements for the feature code (FC) EJ1K adapter.
Overview
- The adapter contains a CAPI accelerator and is connected to two NVMe drives.
- Each NVMe drive is presented as a separate volume to the operating system.
- Each volume is 960 GB totaling 1920 GB (1.92 TB) of flash per adapter.
- The adapter requires volumes to be formatted as 4096-byte sector drives.
- The adapter and its accompanying software libraries will not function with 512-byte sector drives.
- The adapter must be plugged into a direct host-bus PCIe CAPI-capable slot to function.
- Functionality of the adapter depends on operating system and software versions.
- Each of the two NVMe drives on the adapter has a separate drive life or durability meter.
- Linux kernel 4.4 or later is required to enable the adapter.
- Enabled applications must extend or leverage the IBM CAPI Flash libraries. For more information, programming guides, or sample code, see https://github.com/open-power/capiflash/
Specifications
- Item
- Description
- Adapter FRU number
- 01DH749 (Designed to comply with RoHS requirement)
- I/O bus architecture
- PCIe3 x8
- Slot requirement
- One CAPI-capable PCIe3 x16 slot
- Supported Systems
- POWER8® processor-based system
- 8335-GCA
- 8335-GTB
- 8247-21L and 8247-22L
- Voltage
- 3.3 V, 12 V
- Form factor
- Short, low-profile
- Maximum number
- For details about the maximum number of adapters that are supported, see PCIe adapter placement rules and slot priorities and select the system you are working on.
- Attributes provided
- Delivers high IOP and low CPU access to flash memory
- Designed for read intensive workloads with light write activity
- Two 960 GB M.2 NVMe modules with a write capacity of approximately 1,370 TB
Administering the adapter
Administration of the CAPI NVMe Flash accelerator adapters uses common software utilities with the CAPI Fibre Channel Accelerator (FC EJ16).
- The system firmware and operating system are installed and the virtualization configuration is complete.
- The IBM Data Engine for NoSQL User Libraries and AFU microcode are installed in the system default location /opt/ibm/capikv.
- One or more CAPI NVMe Flash accelerator adapters are plugged into CAPI-capable slots.
- The CAPI NVMe Flash accelerator adapter microcode is updated.
Checking Drive Write Durability or “Drive Endurance"
- Find the drive’s SCSI sgX identifier from cxlfstatus:
- username@hostname:~$ sudo /opt/ibm/capikv/bin/cxlfstatus
- [sudo] password for ubuntu:
- CXL Flash Device Status
Output similar to the following example is displayed:
Found 0601 0000:01:00.0 U78C9.001.RST0013-P1-C7
Device SCSI Block Mode LUN WWID sg2: 1:0:0:0 sdb legacy 60025380025382463300116000000000 sg3: 1:1:0:0 sdc legacy 60025380025382463300116000000000 Found 0601 0002:01:00.0 U78C9.001.RST0013-P1-C6
Device SCSI Block Mode LUN WWID sg4: 2:0:0:0 sdd legacy 60025380025382462300037000000000 sg5: 2:1:0:0 sde legacy 60025380025382462300036000000000 - To read the drive’s endurance indicator, use the sg utilities sg_logs:
- uusername@hostname:/opt/ibm/capikv/afu$ sudo sg_logs -a /dev/sg2
- [sudo] password for ubuntu:
- IBM SAMSUNG MZ1LV960 301Q
Output similar to the following example is displayed:Supported log pages [0x0]: 0x00 Supported log pages 0x0d Temperature 0x11 Solid state media 0x2f Informational exceptions (SMART) 0x19 General statistics and performance - Temperature page [0xd]:
- Current temperature = 38 °C
- Reference temperature = 84 °C
- Solid state media page [0x11]:
- Percentage used endurance indicator: 12%
Control access to the CAPI NVMe Flash accelerator
- To control access to the CAPI NVMe Flash accelerator, complete the following step:Note: By default, installation of the IBM Data Engine for NoSQL software libraries creates a cxl user and adds Udev rules to restrict read and write access to the accelerator's volumes to members of the cxl system.
- To enable an account to read and write to the accelerator, add the account to the
cxl by entering the following command:
sudo usermod -a -G cxl userid
- To enable an account to read and write to the accelerator, add the account to the
cxl by entering the following command:
View the status of the CAPI NVMe flash accelerator
- To view the status of each accelerator adapter's LUN, enter the following
command:sudo /opt/ibm/capikv/bin/cxlfstatusNotes:
- Volumes may be in either legacy or superpipe mode. The default mode is legacy.
- Volumes must be in superpipe mode for exploitation by the CAPI Flash block or arkdb APIs.
Table 1. Coherent Accelerator Interface (CXL) Flash device Device SCSI Block Mode LUN WWID sg9: 33:0:0:0 sdc legacy 60050768218b0818200000000400006e sg10: 33:1:0:0 sdd superpipe 60050768218b0818200000000600006f sg11: 34:0:0:0 sde superpipe 60050768218b08182000000007000070 sg12: 34:1:0:0 sdf superpipe 60050768218b0818200000000300006d Table 1 shows two accelerators, each with two ports, and a single volume mapped to each port's WWPN. One volume is in legacy mode, and three volumes are in superpipe mode. The WWID for each volume in the IBM FlashSystem® are displayed for convenience of administration. This matches the WWID shown in the IBM FlashSystem GUI or CLI (ibm@power8:~$ sudo /opt/ibm/capikv/bin/cxlfstatus)
Set the mode for a volume
- To set the mode for a volume, enter the following command:
sudo /opt/ibm/capikv/bin/cxlfsetlunmode <LUN> <Mode>
An output similar to the following example is displayed:
- ibm@power8:~$ /opt/ibm/capikv/bin/cxlfsetlunmode
- 60050768218b0818200000000400006e 1
- INFO: Adding LUN 60050768218b0818200000000400006e to Super IO table. SUCCESS
- LUN modes might be 0 for legacy mode, or 1 for superpipe mode.
- After a LUN is set to superpipe mode, all paths to that volume will also be set to superpipe mode.
- As shown in Table 1, each volume might be in either legacy or superpipe mode.
Operating system or partition requirements
- Linux
- Ubuntu 16.04.1 or later
- Red Hat Enterprise Linux (RHEL) 7.3, or later
- For support details, see the Linux Alert website.
- The latest version of enabling libraries and utilities may be downloaded from Fix Central.