QAT acceleration for encryption and compression

Intel QuickAssist Technology (QAT) can provide extended accelerated encryption and compression services by offloading the actual encryption and compression requests to the hardware QuickAssist accelerators, which are more efficient in terms of cost and power than general purpose CPUs for those specific compute-intensive workloads.

Before you begin

A running IBM Storage Ceph cluster.
Ceph Object gateway installed.

Grub is configured to pass the intel_iommu parameter.

grubby --update-kernel=ALL --args="intel_iommu=on"

About this task

Important: Hardware accelerated compression in Ceph Object Gateway requires RHEL 9.4 on a Sapphire or Emerald Rapids Xeon CPU (or newer) with QAT devices. For more information, see Intel Ark.

Setting up the QAT service

You can set up the QAT service to encrypt and compress the Ceph Object Gateway objects.

Procedure

Install qatlib-service, qatlib, qatzip, and qatengine packages.
```
dnf install -y qatlib-service qatlib qatzip qatengine
```
Add root to the QAT group on the host.
```
usermod -aG qat root
```
Check that the limits.conf file exists with one of the following ServicesEnabled parameters in the configuration file.
- For using data encryption, check that ServicesEnabled is set to asym.
```
cat /etc/sysconfig/qat

ServicesEnabled=sym
POLICY=8
```
- For using data compression, check that ServicesEnabled is set to dc.
```
cat /etc/sysconfig/qat

ServicesEnabled=dc
POLICY=8
```
- For using both data encryption and compression, check that ServicesEnabled is set to asym,dc.
```
cat /etc/sysconfig/qat

ServicesEnabled=sym,dc
POLICY=8
```

Configure the limits.conf file with memlock information.

sudo vim /etc/security/limits.conf

...
root - memlock 500000
ceph - memlock 500000
...

Enable the configurations in the limits.conf file.
```
sudo su -l $USER
```
Enable the QAT service.
```
systemctl enable qat
```
Restart the node.
```
systemctl reboot
```

Create the specification file and provide additional arguments to Podman for Ceph Object Gateway.

Note: You can use the following command to generate the device list:

--device /dev/vfio --device /dev/qat_adf_ctl $(for i in `ls /dev/vfio/* | grep 'dev' | grep -v ':'` ; do echo --device $i;

Specify either sw or hw for the compression parameter in the specification file to enable the QAT service for Ceph Object Gateway.

compression: sw: Specify to enable QAT software fall back mode.
compression: hw: Enables hardware QAT offload if QAT hardware and virtual functions (VFs) are present on the node.

service_type: rgw
service_id: rgw_qat_auto
placement:
  label: rgw
spec:
  rgw_frontend_port: 8000
  rgw_realm: REALM
  rgw_zone: ZONE
  rgw_zonegroup: ZONE_GROUP
  qat:
    compression: sw | hw
  ssl: true
  generate_cert: true

For example,

[root@host01 ~]# cat rgw-test-qat.yaml

service_type: rgw
service_id: rgw_qat_auto
placement:
  label: rgw
spec:
  rgw_frontend_port: 8000
  rgw_realm: REALM
  rgw_zone: ZONE
  rgw_zonegroup: ZONE_GROUP
  qat:
    compression: sw | hw
  ssl: true
  generate_cert: true

cephadm automatically sets qat_compressor_enabled to true when qat:compression is configured.

What to do next

When using hardware compression (compression: hw), verify that the QAT compressor is enabled in the /etc/sysconfig/qat file.

For example,

[root@host01 sysconfig]# cat qat 

ServicesEnabled=dc 

POLICY=8 

QAT_USER=ceph

Enabling QAT-based encryption

Encrypt objects in Ceph Object Gateway by using the QAT-based encryption for OpenSSL.

Procedure

To enable QAT-based encryption, edit the Ceph configuration file to make use of QAT-based crypto plug-in.

plugin crypto accelerator = crypto_qat

Enabling QAT-based compression

Compress objects in Ceph Object Gateway by using the tool class for QAT acceleration.

About this task

Compression is enabled when creating the QAT specification file. For more information, see Setting up the QAT service.

If needed, use this procedure to enable compression manually.

Procedure

To enable QAT-based compression, edit the Ceph configuration file to enable QAT support for compression.

qat compressor enabled=true