Deploying a GREP11 server as a Confidential Computing Container Runtime instance with passthrough support

You can configure the existing c16client.yaml file to deploy GREP11 server to use local passthrough CEX device or use Crypto Appliance API.

If the c16client.yaml file is not present, or when there is no server entry that is configured in this file, the C16 client uses local passthrough CEX device.

The following are the samples of c16client.yaml:

#trace, debug, info, warn, err, error, critical, off
loglevel: 'debug'
 

#trace, debug, info, warn, err, error, critical, off
loglevel: 'debug'
servers:
  - local: true
 

To assign Crypto domains to secure execution VM, complete the following steps in KVM host:

1. Run the lszcrypt -V command to get the Crypto domain details:

   lszcrypt -V
    

   Example:

   lszcrypt -V
   CARD.DOM TYPE  MODE        STATUS     REQUESTS  PENDING HWTYPE QDEPTH FUNCTIONS  DRIVER
   --------------------------------------------------------------------------------------------
   18       CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4card
   18.001c  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue
   18.001d  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue
   19       CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4card
   19.001c  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue
   19.001d  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue
    

   You can see the adapter ID and domain information. For example, in 18.001c , the adapter is 18 and the domain is 001c. The Crypto domain should match to the domain ("${HSM_DOMAIN}") in the grep11server.yaml file mentioned in Prepare configuration files for GREP11 server.

2. Crypto domains control must be passed by a KVM host to guests. To do that:

   1. Load vfio_ap module to give control of Crypto domains to vfio_ap device driver:

      modprobe vfio_ap
       

   2. Free the domains from control of the zcrypt device driver. For more information, see Cryptographic adapter resources.

      echo 0x0 > /sys/bus/ap/apmask
       

      echo 0x0 > /sys/bus/ap/aqmask
       

3. Create a mediated device:

   1. Run the uuidgen command to generate a 16-byte random value:

      uuid=$(uuidgen)
       

   2. Run the following command to create the device:

      echo ${uuid} > /sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/create
       

   For more information, see Create a mediated device.

4. Assign a Crypto adapter to the UUID that is generated in step 3.

   echo <adapter_id> > /sys/devices/vfio_ap/matrix/${uuid}/assign_adapter
    

   For Example:

   echo 0x18 > /sys/devices/vfio_ap/matrix/${uuid}/assign_adapter
    

5. Assign a domain to the generated UUID:

   echo <domain_id> > /sys/devices/vfio_ap/matrix/${uuid}/assign_domain
    

   For Example:

   echo 0x001c > /sys/devices/vfio_ap/matrix/${uuid}/assign_domain
    

   Note: To assign multiple domains to a VSI, repeat steps 4 and 5 with different adapter and domain IDs to the same mediated device.

6. Add the following data under the <devices> section (the section that includes UUID device) of the HPVS instance XML to ensure that the instance is assigned with one or more Crypto domains. For information about setting up an HPVS instance, see Setting up and configuring IBM Confidential Computing Container Runtime.

   <devices> 
     <hostdev mode='subsystem'  type='mdev' managed='no' model='vfio-ap'>
           <source>
               <address uuid='<uuid>'/> 
           </source>
       </hostdev>
   </devices>  
   </domain>
    

Prepare and collect the following certificates. For more information, see Creating OpenSSL certificates for GREP11 Servers.

Sample of grep11server.yaml:

#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
logging:
  # Package log levels
  # Levels: info, warning, debug, error, fatal, trace, and panic
  levels:
    # example below
    # entry: debug
###################################
## GRPC SERVICES TALKING TO GREP11 server ##
ep11crypto:
  enabled: true
  connection:
    address: 0.0.0.0
    port: 9876
    # Secure connection TLS options
    tls:
      enabled: true
      # certfile, keyfile and cacert refer to the pem files that holds the certs
      certfile: /cfg/grep11-server.pem
      keyfile: /cfg/grep11-server.key
      mutual: true
      cacert: /cfg/grep11-ca.pem
      # same as above, but instead of filename, contents of PEM can be in an environment
      # variable (i.e. to avoid mounting files into Docker container)
      cacertbytes:
      certfilebytes:
      keyfilebytes:
    # Server TCP/IP connection monitoring
    # serverKeepaliveTime is the duration in seconds after which if the server
    # does not see any activity from the client it pings the client to see
    # if it is alive
    # serverKeepaliveTimeout is the duration the server waits for a response
    # from the client after sending a ping before closing the connection
    keepalive:
      serverKeepaliveTime: 30
      serverKeepaliveTimeout: 5
  # card.domain tuple and domain are hex numbers
  # i.e. "01.00a9" corresponds to 169th domain on first card.
  domain: "${HSM_DOMAIN}"
 

You can deploy GREP11 server either on a local s390x environment or as a Confidential Computing Container Runtime instance.

If you want to access Cloud HSM on IBM Cloud Confidential Computing Container Runtime remotely to perform cryptographic operations, you need to generate a GREP11 API request in your on-prem environment to enable mutual TLS authentication for GREP11 API instead of IBM Cloud IAM authentication.

Prepare the following mutual TLS client certificates for connection with GREP11 server:

Add the following snippet into your code:

cert, _ := tls.LoadX509KeyPair("grep11-client.pem ", "grep11-client.key")
var callOpts = []grpc.DialOption{
  grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{Certificates: []tls.Certificate{cert}}))
}
 

Now, you can perform cryptographic operations by remotely accessing the functions with the Enterprise PKCS #11 (EP11) over gRPC API calls. For more information, see Cryptographic operations: GREP11 API.