Architecture and components of openCryptoki

openCryptoki consists of a slot manager and an API for slot token dynamic link libraries (STDLLs). The slot manager runs as a daemon, provides the number of configured tokens to applications, and it interacts with the tokens (that are used by the applications) using Unix domain sockets or a shared memory segment. For each device with which a token should be associated, this token must be defined in a slot in the openCryptoki configuration file (/etc/opencryptoki/opencryptoki.conf). The shared memory segment allows for proper sharing of state information between applications to help ensure conformance with the PKCS #11 specification.

Figure 1 shows the architecture of openCryptoki:

openCryptoki supports different token types for SW tokens and for various forms of HW support, for example, IBM® Crypto Express adapters. openCryptoki allows to manage multiple tokens that can be used in parallel by one or more processes. These multiple tokens can have the same or different types.

Users can configure openCryptoki and in particular, the set of tokens, using the opencryptoki.conf file (see Adjusting the openCryptoki configuration file). They can define and exploit multiple tokens of any token type, each with a different token name. EP11 tokens and CCA tokens can be configured using a token-specific configuration file for each token instance. Samples of such configuration files are shown in Figure 3 and in Figure 1. An example of how to define multiple EP11 tokens in the overall openCryptoki configuration file is shown in Figure 1.

Each token uses a unique token directory. This token directory receives the token-individual information (like for example, key objects, User PIN, SO PIN, or hashes). Thus, the information for a certain token is separated from all other tokens. For example, for most Linux® distributions, the CCA token directory is /var/lib/opencryptoki/ccatok. The CCA token is called ccatok, if there is only one instance of a CCA token, and no explicit name is defined in the openCryptoki configuration file.

For information on the location and content of the single token directories read Token specifications.

Slots and tokens

Imagine the use of smart cards: In the same way a smart card is inserted into a smart card reader, a PKCS #11 token is inserted into a PKCS #11 slot, where a slot is identified by its ID. A token is library code that knows how to interface with the cryptographic hardware. However, there is no requirement for the token to use any hardware at all, and accordingly there is a so called soft token (described in Soft token) which represents a pure software library accessible via openCryptoki.

Each token is of a certain token-type, where a token-type is implemented by STDLLs. For example, all EP11 tokens use the libpkcs11_ep11.so STDLL (see also Figure 1). A certain instance of a token is implemented by data structures allocated by an STDLL (and a token directory).

All tokens available to openCryptoki are configured in the opencryptoki.conf configuration file. Each token configuration in opencryptoki.conf defines the token type of the token by specifying the adequate STDLL. The opencryptoki.conf configuration file is used by all processes (applications) using (linking to) openCryptoki. Each process may call some or all tokens defined in opencryptoki.conf.

The PKCS #11 API provides a set of slot and token management functions. For example:

• C_GetSlotList() gets a list of available slots.
• C_GetSlotInfo() obtains information on each slot (for example, whether a token is present, or whether the token represents a removable device).
• C_InitToken() initializes an inserted token.
• C_GetTokenInfo() provides information on such a token (for example, whether a login is required to use the token, a count of failed log-ins, information on whether the token has a random number generator).
• C_InitPIN() and C_SetPIN() manage the PIN used to protect a token from unauthorized access.

View an example for how to obtain slot information using the C_GetSlotInfo() function:

CK_RV getSlotInfo(CK_SLOT_ID slotID, CK_SLOT_INFO_PTR slotInfo);

/*  typedef struct CK_SLOT_INFO {         */
/*    CK_UTF8CHAR slotDescription[64];    */
/*    CK_UTF8CHAR manufacturerID[32];     */
/*    CK_FLAGS flags;                     */
/*    CK_VERSION hardwareVersion;         */
/*    CK_VERSION firmwareVersion;         */
/*  } CK_SLOT_INFO;                       */

/* Flags:                      */              
/* CKF_TOKEN_PRESENT           */
/* CKF_REMOVABLE_DEVICE        */
/* CKF_HW_SLOT                 */ 

{
       rc = C_GetSlotInfo(slotID, &slotInfo);
       if (rc != CKR_OK) {
          printf("Error getting slot information: %x \n", rc);
          return rc;
       }
       
       if ((slotInfo.flags & CKF_TOKEN_PRESENT) == CKF_TOKEN_PRESENT) 
           printf("A token is present in the slot.\n");
       
       if ((slotInfo.flags & CKF_REMOVABLE_DEVICE) == CKF_REMOVABLE_DEVICE) 
          printf("The reader supports removable devices.\n");
       
       if ((slotInfo.flags & CKF_HW_SLOT) == CKF_HW_SLOT)
          printf("The slot is a hardware slot.\n");   /* opposed to a SW slot for a soft token  */
             
    return CKR_OK;
 }

Slot manager

The slot manager daemon (pkcsslotd) manages slots (and therefore the tokens plugged into these slots) in the system. Its main task is to coordinate token accesses from multiple processes. A fixed number of processes can attach themselves implicitly to pkcsslotd during openCryptoki initialization, so a static table in shared memory is used. The current limit of the table is 1000 processes using the subsystem. The daemon sets up this shared memory upon initialization and acts as a garbage collector thereafter, helping to ensure that only active processes remain registered.

Starting with openCryptoki 3.21, the pkcsslotd daemon does no longer run as root user. Though you can still start the daemon as root user, it changes its user ID to the pkcsslotd user shortly after startup, thus dropping any root privileges. This protects your environment against privilege escalation attacks. Applications must still be running under a user that is a member in the pkcs11 group (see General access control).

The POSIX shared memory segments for the available tokens are located in path /dev/shm and are named either by their default names or by the name defined in the opencryptoki.conf file. For example, the shared memory segments may be located and named as shown in the following example:

[root@system01 shm]# pwd
/dev/shm
[root@system01 shm]# ls -l
...
-rw-rw----. 1 root pkcs11 82792 Apr  8 12:04 var.lib.opencryptoki.ccatok
-rw-rw----. 1 root pkcs11 82792 Apr  8 12:04 var.lib.opencryptoki.ep11tok
-rw-rw----. 1 root pkcs11 82792 Apr  8 12:04 var.lib.opencryptoki.lite /* legacy name: ICA Tok */
-rw-rw----. 1 root pkcs11 82792 Apr  8 12:04 var.lib.opencryptoki.swtok
...

The pkcsslotd daemon also uses one System V shared memory segment in addition to the mentioned POSIX shared memory segments. This System V shared memory segment can be listed with the ipcs command. It is used to share information about the processes currently using openCryptoki services and to share the global session count per slot.

/run/opencryptoki/pkcsslotd.socket
/run/opencryptoki/pkcsslotd.admin.socket

The /run/opencryptoki directory also contains the PID file (process identification file) that the pkcsslotd daemon creates.

The /run/opencryptoki directory is owned by the pkcsslotd user and pkcs11 group, but only the pkcsslotd user is allowed to write (drwx--x---). That way, only the pkcsslotd user (or root user) is able to start the pkcsslotd daemon.

When a process attaches to a slot and opens a session, pkcsslotd makes future processes aware that a process has a session open and locks out certain function calls, if the process needs exclusive access to the given token. The daemon constantly searches through its shared memory and ensures that when a process is attached to a token, this process is actually running. If an attached process terminates abnormally, pkcsslotd cleans up after the process and frees the slot for use by other processes.

Starting the slot manager

A prerequisite for accessing a token is a running slot manager daemon (pkcsslotd) .

$ systemctl start pkcsslotd.service   /* for Linux distributions providing systemd */     

$ systemctl enable pkcsslotd.service  /* for Linux distributions providing systemd */

To start the pkcsslotd daemon under the pkcsslotd user manually, use the following command:

runuser -u pkcsslotd pkcsslotd

This command can only be used as root user. Using the command su pkcsslotd pkcsslotd does not work, because the pkcsslotd user does not allow a login.

Main API

The main API for the STDLLs lies in /usr/lib/pkcs11/PKCS11_API.so. This API includes all the functions as outlined in the PKCS #11 API specification. The main API provides each application with the slot management facility. The API also loads token-specific modules (STDLLs) that provide the contained operations (cryptographic operations and session and object management). STDLLs are customized for each token type and have specific functions, such as an initialization routine, to allow the token to work with the slot manager. When an application initializes the subsystem with the C_Initialize call, the API loads the STDLL shared objects for all the tokens that exist in the configuration (residing in the shared memory) and invokes the token-specific initialization routines. In addition, a connection to the pkcsslotd slot manager and its shared memory segment is established.

Roles and sessions

PKCS #11 knows two different roles per token. Each role can authenticate itself by a PIN specific to that role and a token.

• The security officer (SO) initializes and manages the token and can set the PIN of the User.
• The (normal) User can login to sessions, create and access private objects and perform cryptographic operations. Users can also change their PINs.

A session is a token-specific context for one or more cryptographic operations. It maintains the intermediate state of multi-part functions, like an encryption of a message that is worked on one network packet at a time. Roughly speaking, within one session only one cryptographic operation can be processed at a time, but a program may open multiple sessions concurrently.

There are different types of sessions: Each session is either a read-only session or a read-write session and each session is either a public session or a User session. Read-only sessions may not create or modify objects. A public session can only access public objects whereas a User session can access the User's private and public objects. All sessions of a token become User sessions (PKCS #11 User sessions) after a login to one of the sessions of that token. Access to a specific token is controlled using PINs (User or security officer PINs).

The PKCS #11 API provides session functions like:

• C_OpenSession() and C_CloseSession() are used to open and close a session. A parameter in the C_OpenSession() invocation indicates the type of the session.
• C_Login() and C_Logout() are used to toggle sessions from public to User sessions and reverse. In order to login, the User PIN is required.
• C_GetSessionInfo() provides information on the type of a session.
• C_GetOperationState() and C_SetOperationState() are used to checkpoint and restart a multi-part cryptographic operation. Note that not all operations may support saving and restoring the state of operations.

Functions and mechanisms

The PKCS #11 API defines a small set of generic cryptographic functions to do the following tasks:

• encrypting and decrypting messages,
• computing digests (also called hashes) of messages,
• signing messages and verifying signatures,
• generating symmetric keys or asymmetric key pairs and deriving keys,
• wrapping and unwrapping keys,
• generating random numbers.

With the exception of the functions to generate random numbers, these generic cryptographic functions accept a mechanism parameter that defines the specific instance of that function. This is depicted in Figure 2 where an encryption function takes an AES_CBC mechanism as argument to encrypt a message with AES encryption in the cipher block chaining (CBC) mode of operation, using a key (and an initialization vector not shown in the figure).

Each cryptographic function is executed in the context of a session. Most cryptographic functions must be initialized by calling an initialization function that takes a session parameter, a mechanism parameter and function specific parameters like keys. For a cryptographic function Xyz the initialization function is called C_XyzInit(). Once a function is initialized, the actual function invocation can take place: either as a single part function of the form C_Xyz() or as a multi-part function where one or more calls to a function of the form C_XyzUpdate() are finalized by a call to C_XyzFinal().

Each token supports token-specific functions. The PKCS #11 API provides the function C_GetFunctionList() to obtain the functions available with a specific token. A mechanism describes a specific set of cryptographic operations. For example, the mechanism CKM_AES_CBC refers to AES encryption with the cipher block chaining (CBC) mode of operation. A mechanism may be required for performing one or more cryptographic functions, for example, the CKM_AES_CBC mechanism may be used to define encryption, decryption, wrapping and unwrapping functions whereas the mechanism CKM_ECDSA_KEY_PAIR_GEN which is a mechanism to generate keys for elliptic curve DSA signatures, only supports the key (pair) generation function.

The list of mechanisms supported depends on the tokens and can be queried using the C_GetMechanismList() function. Each mechanism has token-specific attributes like the set of supported functions, minimal and maximal supported key sizes, or a hardware support flag. These attributes are token-specific and can be queried with C_GetMechanismInfo(). Some mechanisms have mechanism parameters, for example, CKM_AES_CBC has a mechanism parameter to define the initialization vector (IV) required by the CBC mode of operation.

Prefixes for API components

View a table of prefixes that designate the most important components of the openCryptoki API.

Slot token dynamic link libraries (STDLLs)

STDLLs are plug-in modules to the main API. They provide token-specific functions beyond the main API functions. Specific devices can be supported by building an STDLL for the device. Each STDLL must provide at least a token-type specific initialization function. If the device is an intelligent device, such as a hardware adapter that supports multiple mechanisms, the STDLL can be thin because much of the session information can be stored on the device. If the device only performs a simple cryptographic function, all of the objects and the device status must be managed by the STDLL. This flexibility allows STDLLs to support any cryptographic device.

Shared memory

The slot manager sets up its database in a region of shared memory. Since the maximum number of processes allowed to attach to pkcsslotd is finite, a fixed amount of memory can be set aside for token management. This fixed memory allocation for token management allows applications easier access to token state information and helps ensure conformance with the PKCS #11 specification. In addition, the slot manager (pkcsslotd) communicates with the API layer using Unix domain sockets.

Also, each token sets up a shared memory segment to synchronize token objects across multiple processes using the token.

Objects and keys

openCryptoki provides various functions to generate the applicable types of objects, for example, certain types of keys.

In addition, openCryptoki recognizes a number of classes of objects, as defined in the CK_OBJECT_CLASS data type. Object classes are defined with the objects that use them. Typically, objects are generated by function calls that specify an appropriate mechanism used for the generation. Additionally, there is a function C_CreateObject() to create objects that are defined by input templates containing appropriate attributes. For example, data objects are defined by a data template CK_ATTRIBUTE dataTemplate[], or certificate objects are defined by a certificate template CK_ATTRIBUTE certificateTemplate[].

An object comprises a set of attributes, each of which has precisely one given value. For an example, have a look at Step 6 of the Sample openCryptoki program, where function C_GenerateKeyPair() uses the mechanism CKM_RSA_PKCS_KEY_PAIR_GEN to create an RSA private and public key pair. The attributes are assigned to the public and private keys to be generated with two different templates of type array of CK_ATTRIBUTE (one applicable for the private key and one applicable to the public key), which are input to the function call.

PKCS #11 objects belong to different orthogonal classes depending on their life span, access restrictions and modifiability:

• Session objects exist during the duration of a session whereas token objects are associated with a token and not with any running code. In other words, token objects are stored persistently across sessions and are visible for multiple processes using the token. Session objects disappear when the session is closed, and are only visible within the session that created the object.
• Private objects can only be accessed by the PKCS #11 Users if they have logged into the token, whereas public objects can always be accessed by both User and security officer.
• A token object that is read-write for a read-write session is read-only for a read-only session of another application at the same time.

In addition, each object has a set of attributes, especially the CKA_CLASS attribute, which determines which further attributes are associated with an object. Other typical attributes contain the value of an object or determine whether an object is a token object.

The PKCS #11 API provides a set of functions to manage objects, like for example, C_CreateObject(), C_CopyObject(), C_DestroyObject(), C_GetObjectSize(), C_GetAttributeValue(), C_SetAttributeValue(), C_FindObjects().

The most important object classes are those that implement keys: private keys (CKO_PRIVATE_KEY), public keys (CKO_PUBLIC_KEY) and secret keys (CKO_SECRET_KEY). Private and public keys are the members of an asymmetric key pair, whereas secret keys are symmetric keys or MAC keys.

There are many key specific attributes. For example, the Boolean attribute CKA_WRAP denotes whether a key may be used to wrap another key. The Boolean attribute CKA_SENSITIVE is only applicable for private and secret keys. If this attribute is TRUE, this means that the value of the key may never be revealed in clear text. There are key-type specific attributes like CKA_MODULUS, which is an attribute specific to RSA keys. Not all tokens support all key types with all possible attributes. CKA_PRIVATE causes the object data to be encrypted when stored in the token directory.

Object management functions to create keys are C_GenerateKey(), C_GenerateKeyPair() and C_DeriveKey(). To import a key not generated by one of the previously mentioned functions, you can use C_CreateObject() with all key specific attributes specified in the template. Alternatively, a wrapped key can be imported with C_Unwrap(), where a wrapped key is a standard representation of a key specific to the key type (for example, a byte array for secret keys or a BER encoding for other key types) that is encrypted by a wrapping key.

PKCS #11 defines multiple object classes, for example:

• Data objects
• Key objects
• Public key objects
• Private key objects
• Secret key objects
• Certificate objects

Each object class has its own set of attributes, where these attributes define an instance of an object from this class. There is one common attribute called CKA_CLASS for all object classes. This attribute defines the type (or class) of an object. For more information about objects, read How to create and modify objects and How to apply attributes to objects).

When an object is created or found on a token by an application, openCryptoki assigns it an object handle for that application’s sessions to access it (CK_OBJECT_HANDLE, CK_OBJECT_HANDLE_PTR).

PKCS #11 also defines objects for certificates. However, other than functions to operate on generic objects, no functions to operate on certificate objects are part of the PKCS #11 API.

Logging and tracing in openCryptoki

You can enable logging support by setting the environment variable OPENCRYPTOKI_TRACE_LEVEL. If the environment variable is not set, logging is disabled by default.

If a log level > 0 is defined in the environment variable OPENCRYPTOKI_TRACE_LEVEL, then log entries are written to file /var/log/opencryptoki/trace.<pid>. In this file name specification, <pid> denotes the ID of the running process that uses the current token.

The log file is created with ownership user, and group pkcs11, and permission 0640 (user: read, write; group: read only; others: nothing). For every application, which is using openCryptoki, a new log file is created during token initialization.

A log level > 3 is only recommended for developers and for collecting more information during problem reproductions.

Lock files

As of release 3.8, openCryptoki maintains the following lock files in the system: one global API lock file, one lock file per token instance, except for the TPM token. For the TPM token, openCryptoki keeps one lock file per user.

The lock files are stored in the following directories, if applicable:

# ls -lh /var/lock/opencryptoki/

LCK..APIlock
ccatok/LCK..ccatok
ep11tok/LCK..ep11tok
icsf/LCK..icsf
lite/LCK..lite
swtok/LCK..swtok
tpm/<USER>/LCK..tpm

The LCK..APIlock file serializes access to the shared memory from the pkcsslotd daemon and from the API calls issued from libopencryptoki.so.

The token-specific lock files serialize access to the shared memory and to objects in the token directory from the respective token library (for example, from libpkcs11_cca.so for the CCA token).

Thus, each lock file is used to protect the respective shared memory segments and objects in the token directory by letting threads wait for a required lock.

Use and purpose of openCryptoki features

In normal operation, the mentioned shared memory segments, the Unix domain sockets of the slot manager daemon (pkcsslotd), and the lock files should be transparent to users of openCryptoki. But in case of certain errors, such objects may remain from some previous use of openCryptoki and thus block new operations. In such cases, you need to know the locations of these objects and you must typically use certain operating system tools to remove them and enable a normal use of openCryptoki again.

Figure 3 shows the process flow within the Linux on IBM Z and IBM LinuxONE crypto stack. For example, an application sends an encryption request to the crypto adapter. Through various interfaces, such a request is propagated from the application layer down to the target crypto adapter. On its way down, the request passes through the involved layers: the standard openCryptoki interfaces, the adequate IBM Z® crypto libraries, and the operating system kernel. The zcrypt device driver finally sends the request to the appropriate cryptographic coprocessor. The resulting request output is sent back to the application just the other way round through the layer interfaces.