IBM Support

Enabling hardware cryptography for Liberty for z/OS using Java 8

How To


Summary

The JVM on z/OS can make use of the the following cryptographic providers for encryption, decryption and SSL operations.

IBMJCE (software provider)
IBMJCECCA (hardware provider)
IBMJCEHYBRID (hybrid provider)

The JVM ships with the IBMJCE provider enabled by default.

Objective

This document provides the steps necessary to enable the IBMJCECCA and IBMJCEHYBRID providers to make use of hardware cryptography,
and to provide failover from hardware to software providers when running in a Liberty for z/OS server.
When using the IBMJCE provider, the cryptographic operations for SSL are performed in software and the certificate's private key is stored in RACF.

When using the IBMJCECCA provider, the certificate's private key can be stored in either RACF, ICSF, or PKDS and the IBMJCECCA provider can be enabled to use the crypto express hardware to assist with off-loading public key operations during SSL handshaking via ICSF.

If the personal certificate's private key is stored in ICSF or PKDS, the IBMJCECCA provider is needed in order to allow ICSF to access the certificate's private key.

 

Steps

To enable the IBMJCECCA provider.
1. Create a file called java.security in the same directory as your server.xml
with the
IBMJCECCA as the first provider,
IBMJCE as the second provider
as in the example below:
------------
security.provider.1=com.ibm.crypto.hdwrCCA.provider.IBMJCECCA        
security.provider.2=com.ibm.crypto.provider.IBMJCE
security.provider.3=com.ibm.crypto.plus.provider.IBMJCEPlus
security.provider.4=com.ibm.jsse2.IBMJSSEProvider2
security.provider.5=com.ibm.security.jgss.IBMJGSSProvider             
security.provider.6=com.ibm.security.cert.IBMCertPath                 
security.provider.7=com.ibm.security.sasl.IBMSASL  
security.provider.8=com.ibm.xml.crypto.IBMXMLCryptoProvider           
security.provider.9=com.ibm.xml.enc.IBMXMLEncProvider                 
security.provider.10=com.ibm.security.jgss.mech.spnego.IBMSPNEGO      
security.provider.11=sun.security.provider.Sun
------------

The providers above come from JAVA_HOME/lib/security/java.security file and are included in the list after IBMJCECCA and IBMJCE providers.  
The providers are renumbered starting with the IBMJCECCA.
2.  In the same directory as your server.xml, create a file called jvm.options containing the location of a new java.security file.
-Djava.security.properties=${server.config.dir}/java.security
Although it is possible to edit the JAVA_HOME/lib/security/java.security file, the JVM is usually installed in a file system
that is read only, and the file will get overwritten when new Java maintenance is installed.   By using the custom property java.security.properties, the server can append new providers to the existing java.security file shipped by the JVM,
and changes will remain when Java maintenance is applied.
Note that a single "=" sign used by java.security.properties indicates the changes in this file will append to the JVM's java.security file.  
So the providers in the newly created java.security will take affect over the settings in the JAVA_HOME/lib/security/java.security file.
Avoid using a double "==" sign after java.security.properties as this will override all settings the JAVA_HOME/lib/security/java.security  file with the new java.security file that was created.
3.  Ensure that the server.xml has the either the transportSecurity-1.0 or ssl-1.0 enabled.
------------
<feature>transportSecurity-1.0</feature>
or
<feature>ssl-1.0</feature>
------------
4.  Ensure the correct location and type are specified in the keystore tag in the server.xml.
If the certificate has its private key in software, but the JVM is leveraging the crypto device to offload crypto operations.
------------
<keyStore id="CellDefaultKeyStore"
location="safkeyring:///LibertyKeyring"
filebased="false"
password="password" type="JCERACFKS"/>
<keyStore id="CellDefaultTrustStore"
location="safkeyring:///LibertyKeyring"
filebased="false"
password="password" type="JCERACFKS"/>
------------
If the certificate has its private key in hardware or both hardware and software, and the JVM is leveraging the crypto device to offload crypto operations as well as access the certificate's private key from the crypto device.
------------
<keyStore id="CellDefaultKeyStore"
location="safkeyringhw:///LibertyKeyring"
filebased="false"
password="password" type="JCECCARACFKS"/>
<keyStore id="CellDefaultTrustStore"
location="safkeyringhw:///LibertyKeyring"
filebased="false"
password="password" type="JCECCARACFKS"/>
------------
When using only the IBMJCECCA provider with a personal certificate that has its private key in hardware,
the location must be safkeyringhw and the type must be JCECCARACFKS.
The following RACF command can be used to identify certificates with keys in hardware crypto.
RACDCERT LIST(label('personal_certificate_labelname')) ID(Liberty_ID)

For example the certificate might show a private key of type ICSF or a private key with a PKDS label.
------------
Private Key: ICSF
or
Private Key: YES
PKDS Label: IRR.DIGTCERT.Liberty_ID.xxx.nnnnnnnnnn
------------
5.  If you protect ICSF services using rules in the CSFSERV class, the Liberty server's userid must be permitted to the necessary ICSF callable services.

The following RACF command can be used to determine if CSFSERV is active under "ACTIVE CLASSES"
SETROPTS LIST

The following RACF command can be used to determine which ICSF callable services in the CSFSERV class are defined.
RLIST CSFSERV * ALL
Permit the Liberty for z/OS id to the ICSF callable services in the CSFSERV resource class which can be found in
RACF CSFSERV resource requirements from z/OS Cryptographic Services infocenter.
Below is an example command to permit the
Liberty_ID to ICSF callable service CSFDSV in the CSFSERV resource class.
------------
RDEFINE CSFSERV CSFDSV UACC(NONE)
PERMIT CSFDSV CLASS(CSFSERV) ID(Liberty_ID) ACCESS(READ)
SETROPTS RACLIST(CSFSERV) REFRESH
------------
To enable the IBMJCEHYBRID provider. 
1 . Follow all steps To enable the IBMJCECCA provider first, and confirm that SSL is working with the IBMJCECCA provider.
The IBMJCEHYBRID provider does not do any cryptographic operations, but routes requests to JCE providers registered with the Java Security Framework.   It will make use of the IBMJCECCA provider, and provides failover to the IBMJCE provider in the event that the IBMJCECCA provider experiences a problem.   For example, the Liberty server is started with the ICSF address space stopped. 
It is recommended that you attempt the setup for the IBMJCECCA provider first before switching to the IBMJCEHYBRID provider.  It becomes difficult to confirm whether the IBMJCECCA is working as wanted since the IBMJCEHYBRID may be in its failover process to the IBMJCE provider when an SSL operation is taking place.
2. Update the java.security file to contain the
IBMJCEHYBRID as the first provider,
IBMJCECCA as the second provider,
IBMJCE as the third provider
as in the example below:
------------
security.provider.1=com.ibm.crypto.ibmjcehybrid.provider.IBMJCEHYBRID
security.provider.2=com.ibm.crypto.hdwrCCA.provider.IBMJCECCA        
security.provider.3=com.ibm.crypto.provider.IBMJCE
security.provider.4=com.ibm.crypto.plus.provider.IBMJCEPlus
security.provider.5=com.ibm.jsse2.IBMJSSEProvider2
security.provider.6=com.ibm.security.jgss.IBMJGSSProvider             
security.provider.7=com.ibm.security.cert.IBMCertPath                 
security.provider.8=com.ibm.security.sasl.IBMSASL  
security.provider.9=com.ibm.xml.crypto.IBMXMLCryptoProvider           
security.provider.10=com.ibm.xml.enc.IBMXMLEncProvider                 
security.provider.11=com.ibm.security.jgss.mech.spnego.IBMSPNEGO      
security.provider.12=sun.security.provider.Sun
------------
3.  Change location and type specified in the keystore tag in the server.xml.
------------
<keyStore id="CellDefaultKeyStore"
location="safkeyringhybrid:///LibertyKeyring"
filebased="false"
password="password" type="JCEHYBRIDRACFKS"/>
<keyStore id="CellDefaultTrustStore"
location="safkeyringhybrid:///LibertyKeyring"
filebased="false"
password="password" type="JCEHYBRIDRACFKS"/>
------------
When using the IBMJCEHYBRID provider,
the location must be safkeyringhybrid and the type must be JCEHYBRIDRACFKS.

Additional Information

1. In case of an SSL error enable Java trace to confirm all ICSF callable services in the CSFSERV resource class have been permitted to the Liberty for z/OS address space id.

In jvm.options add:
-Djava.security.auth.debug=all
Restart the server, and recreate the SSL error.
Search on all cases of "SEVERE" or "reason code" to identify permission problems.
The following USS commands can be used to display the "SEVERE" errors from messages.log to standard out.
cat messages.log | iconv -f ISO8859-1 -t IBM-1047 | grep -i 'SEVERE'

For example:
SEVERE: Hardware error from call CSNDDSV returnCode=12, reasonCode=11060
The following USS commands can be used to display ICSF callable services "reason code"  values from messages.log to standard out. 
This can be useful for confirming that requests are being handled by hardware when using the IBMJCEHYBRID provider.
cat messages.log | iconv -f ISO8859-1 -t IBM-1047 | grep -i 'reason code'

For example:
FINE: CSNBRNGL: return code = 0 reason code = 0
FINE: CSNDDSG: return code = 0 reason code = 0
FINE: CSNBSYD: return code = 0 reason code = 0
FINE: CSNBSYE: return code = 0 reason code = 0

Review
to see what the returnCode and reasonCode mean.

This trace requires a restart of the Liberty for z/OS server to enable and disable.
2. If there are no "SEVERE" errors, but SSL errors still persist,
the following JSSE trace can be enabled to narrow down the SSL issue.
Add line to jvm.options
-Djavax.net.debug=true
This trace requires a restart of the Liberty for z/OS server to enable and disable.

3. In the server.xml add the Liberty SSL traceSpecification to the logging tag:

<logging traceSpecification="*=info:SSL=all:SSLChannel=all" />

Alternatively the Liberty trace can also be enabled dynamically to limit trace output by issuing the MVS console command

F JOBNAME,LOGGING='SSL=all'
F JOBNAME,LOGGING='SSLChannel=all'

Where JOBNAME is the Liberty Application Server region jobname.

Recreate the problem

Then reset the trace back to what the server started with (ie. *=info)

F JOBNAME,LOGGING=RESET

4. For production environments where it's too difficult to obtain traces by restarting the server,
or if there are too many SSL requests for a dynamic trace, the following JVM keytool/hwkeytool
commands can be used to confirm if certificates are accessible by the JVM.

The user must be logged in to OMVS shell or USS with the same ID the Liberty for z/OS id runs under.
If your Liberty userid does not have a password, the USS commands below can be copied into a shell script, and run from JCL with the "USER=Liberty_ID" specified in the JCL. 

//Indicate to the JVM where your java.security file is located with the security providers
export IBM_JAVA_OPTIONS=-Djava.security.properties=/WebSphere/Liberty/servers/defaultServer/java.security

//If your certificate keys are in software, use the keytool command to list the certificates on the Liberty keyring
keytool -list -v -storetype JCERACFKS -keystore safkeyring:///LibertyKeyring
-J-Djava.protocol.handler.pkgs=com.ibm.crypto.provider

//If your keys are in hardware, use the hwkeytool command to list the certificates on the Liberty keyring
hwkeytool -list -storetype JCECCARACFKS -keystore safkeyring:///LibertyKeyring
-J-Djava.protocol.handler.pkgs=com.ibm.crypto.hdwrCCA.provider 

The keytool command and hwkeytool command should list the certificates on the keyring for the user logged in to the OMVS shell (ie. Liberty_ID). 
If not, this is an indication that there is a problem accessing the certificate from the keyring and the
-Djava.security.auth.debug=all can be added to keytool or hwkeytool to determine the reason. 

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSD28V","label":"WebSphere Application Server Liberty Core"},"ARM Category":[{"code":"a8m0z0000001h7hAAA","label":"Liberty-\u003ELiberty z\/OS"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
25 January 2023

UID

ibm16209109

Page Feedback