IBM Support

How do I configure Host On-Demand to use Java Secure Socket Extension for Transport Layer Security support

Question & Answer


How do you configure IBM Rational Host On-Demand v12 and higher to use Java Secure Socket Extension (JSSE) for Transport Layer Security (TLS) v1.0, v1.1 and v1.2?


Host On-Demand v12 and higher now support TLS V1.0, v1.1 and v1.2 using JSSE in addition to support of SSLite which supports TLS v1.0 (same as SSL V3.1). SSLite is no longer scheduled to be sunset, so MSIE and smart card support will continue to be supported with SSLite.


The following information is provided for the support of JSSE in Host On-Demand v12 and higher for 3270 Display, 5250 Display, and VT Display sessions and Host On-Demand v12 and higher for 3270 Printer, 5250 Printer sessions.

Transport Layer Security (TLS) and Secure Sockets Layer (SSL) are two different communication protocols that enable applications to communicate securely over the Internet using data encryption. TLS is based on SSL, but has a different initial handshake protocol and is more extensible. TLS and SSL are not inter-operable. That is, an application using TLS cannot communicate with an application running SSL. Both protocols are widely used.

NOTE: SSL V3 has been disabled in Host On-Demand v11.0.11 and higher due to the POODLE vulnerability.

Host On-Demand includes an option to securely connect 3270, 5250, VT display sessions and 3270, 5250 printer sessions over TLS v1.0, TLS v1.1, and TLS v1.2 using Java Secure Socket Extension (JSSE). To use this option, the JRE (IBM or Oracle) must be V7 or later. On JRE V6 or earlier, it is possible to configure a session to use JSSE, but the JSSE-based session can be run only with JRE V7 or later.

To enable this option, go to the session properties and follow these steps.

  1. Select TLS/SSL on the left panel of the session properties window.

  2. Select Yes for the Use JSSE setting. This enables using the JSSE security library instead of SSLite for the session.

  3. Select TLSv1.0, TLSv1.1, or TLSv1.2 for the TLS Version setting as appropriate.

    Note: If you select No for the Use JSSE setting, the SSLite library is used and TLS v1.1 and TLS v1.2 are not available for the session.

When you select Yes to use the Use JSSE setting and select one of the three available TLS protocols, all lower protocols, starting from TLS v1.0 are enabled.

If FIPS mode is not enabled, the SSL v3 protocol is enabled along with the other configured TLS protocols. If FIPS mode is enabled, the SSL v3 protocol is not enabled.

In the TLS/SSL panel of the session properties window, you cannot configure JSSE to use the SSL v3 protocol. If you select Yes for the Use JSSE setting, and then on the Connection panel select Telnet - SSL only  for the Protocol setting, the Use JSSE setting is automatically changed to No, and you are advised to review the TLS/SSL settings. For example, you might need to configure a PKCS12 formatted keystore containing a client certificate for SSLite.

Using server certificates

The type of truststore and keystore used with JSSE is in Java KeyStore (JKS) format. The CustomizedCAs.jks file is a certificate truststore used by Host On-Demand, if necessary, to trust the server's certificate during the TLS or SSL handshake. If you use a certificate from an unknown Certificate Authority (CA) or a self-signed certificate, you must create or update the CustomizedCAs.jks file. This file must contain the root certificates of unknown CAs and self-signed certificates that are not in the WellKnownTrusted list. Host On-Demand does not install a CustomizedCAs.jks file by default.

The CustomizedCAs.jks file is different from the CustomizedCAs.p12 file which is used for SSLite when the Use JSSE setting is set to No. You can create a CustomizedCAs.jks file either by converting the existing CustomizedCAs.p12 to JKS format or by creating a new file in this format. You can use the Certificate Management utility that is installed with Host On-Demand for this purpose. You can also use the keytool.exe command-line tool, which is a Java Key and Certificate Management Tool available in the JRE. Refer to Java documentation for further details and instructions on using this tool for both IBM JDK Version 7 and Oracle Key and Certificate Management Tool or How to convert Host On-Demand CustomizedCAs.p12 to CustomizedCAs.jks file. The password for CustomizedCAs.jks must be hodpwd. The password for CustomizedCAs.p12 must be hod. The CustomizedCAs.jks file must be present in the Host On-Demand publish directory. CustomizedCAs.class file is not supported by JSSE.

For HOD servers on UNIX, Linux, or z/OS platforms, verify the CustomizedCAs.jks file has permissions of 644 (rw-r--r--).

Using client certificates

In the TLS/SSL panel of the session properties window, you can configure the session to send a client certificate to the server. In this case, you must provide a Java KeyStore (JKS) formatted file, a file with .jks extension, containing the client certificate. If the client certificate is signed using signer certificates, the entire certificate chain must be present in the same JKS keystore file. You can use the Certificate Management utility installed with Host On-Demand or the Java keytool.exe command-line tool to create and edit the JKS files. You can use these same tools to convert existing PKCS12 keystores, with .p12 file extensions, to JKS format for use with JSSE.

Note: Due to certain differences in design between JSSE and SSLite (the TLS/SSL library also supported by Host On-Demand), some change in behavior might be observed when the session is run in JSSE mode as compared to SSLite. For example, using SSLite, the client user is prompted for a client certificate when the server requests a client certificate during TLS/SSL negotiations. However, using JSSE, a request for a client certificate from the server does not trigger the prompt to the client user. Instead, the client user is prompted for a client certificate only if the Host On-Demand session properties are configured to send a client certificate. Therefore, if a host requests a client certificate, but the session properties are not configured to send a certificate, the host might disconnect the session with a handshake_failure error message.

Proxy support

Enabling the JSSE function does not affect the existing function of the Proxy configuration. Host On-Demand sessions configured to use HTTPS proxy continue to use SSLite to connect to the proxy server. If you select Yes for the Use JSSE setting on the TLS/SSL panel and select Yes for the Use HTTPS to proxy setting on the Proxy Server panel, both the CustomizedCAs.jks and the CustomizedCAs.p12 files must be made available on the Host On-Demand Server. The CustomizedCAs.jks file is used for the JSSE-based TLS connections for the display session and printer session, and the CustomizedCAs.p12 file is used for the SSLite-based SSL connections to the proxy server. Each must have the appropriate trusted CA server certificates.

Error messages

Error messages during TLS/SSL negotiations are thrown by Java. If Host On-Demand is able to recognize the error, it maps the error to a known Host On-Demand message before displaying it on the session status bar. If not, a generic error message is displayed on the status bar to indicate that there is a problem in the Security component, for example when a handshake_failure error message is received from the JRE. It also includes the specific Java error message, if any. Along with this, the error and stack trace are generally printed on the java console.

Handshake failure error messages can be due to several reasons, some of which are:

  • A client certificate was requested by the server but not received.
  • The set of ciphers or other TLS/SSL parameters supported by the client did not match those of the server.
  • The Host On-Demand session was configured with a client certificate, but the certificate was not accepted by the server.


Limitations of support for Host On-Demand v12 and higher:

  • JSSE function is supported only for Host On-Demand 3270, 5250, and VT display session, and 3270 ,5250 printer sessions running in the web browser (Cached or Download) and in Web Start clients.
  • JSSE function is supported for FTP client but only v13 on-wards.
  • JSSE function is not supported for CICS sessions or going through the redirector. Sessions that do not have support for JSSE can still use the existing TLS/SSL function up to TLS v1.0 protocol using SSLite.
  • IBM or Oracle Java V7, or later, must be used in order to run sessions configured with JSSE.
  • JSSE function does not support:
    • PKCS12 (.p12) file format
    • Telnet-negotiation (3270)
    • Expeditor plugin or Portal environments
    • Option to Retrieve certificate before connect
    • Option to set FIPS Mode as a session parameter (use HTML parameter instead shown below)
  • TLS/SSL-based secure connections may be slightly slower in comparison with non-secure Telnet connections due to additional computation involved.

You can use the following HTML parameters to configure other functions for your Host On-Demand JSSE-based sessions.

FIPS Mode:

HTML parameter name: FipsMode

Parameter values: true or false

Default value: true

Description: FIPS mode is supported only in IBM Java. When using IBM Java, Host On-Demand enables FIPS mode by default. The FIPS function is implemented by the JRE. When this parameter is set to false, Host On-Demand does not enable FIPS mode in IBM Java. When FIPS mode is enabled, the following FIPS ciphers are set by Host On-Demand:


Cipher customization:

HTML parameter name: JSSECustomCiphers

Parameter values: The value is divided into two parts separated by a colon (:), for example, Part1:Part2.

A third part, which is optional, can be added with another colon (:), for example, Part1:Part2:Part3.

Valid values for Part1:Part2:Part3 are described below.

Part1 : One of the following values:

  • Add
    Use this value to indicate that the ciphers specified in this HTML parameter are to be added to the existing ciphers that are used by default. That is, JSSE will use its pre-configured ciphers, in addition to the ones specified in this HTML parameter.
  • UseOnly
    Use this value to indicate that the ciphers specified in this HTML parameter are the only ciphers to be used. In this case, the default pre-configured ciphers are replaced with the ones specified in this parameter.
Note: If none of the specified ciphers are supported by JSSE, and the Part 3 value is not specified, the cipher list enabled in the client will be empty, and the session might not connect.

Part 2 : A comma (,) separated list of ciphers to be used by JSSE.

Part 3 : (Optional) The following value:

  • SkipValidation
    Use this value to indicate not to validate the specified ciphers against the supported list in the client JSSE library. In other words, each cipher specified in this HTML parameter is set in JSSE in the Host On-Demand client regardless of whether it is supported by JSSE. In this case, if an unsupported cipher is included, any resulting error is thrown by Java and the session might not connect.

    This adds the two ciphers SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA and SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA to the pre-configured list of ciphers in the JSSE for the client.
    This specifies that the two listed ciphers SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA and SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA are the only ciphers enabled to be used by JSSE for the client.
    Note : If for any reason these two ciphers are not supported by JSSE, an empty list of ciphers is set in JSSE resulting in the session possibly getting disconnected.
    This adds the two ciphers SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA and SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA to the pre-configured list of ciphers in the JSSE client. Here, since SkipValidation is also added, the ciphers are not validated to ensure support in JSSE.

Default value: Not applicable. When this parameter is not set, every JSSE client uses its pre-configured list of ciphers.

Description: By default, a JSSE client uses a pre-configured set of ciphers when connecting to a server over TLS/SSL. Using this HTML parameter, you can customize the set of ciphers that are enabled by JSSE in the client.

You can specify ciphers to be added to the default set that are already used by the JSSE client, or specify to use only those ciphers that are listed by the administrator.

In each case, Host On-Demand validates each cipher specified in this HTML parameter to ensure that the JSSE client supports the cipher. Only the ones supported by the JSSE client are used. An option to skip this validation is also supported.

Note:The Java on the client workstation must be upgraded with the JCE Unlimited Strength Jurisdiction Policy Files, If user is using TLS v1.2 connection with an AES 256-bit key.

Disable JSSE Truststore Caching:

HTML parameter name: DisableCachingInJSSE

Parameter values: true or false

Default value: false

Description: In an attempt to avoid multiple downloads of the truststore from the Host On-Demand server, Host On-Demand client support for JSSE uses a caching mechanism to store the truststore, CustomizedCAs.jks. You can use this HTML parameter to disable this function. When this parameter is set to true, Host On-Demand disables caching of the truststore in JSSE mode.

[{"Product":{"code":"SSS9FA","label":"Rational Host On-Demand"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Component":"Documentation","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF012","label":"IBM i"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"11.0.9;11.0.10;11.0.11;11.0.12;11.0.13;11.0.14;12.0.0","Edition":"","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
02 August 2018