SSL fails when host name configuration fails

IBM® Business Process Manager uses host name verification for outbound connections that use Secure Sockets Layer (SSL). Connections are refused if the host name that the server connects to does not match the common name (CN) in the SSL certificate. This problem is most likely to occur when the initial configuration used localhost as a host name.

Host name verification was introduced as a security update in IBM BPM 8.0.1.1 and is applied in later releases. The update was also provided as an interim fix for earlier releases. When SSL fails verification, you receive this exception message:
javax.net.ssl.SSLException: hostname in certificate didn't match: requested_URL_or_SSL_connection != certificate_common_name 

For more information about host names, see the related links at the end of this topic.

Certificate configuration

When a connection is established to a secure port, the initial handshake involves verifying the certificates. When you connect to a remote server over HTTPS, IBM BPM expects the common name in the SSL certificate of the remote server to match the host name of the computer that it connected to. However, there are several scenarios in which IBM BPM connects to itself using HTTPS. Therefore, IBM BPM must be set up with a certificate that has a common name that matches the host name that IBM BPM uses when it connects to itself.

When a profile is created, IBM WebSphere® Application Server by default generates a self-signed root certificate that is valid for 15 years. In a distributed environment, a certificate is generated for each node and signed with the root certificate. The common name (CN) in the certificate is the same as the host name that is specified during profile creation.

SSL is a point-to-point connection. The common name in the certificate must match the host name of the computer that is trying to connect. When IBM BPM is configured to connect to itself through a web server, the web server must be set up with a certificate that has a common name that must match the host name that is used by IBM BPM to connect to this web server.

Problem scenarios and solutions

One problem occurs if you install and test IBM BPM using localhost as your host name. Later, if you try to connect with an external name, for example https://myname.mycompany.com:9443/bpm/rest, or if you try to connect from another computer, the verification fails. The failure of the connection generates an error in Process Inspector and the Process Admin Console. To avoid this problem, the configuration documentation warns against using localhost as the host name. Particularly, the configuration documentation warns against using localhost for environments that are spread across multiple computers.

If you have a locally installed Process Center that you use for your own development purposes, set up your environment with a host name such as bpm.company.com. On the Windows operating system, set this environment in your Windows hosts file. Always use that host name to access the Process Center server.

In a production environment, always access clusters through the HTTP server. The HTTP server must be accessible from IBM BPM and must have a fully qualified host name with a matching certificate.