Installation error: SSL handshake failure

Description

During InfoSphere MDM installation, if the IBM® WebSphere® Application Server SSL handshake popup window cannot open as expected, then the node agent cannot process interactions with the InfoSphere MDM installation application. This issue causes the InfoSphere MDM installation to fail and trigger a complete installation rollback.

The warning message is similar to this IBM Installation Manager log file example message:

Updating property file: /app_2/IBM/MDM/KM1Dev05/mdm/properties/sync_nodes.properties
Updating property file: /app_2/IBM/MDM/KM1Dev05/mdm/properties/sync_nodes.properties
Updating property file: /app_2/IBM/MDM/KM1Dev05/mdm/properties/sync_nodes.properties
Updating property file: /app_2/IBM/MDM/KM1Dev05/mdm/properties/sync_nodes.properties
Updating property file: /app_2/IBM/MDM/KM1Dev05/mdm/properties/sync_nodes.properties
########################### entering SyncNodesTask ###########################
 wasUser is mdmadmin 
 wasPwd is ******** 
 hostname is M2848 
 port is 28879 
 trustFile is /m1/mdm/WebSphere85dev/AppServer/etc/DummyClientTrustFile.jks 
 trustFilePwd is ***** 
 node is None 
 CWPKI0308I: Adding signer alias "CN=M2848, OU=Root Certifica" to local 
            keystore "null" with the following SHA digest: 
            33:1A:02:2A:C1:10:77:AD:09:3F:FF:CB:F8:8B:49:53:22:3F:5A:89 
   
 CWPKI0022E: SSL HANDSHAKE FAILURE: A signer with SubjectDN "CN=M2848, OU=c1_mdm_dev, OU=dm_c1_mdm_dev, O=ACME5, C=US" was sent from target host:port "10.87.17.230:28879". The signer may need to be added to local trust store "/m1/mdm/WebSphere85dev/AppServer/etc/DummyClientTrustFile.jks" located in SSL configuration alias "null" loaded from SSL configuration file "null".  The extended error message from the SSL handshake exception is: "PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is: 
             java.security.cert.CertPathValidatorException: The certificate issued by CN=M2848, OU=Root Certificate, OU=O1_mdm_dev, OU=dm_c1_mdm_dev, O=ACME5, C=US is not trusted; internal cause is: 
             java.security.cert.CertPathValidatorException: Certificate chaining error". 
 CWPKI0040I: An SSL handshake failure occurred from a secure client.  The server's SSL signer has to be added to the client's trust store.  A retrieveSigners utility is provided to download signers from the server but requires administrative permission.  Check with your administrator to have this utility run to setup the secure environment before running the client.  Alternatively, the com.ibm.ssl.enableSignerExchangePrompt can be enabled in ssl.client.props for "DefaultSSLSettings" in order to allow acceptance of the signer during the connection attempt. 
 Exception in com.ibm.mdm.de.ant.task.SyncNodesTask 
 com.ibm.websphere.management.exception.ConnectorException: ADMC0016E: The system cannot create a SOAP connector to connect to host M2848 at port 28879. 
 ADMC0016E: The system cannot create a SOAP connector to connect to host M2848 at port 28879. 
 ########################### existing SyncNodesTask ###########################

Solution

1. Apply one of the following workarounds:
   • Add the server's SSL signer to the client's trust store:
     1. Use the retrieveSigners utility to download the signers from the server.
     2. Add the signer to the client's trust store.
     Note: The retrieveSigners utility requires administrative permissions. Check with your administrator to run this utility to set up the secure environment before running the client.
   • Allow acceptance of the signer during connection attempts:
     • In the ssl.client.props file for DefaultSSLSettings, enable the com.ibm.ssl.enableSignerExchangePrompt property to allow acceptance of the signer during the connection attempt.
2. Run the InfoSphere MDM installation again.