Creating TEMADB database failed with "The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption" error.

Cause

Environment:  BigFix, MS SQL Server and ILMT installed on the same Windows server


BigFix server is installed successfully with its dbs created.   When attempting to create TEMADB db for ILMT, it failed to contact the MS SQL Server.   The tema.log contained this errors below.


[6/14/17 16:19:07:996 UTC] 00000054 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[ERROR] RECOMMENDED USER ACTION: Unreachable Database. Check your database name or host connection. It looks like the configuration you have provided is incorrect.

[6/14/17 16:19:08:246 UTC] 00000054 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[INFO]   Rendered setup/database.html.rb (235.0ms)

[6/14/17 16:19:08:246 UTC] 00000054 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[INFO] Completed 200 OK in 46831ms (Views: 235.0ms)

[6/14/17 16:20:06:889 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[INFO] Started POST "/setup/database" for 0:0:0:0:0:0:0:1 at 2017-06-14 16:20:06 +0000

[6/14/17 16:20:06:889 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[INFO] Processing by SetupController#database as HTML

[6/14/17 16:20:06:889 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[INFO]   Parameters: {"utf8"=>"?", "authenticity_token"=>"[FILTERED]", "database_config"=>{"database_type"=>"mssql", "mssql_host"=>"localhost", "mssql_database"=>"temadb", "mssql_windows_authenticated"=>"true", "mssql_username"=>"", "mssql_password"=>"[FILTERED]"}, "commit"=>"Creating..."}

[6/14/17 16:20:06:905 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[WARN] Can't verify CSRF token authenticity

[6/14/17 16:20:53:407 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[ERROR] Database connection error: Java::ComMicrosoftSqlserverJdbc::SQLServerException: The TCP/IP connection to the host ilmthostserver, port 1433 has failed. Error: "null. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".


.
.
.
.
.

[6/14/17 16:20:53:407 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[ERROR] RECOMMENDED USER ACTION: Unreachable Database. Check your database name or host connection. It looks like the configuration you have provided is incorrect.

[6/14/17 16:20:53:454 UTC] 00000089 com.microsoft.sqlserver.jdbc.internals.TDS.Channel           I java.security path: C:\Program Files\ibm\LMT\jre\jre\lib\security
Security providers: [IBMJSSE2 version 1.8, IBMJCE version 1.8, IBMJGSSProvider version 8.0, IBMCertPath version 1.8, IBMSASL version 1.8, IBMXMLCRYPTO version 8.0, IBMXMLEnc version 8.0, IBMSPNEGO version 8.0, SUN version 1.8]
SSLContext provider info: IBM JSSE provider2 (implements IbmX509/PKIX key/trust factories, SSLv3/TLSv1/TLSv1.1/TLSv1.2)
SSLContext provider services:
[IBMJSSE2: KeyManagerFactory.IbmX509 -> com.ibm.jsse2.ae$a
, IBMJSSE2: KeyManagerFactory.NewIbmX509 -> com.ibm.jsse2.ae$b
 aliases: [PKIX]
, IBMJSSE2: TrustManagerFactory.IbmX509 -> com.ibm.jsse2.at$b
, IBMJSSE2: TrustManagerFactory.PKIX -> com.ibm.jsse2.at$a
 aliases: [IbmPKIX, X509, X.509]
, IBMJSSE2: SSLContext.SSL -> com.ibm.jsse2.ah
, IBMJSSE2: SSLContext.TLS -> com.ibm.jsse2.al
, IBMJSSE2: SSLContext.TLSv1 -> com.ibm.jsse2.am
, IBMJSSE2: SSLContext.TLSv1.1 -> com.ibm.jsse2.an
, IBMJSSE2: SSLContext.TLSv1.2 -> com.ibm.jsse2.ao
, IBMJSSE2: SSLContext.SSL_TLS -> com.ibm.jsse2.ai
, IBMJSSE2: SSLContext.SSL_TLSv2 -> com.ibm.jsse2.aj
, IBMJSSE2: SSLContext.Default -> com.ibm.jsse2.ag
]
java.ext.dirs: C:\Program Files\ibm\LMT\jre\jre\lib\ext
[6/14/17 16:20:53:454 UTC] 00000089 com.microsoft.sqlserver.jdbc.internals.TDS.Channel           I java.security path: C:\Program Files\ibm\LMT\jre\jre\lib\security
Security providers: [IBMJSSE2 version 1.8, IBMJCE version 1.8, IBMJGSSProvider version 8.0, IBMCertPath version 1.8, IBMSASL version 1.8, IBMXMLCRYPTO version 8.0, IBMXMLEnc version 8.0, IBMSPNEGO version 8.0, SUN version 1.8]
SSLContext provider info: IBM JSSE provider2 (implements IbmX509/PKIX key/trust factories, SSLv3/TLSv1/TLSv1.1/TLSv1.2)
SSLContext provider services:
[IBMJSSE2: KeyManagerFactory.IbmX509 -> com.ibm.jsse2.ae$a
, IBMJSSE2: KeyManagerFactory.NewIbmX509 -> com.ibm.jsse2.ae$b
 aliases: [PKIX]
, IBMJSSE2: TrustManagerFactory.IbmX509 -> com.ibm.jsse2.at$b
, IBMJSSE2: TrustManagerFactory.PKIX -> com.ibm.jsse2.at$a
 aliases: [IbmPKIX, X509, X.509]
, IBMJSSE2: SSLContext.SSL -> com.ibm.jsse2.ah
, IBMJSSE2: SSLContext.TLS -> com.ibm.jsse2.al
, IBMJSSE2: SSLContext.TLSv1 -> com.ibm.jsse2.am
, IBMJSSE2: SSLContext.TLSv1.1 -> com.ibm.jsse2.an
, IBMJSSE2: SSLContext.TLSv1.2 -> com.ibm.jsse2.ao
, IBMJSSE2: SSLContext.SSL_TLS -> com.ibm.jsse2.ai
, IBMJSSE2: SSLContext.SSL_TLSv2 -> com.ibm.jsse2.aj
, IBMJSSE2: SSLContext.Default -> com.ibm.jsse2.ag
]
java.ext.dirs: C:\Program Files\ibm\LMT\jre\jre\lib\ext
[6/14/17 16:20:53:469 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[ERROR] Database connection error: Java::ComMicrosoftSqlserverJdbc::SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "SQL Server did not return a response. The connection has been closed. ClientConnectionId:a1ba20cc-9292-43e7-8477-ebb1bc048068".

[6/14/17 16:20:53:469 UTC] 00000089 com.ibm.ws.webcontainer.webapp                               I SRVE0292I: Servlet Message - [tema]:.[DEBUG] com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:1667)


==============

The errors above clearly indicated problem with SSL connection between the ILMT application and MS SQL Server.  With BigFix application, it has no issue connecting to MS SQL Server via ODBC connection.  With ILMT application, it does not use ODBC connection as it uses the "IBM JSSE provider2 (implements IbmX509/PKIX key/trust factories, SSLv3/TLSv1/TLSv1.1/TLSv1.2)"  to communicate with the MS SQL server.  

Also verified the MS SQL Server TCP/IP port 1433 is listening and Windows Firewall is also disabled as well.  Nothing helped.