Test Connectivity to a Secure API Using OpenSSL

Problem

While configuring a IBM Manta Data Lineage scanner, you may get a validation error relating a failure to establish a connection to the service that should be scanned. In order to troubleshoot this error and understand whether it is a Manta Data Lineage related issue or some other issue (typically networking, firewall, or proxy related), it is good to test if the machine you are trying to connect is accessible from the machine where Manta Data Lineage is installed. To test connectivity to a secure API independently of Manta Data Lineage, we can use third-party tools such as OpenSSL (command line) or Postman (the tool with a graphical front-end).

This article focuses on the OpenSSL command line tool that allows for the testing and debugging of secure HTTPS connections (SSL or TLS enabled HTTP connections). This approach can also be used for some other protocols (e.g., secure JDBC for Oracle), but it is not universal. (It cannot be used directly for MS SQL, Netezza, etc.)

More Information

The typical messages indicating connectivity issues are:

  1. Connection timed out

    Connection timed out: connect
                at java.net.DualStackPlainSocketImpl.connect0(Native Method)
                at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
                at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
                at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
                at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
                at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
                at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
                at java.net.Socket.connect(Socket.java:589)
                at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
                at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
                at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
                at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
                at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
                at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
                at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
                at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
                at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)
                at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
                at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
                at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1316)
                at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1291)
    

    This means that Manta Data Lineage was not able to reach the target machine at all. There is not much to test. Please contact the network/infrastructure team to open the firewall/proxy.

  2. Connection reset

    java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:127)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112)
    at java.io.InputStreamReader.read(InputStreamReader.java:168)
    

    It was possible to open the connection, but it was closed by a resource other than Manta Data Lineage.

Both of the cases above are good candidates for testing the connection using the openssl command line tool to show that the issue is independent of Manta Data Lineage and provide a clear way for the network/infrastructure team to test independently.

Solution

  1. Verify that openssl is installed by running

    openssl version
    

    If it is not installed, ask your admin to install it.

    1. Linux: Usually a standard part of Linux distribution, the installation process depends on your Linux distribution

    2. Windows: The OpenSSL build can be downloaded, for example, from https://curl.se/windows/; only the OpenSSL component is needed

  2. In the command line, enter

    openssl s_client -connect <hostname>:<port>  -debug
    

    This opens an SSL connection to the specified hostname and port and prints the info about the SSL certificate. It will also provide us with other relevant output such as the certificate chain, the ciphers that are in use, and other characteristics of the SSL/TLS session. The connection details are:

    • <hostname> is the DNS name of the server you are connecting to

    • <port> is the port number on which the service you are connecting to is running

  3. Check the availability of the domain from the connection results.

    1. If the verification is successful and your machine can read and write from the server, you will see something like this in the output.

      > openssl s_client -connect google.com:443
      CONNECTED(000001BC)
      depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1
      verify error:num=20:unable to get local issuer certificate
      verify return:1
      depth=1 C = US, O = Google Trust Services LLC, CN = GTS CA 1C3
      verify return:1
      depth=0 CN = *.google.com
      verify return:1
      ---
      Certificate chain
       0 s:CN = *.google.com
         i:C = US, O = Google Trust Services LLC, CN = GTS CA 1C3
         a:PKEY: id-ecPublicKey, 256 (bit); sigalg: RSA-SHA256
         v:NotBefore: Sep 13 01:38:37 2021 GMT; NotAfter: Nov 20 01:38:36 2021 GMT
       1 s:C = US, O = Google Trust Services LLC, CN = GTS CA 1C3
         i:C = US, O = Google Trust Services LLC, CN = GTS Root R1
         a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
         v:NotBefore: Aug 13 00:00:42 2020 GMT; NotAfter: Sep 30 00:00:42 2027 GMT
       2 s:C = US, O = Google Trust Services LLC, CN = GTS Root R1
         i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
         a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
         v:NotBefore: Jun 19 00:00:42 2020 GMT; NotAfter: Jan 28 00:00:42 2028 GMT
      ---
      Server certificate
      -----BEGIN CERTIFICATE-----
      MIINxDCCDKygAwIBAgIQaczIXdXLFjQKAAAAAP9gXDANBgkqhkiG9w0BAQsFADBG
      ...
      iarLqQ7sR3xVM76Qgl+Be4A4z4fE7Dz1zRIbnetKn8YO5SO/B/H91EeIB4q+nmRO
      tYxFcW8qt9iyYP/rqp2q6lBmzkPLEpJH
      -----END CERTIFICATE-----
      subject=CN = *.google.com
      issuer=C = US, O = Google Trust Services LLC, CN = GTS CA 1C3
      ---
      No client certificate CA names sent
      Peer signing digest: SHA256
      Peer signature type: ECDSA
      Server Temp Key: X25519, 253 bits
      ---
      SSL handshake has read 6658 bytes and written 396 bytes
      Verification error: unable to get local issuer certificate
      ---
      New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
      Server public key is 256 bit
      Secure Renegotiation IS NOT supported
      Compression: NONE
      Expansion: NONE
      No ALPN negotiated
      Early data was not sent
      Verify return code: 20 (unable to get local issuer certificate)
      ---
      

      Notice the details about the bytes read and written as well as the details about the established connection, such as the TLS version and encryption used.

    2. In the OpenSSL output, if the connection prints 0 bytes read or written, then you need to revisit the firewall/proxy from your networking aspect which might be blocking communication from the machine Manta Data Lineage is installed on to the server we are trying to connect to. The output of the connection reset error may appear as follows.

      [ju@mantadev ~]$ openssl s_client -connect mantaschdev.company.com:16789 -debug
      CONNECTED(00000003)
      write to 0x556f24b73870 [0x556f24b8a700] (321 bytes => 321 (0x141))
      0000 - 16 03 01 01 3c 01 00 01-38 03 03 74 bb e5 b6 fa   ....<...8..t....
      0010 - b0 4a 97 2a af af 4b 54-7a 0f 65 c7 47 fc 7f 74   .J.*..KTz.e.G..t
      ...
      0120 - 20 47 90 b8 02 ca c2 20-07 de 36 9c eb 2b 06 5f    G..... ..6..+._
      0130 - fd 91 ac 7f 2e 3a d4 f1-f8 9c f8 0f a9 f8 b5 0b   .....:..........
      0140 - 18                                                .
      read from 0x556f24b73870 [0x556f24b814e3] (5 bytes => -1 (0xFFFFFFFFFFFFFFFF))
      write:errno=104
      ---
      no peer certificate available
      ---
      No client certificate CA names sent
      ---
      SSL handshake has read 0 bytes and written 321 bytes
      Verification: OK
      ---
      New, (NONE), Cipher is (NONE)
      Secure Renegotiation IS NOT supported
      Compression: NONE
      Expansion: NONE
      No ALPN negotiated
      Early data was not sent
      Verify return code: 0 (ok)
      ---
      read from 0x556f24b73870 [0x556f24aa8bb0] (8192 bytes => 0 (0x0))
      

      Discuss it with your network/infrastructure team and test what is blocking the communication. If the connection is not working even when using a standard tool like OpenSSL, it is clear that the problem is not Manta Data Lineage related and can be investigated and tested independently. Only when the communication via OpenSSL works does it make sense to retest in Manta Data Lineage.

  4. If the connection is successfully established, it is possible to test the communication between Manta Data Lineage and the target service.
    At this point, it makes sense to retest the connectivity in Manta Data Lineage. If you need to continue investigating the communication independently of Manta Data Lineage, it is possible to do that via openssl; however, there are other tools that are more convenient for testing the communication using actual content. For testing the communication with REST API endpoints, wget and curl are better options as they handle HTTP(S) protocol for you. For testing JDBC connections over TLS, there are plenty of open source utilities that can be used such as https://github.com/ptaucher/jdbc-tester.