IC SunsetThe developerWorks Connections Platform is now in read-only mode and content is only available for viewing. No new wiki pages, posts, or messages may be added. Please see our FAQ for more information. The developerWorks Connections platform will officially shut down on March 31, 2020 and content will no longer be available. More details available on our FAQ. (Read in Japanese.)
Topic
  • 6 replies
  • Latest Post - ‏2018-11-20T16:44:59Z by arnar75
arnar75
arnar75
4 Posts

Pinned topic Installation problems

‏2018-11-13T16:45:56Z |

Hi,

I'm trying to install and use the data collector service but it says it is unable to connect to the server.  I'm guessing that the "server" is the IBM storage insights server.

I have verified that the host can connect to the service via a web browser on the host.

What am I missing ?

Procedure used shown below:

[x@x storage-insights]$ sudo ./installDataCollectorService.sh
./installDataCollectorService.sh: line 428: [: too many arguments
./installDataCollectorService.sh: line 432: [: too many arguments


  Does your data collector require a proxy server? [yes/no] (default = yes): no
You made no configuration modification.

Verifying the connection to the storage management service...
The application failed to connect to the storage management service. Verify the entered data and retry the operation.
The connection verification failed.
The data collector was installed as a service: "dataCollector_xxxxxxxx". Base directory: "/home/art/Downloads/IBM/storage-insights".
Starting the service:...
./installDataCollectorService.sh: line 459: [: too many arguments
The data collector was started with PID: 3628
Trying to connect to the server.
The data collector failed to connect to the server. The message logged by the data collector is: "2018-11-13 16:42:15.270 BPCCA0006E The data collector failed to connect to the storage management service at https://Agent-xxxxxxxxxxxxxxx.ib.ibmserviceengage.com:443 because of an unknown error.". Check the connection information in "setup.properties" and "server.properties" under "conf" folder.

 

 

  • Tiberiu M
    Tiberiu M
    4 Posts
    ACCEPTED ANSWER

    Re: Installation problems

    ‏2018-11-20T15:27:10Z  
    • arnar75
    • ‏2018-11-20T11:59:12Z

    Hi Tiberiu,

    Thanks for the assistance, I think I have figured this out.

    After taking a better look at log/trace and log/message files I think that this is a SSL cert rewrite problem or a cert chaining error.  I've had similar problems with that before.

    Our CybSec people are inspecting SSL traffic and rewriting SSL certs because of that.  Guess I have to have a talk with them again.  I will update this thread when I have tried bypassing that SSL rewrite.

     

    from trace logs:

    2018-11-16 16:52:16.287 [Dispatch] [CollectorStatus.setConnectionStatus] [INFO] Failed to connect to the server
    2018-11-16 16:52:16.288 [Dispatch] [CollectorStatus.handleConnectionError] [ERROR] Connection failed from Dispatch.run
    [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2
    .util.h: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not buil
    d a valid CertPath.; internal cause is:
            java.security.cert.CertPathValidatorException: The certificate issued by CN=xxxxxxx, DC=xxxxxx, DC=lo
    cal is not trusted; internal cause is:
            java.security.cert.CertPathValidatorException: Certificate chaining error; targetException=java.lang.IllegalArg
    umentException: Error opening socket: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.h: PKIX path building fai
    led: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.;

     

    Hello Arnar,


    Thanks for the update !  Glad that you figured it out from the traces, that seems to be certainly the case if the trace logs indicate some SSL mangling.  Keep us posted, thanks ! 

  • Tiberiu M
    Tiberiu M
    4 Posts

    Re: Installation problems

    ‏2018-11-16T02:32:02Z  

    hello Arnar,

     

    Just out of curiosity, are you using RHEL 6 or 7 for the data collector ? 

     

    Given that you are getting 

    [x@x storage-insights]$ sudo ./installDataCollectorService.sh
    ./installDataCollectorService.sh: line 428: [: too many arguments

    that too many arguments error, I would suspect that might have some special characters in your directory path.  Please note that there are a series of special characters that are not supported : https://www.ibm.com/support/knowledgecenter/en/SSQRB8/com.ibm.spectrum.si.doc/tpch_saas_t_data_collector_install_aix_linux.html

    you also have to open up the firewall for outbound https 443 port communication. 

    Since you are installing using sudo, can you double check the sudoers configuration ? just in case anything is too restrictive. Also, if you can give it a try installing using root.

     

    Keep us posted, thanks ! 

  • arnar75
    arnar75
    4 Posts

    Re: Installation problems

    ‏2018-11-16T17:28:03Z  

    Hello,

    I decided to try the installation on  another host just to be sure. This time on a CentOS6.10 host as root and with iptables turned off.  Same outcome, but without the "too many arguments" warnings.

    output below:

    [root@management storage-insights]# ./installDataCollectorService.sh


      Does your data collector require a proxy server? [yes/no] (default = yes): no
    You made no configuration modification.

    Verifying the connection to the storage management service...
    The application failed to connect to the storage management service. Verify the entered data and retry the operation.
    The connection verification failed.
    The data collector was installed as a service: "dataCollector_5330b7134b1b3af5ac1d54785e474b4e". Base directory: "/root/storage-insights".
    Starting the service:...
    The data collector was started with PID: 11147
    Trying to connect to the server.
    The data collector failed to connect to the server. The message logged by the data collector is: "2018-11-16 16:58:07.499 BPCCA0006E The data collector failed to connect to the storage management service at https://Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com:443 because of an unknown error.". Check the connection information in "setup.properties" and "server.properties" under "conf" folder.
    [root@management storage-insights]#

     

     

    I also ran openssl on the command line to verify that the collector could reach the service.  Output below (after truncating keys and such):

    [root@management storage-insights]# openssl s_client -connect Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com:443 -servername Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com
    CONNECTED(00000003)
    depth=2 DC = local, DC = isavia, CN = isavia-RK-DC01-CA
    verify error:num=19:self signed certificate in certificate chain
    verify return:0
    ---
    Certificate chain
     0 s:/C=US/ST=New York/L=Armonk/O=International Business Machines Corporation/CN=*.ib.ibmserviceengage.com
       i:/CN=pa-decrypt07
     1 s:/CN=pa-decrypt07
       i:/DC=local/DC=isavia/CN=isavia-RK-DC01-CA
     2 s:/DC=local/DC=isavia/CN=isavia-RK-DC01-CA
       i:/DC=local/DC=isavia/CN=isavia-RK-DC01-CA
    ---
    Server certificate
    -----BEGIN CERTIFICATE-----
    ...............==
    -----END CERTIFICATE-----
    subject=/C=US/ST=New York/L=Armonk/O=International Business Machines Corporation/CN=*.ib.ibmserviceengage.com
    issuer=/CN=pa-decrypt07
    ---
    No client certificate CA names sent
    Server Temp Key: ECDH, prime256v1, 256 bits
    ---
    SSL handshake has read 3696 bytes and written 444 bytes
    ---
    New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
    Server public key is 2048 bit
    Secure Renegotiation IS NOT supported
    Compression: NONE
    Expansion: NONE
    SSL-Session:
        Protocol  : TLSv1.2
        Cipher    : ECDHE-RSA-AES128-GCM-SHA256
        Session-ID: AE6CE5C47B033AA028B991F533340205FA807867112D3D78202CAB035AD35E96
        Session-ID-ctx:
        Master-Key: 28ED384EDF2A7A83D1777D34CE8BE1338FE5C18CB952799993F2FBD128AF5194F078B2C858ECDB9060FAF7708E9A24C5
        Key-Arg   : None
        Krb5 Principal: None
        PSK identity: None
        PSK identity hint: None
        Start Time: 1542388593
        Timeout   : 300 (sec)
        Verify return code: 19 (self signed certificate in certificate chain)
    ---
    GET
    HTTP/1.1 400 Bad Request
    Date: Fri, 16 Nov 2018 17:16:35 GMT
    Vary: Accept-Encoding
    Content-Length: 304
    Connection: close
    Content-Type: text/html; charset=iso-8859-1

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>400 Bad Request</title>
    </head><body>
    <h1>Bad Request</h1>
    <p>Your browser sent a request that this server could not understand.<br />
    </p>
    <hr>
    <address>IBM_HTTP_Server at *.ib.ibmserviceengage.com Port 443</address>
    </body></html>
    read:errno=0

     

    So right now I'm not sure what the problem is.... is it the service on IBMs end or a SSL error ??

  • Tiberiu M
    Tiberiu M
    4 Posts

    Re: Installation problems

    ‏2018-11-20T03:28:34Z  
    • arnar75
    • ‏2018-11-16T17:28:03Z

    Hello,

    I decided to try the installation on  another host just to be sure. This time on a CentOS6.10 host as root and with iptables turned off.  Same outcome, but without the "too many arguments" warnings.

    output below:

    [root@management storage-insights]# ./installDataCollectorService.sh


      Does your data collector require a proxy server? [yes/no] (default = yes): no
    You made no configuration modification.

    Verifying the connection to the storage management service...
    The application failed to connect to the storage management service. Verify the entered data and retry the operation.
    The connection verification failed.
    The data collector was installed as a service: "dataCollector_5330b7134b1b3af5ac1d54785e474b4e". Base directory: "/root/storage-insights".
    Starting the service:...
    The data collector was started with PID: 11147
    Trying to connect to the server.
    The data collector failed to connect to the server. The message logged by the data collector is: "2018-11-16 16:58:07.499 BPCCA0006E The data collector failed to connect to the storage management service at https://Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com:443 because of an unknown error.". Check the connection information in "setup.properties" and "server.properties" under "conf" folder.
    [root@management storage-insights]#

     

     

    I also ran openssl on the command line to verify that the collector could reach the service.  Output below (after truncating keys and such):

    [root@management storage-insights]# openssl s_client -connect Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com:443 -servername Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com
    CONNECTED(00000003)
    depth=2 DC = local, DC = isavia, CN = isavia-RK-DC01-CA
    verify error:num=19:self signed certificate in certificate chain
    verify return:0
    ---
    Certificate chain
     0 s:/C=US/ST=New York/L=Armonk/O=International Business Machines Corporation/CN=*.ib.ibmserviceengage.com
       i:/CN=pa-decrypt07
     1 s:/CN=pa-decrypt07
       i:/DC=local/DC=isavia/CN=isavia-RK-DC01-CA
     2 s:/DC=local/DC=isavia/CN=isavia-RK-DC01-CA
       i:/DC=local/DC=isavia/CN=isavia-RK-DC01-CA
    ---
    Server certificate
    -----BEGIN CERTIFICATE-----
    ...............==
    -----END CERTIFICATE-----
    subject=/C=US/ST=New York/L=Armonk/O=International Business Machines Corporation/CN=*.ib.ibmserviceengage.com
    issuer=/CN=pa-decrypt07
    ---
    No client certificate CA names sent
    Server Temp Key: ECDH, prime256v1, 256 bits
    ---
    SSL handshake has read 3696 bytes and written 444 bytes
    ---
    New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
    Server public key is 2048 bit
    Secure Renegotiation IS NOT supported
    Compression: NONE
    Expansion: NONE
    SSL-Session:
        Protocol  : TLSv1.2
        Cipher    : ECDHE-RSA-AES128-GCM-SHA256
        Session-ID: AE6CE5C47B033AA028B991F533340205FA807867112D3D78202CAB035AD35E96
        Session-ID-ctx:
        Master-Key: 28ED384EDF2A7A83D1777D34CE8BE1338FE5C18CB952799993F2FBD128AF5194F078B2C858ECDB9060FAF7708E9A24C5
        Key-Arg   : None
        Krb5 Principal: None
        PSK identity: None
        PSK identity hint: None
        Start Time: 1542388593
        Timeout   : 300 (sec)
        Verify return code: 19 (self signed certificate in certificate chain)
    ---
    GET
    HTTP/1.1 400 Bad Request
    Date: Fri, 16 Nov 2018 17:16:35 GMT
    Vary: Accept-Encoding
    Content-Length: 304
    Connection: close
    Content-Type: text/html; charset=iso-8859-1

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>400 Bad Request</title>
    </head><body>
    <h1>Bad Request</h1>
    <p>Your browser sent a request that this server could not understand.<br />
    </p>
    <hr>
    <address>IBM_HTTP_Server at *.ib.ibmserviceengage.com Port 443</address>
    </body></html>
    read:errno=0

     

    So right now I'm not sure what the problem is.... is it the service on IBMs end or a SSL error ??

    Hello Arnar,

    Even though CentOS is similar to Redhat (minus enterprise level support) unfortunately it is not supported. "The supported Linux operating systems are Linux RedHat Enterprise Linux 6 and later versions."   since there are some differences which sometimes are a deal breaker. Please note that windows/aix is also support for the data collector if you have a serer handy.

     

    If I check with a web browser your service address : 

    it seems to be fine, since I have on my host port 443 open 

     

    telnet Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com 443
    Trying 169.54.46.40...
    Connected to Agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com.
    Escape character is '^]'.

    ^CConnection closed by foreign host.

     

    but if i try to openssl, I get the same http result  "400 Bad Request"  which I suspect it's fine since openssl is not sending any http get request, hence the return is http 400 - bad request. The certificate handling part seems fine from your excerpt.

    Try instead wget or curl 

     

     wget https://agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com/
    --2018-11-19 20:18:33--  https://agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com/
    Resolving agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com... 169.54.46.40
    Connecting to agent-4cee94d0010f40fcbaa7c89c4d73016d.ib.ibmserviceengage.com|169.54.46.40|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 3537 (3.5K) [text/html]
    Saving to: "index.html"

    100%[========================================================================================================================>] 3,537       --.-K/s   in 0s

    2018-11-19 20:18:33 (195 MB/s) - "index.html" saved [3537/3537]

     

    and I've got (in index.html) the same content as I've posted in the screen shot with "You have successfully established a secure connection to Storage Insights" .

     

    From what I can tell, the service on the IBM end is fine, it must be either your platform (CentOs or the redhat you had in your first post) or the firewall/network that is blocking access to outside on 443.  Again, this is my guess and I'm just trying to be helpful :)

     

    As always, you can open a support case : 

    https://www.ibm.com/mysupport/ and click on My Cases to open a support case.  You should be able to select Storage Insights or Storage Insights Pro when prompted for a product.

     

  • arnar75
    arnar75
    4 Posts

    Re: Installation problems

    ‏2018-11-20T11:59:12Z  

    Hi Tiberiu,

    Thanks for the assistance, I think I have figured this out.

    After taking a better look at log/trace and log/message files I think that this is a SSL cert rewrite problem or a cert chaining error.  I've had similar problems with that before.

    Our CybSec people are inspecting SSL traffic and rewriting SSL certs because of that.  Guess I have to have a talk with them again.  I will update this thread when I have tried bypassing that SSL rewrite.

     

    from trace logs:

    2018-11-16 16:52:16.287 [Dispatch] [CollectorStatus.setConnectionStatus] [INFO] Failed to connect to the server
    2018-11-16 16:52:16.288 [Dispatch] [CollectorStatus.handleConnectionError] [ERROR] Connection failed from Dispatch.run
    [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2
    .util.h: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not buil
    d a valid CertPath.; internal cause is:
            java.security.cert.CertPathValidatorException: The certificate issued by CN=xxxxxxx, DC=xxxxxx, DC=lo
    cal is not trusted; internal cause is:
            java.security.cert.CertPathValidatorException: Certificate chaining error; targetException=java.lang.IllegalArg
    umentException: Error opening socket: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.h: PKIX path building fai
    led: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.;

     

  • Tiberiu M
    Tiberiu M
    4 Posts

    Re: Installation problems

    ‏2018-11-20T15:27:10Z  
    • arnar75
    • ‏2018-11-20T11:59:12Z

    Hi Tiberiu,

    Thanks for the assistance, I think I have figured this out.

    After taking a better look at log/trace and log/message files I think that this is a SSL cert rewrite problem or a cert chaining error.  I've had similar problems with that before.

    Our CybSec people are inspecting SSL traffic and rewriting SSL certs because of that.  Guess I have to have a talk with them again.  I will update this thread when I have tried bypassing that SSL rewrite.

     

    from trace logs:

    2018-11-16 16:52:16.287 [Dispatch] [CollectorStatus.setConnectionStatus] [INFO] Failed to connect to the server
    2018-11-16 16:52:16.288 [Dispatch] [CollectorStatus.handleConnectionError] [ERROR] Connection failed from Dispatch.run
    [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2
    .util.h: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not buil
    d a valid CertPath.; internal cause is:
            java.security.cert.CertPathValidatorException: The certificate issued by CN=xxxxxxx, DC=xxxxxx, DC=lo
    cal is not trusted; internal cause is:
            java.security.cert.CertPathValidatorException: Certificate chaining error; targetException=java.lang.IllegalArg
    umentException: Error opening socket: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.h: PKIX path building fai
    led: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.;

     

    Hello Arnar,


    Thanks for the update !  Glad that you figured it out from the traces, that seems to be certainly the case if the trace logs indicate some SSL mangling.  Keep us posted, thanks ! 

  • arnar75
    arnar75
    4 Posts

    Re: Installation problems

    ‏2018-11-20T16:44:59Z  
    • Tiberiu M
    • ‏2018-11-20T15:27:10Z

    Hello Arnar,


    Thanks for the update !  Glad that you figured it out from the traces, that seems to be certainly the case if the trace logs indicate some SSL mangling.  Keep us posted, thanks ! 

    Hi Tiberiu,

    The problem was the SSL cert rewrite.

    After bypassing that the data collector works fine and on CentOS 6.10 ;)

     

    Thanks for the help.