Topic
  • 3 replies
  • Latest Post - ‏2012-01-17T10:15:56Z by NicolasB
SystemAdmin
SystemAdmin
2262 Posts

Pinned topic SSL handshake problems

‏2004-05-06T11:47:51Z |
Hi,

I am facing a problem similar to the one described here
http://www-106.ibm.com/developerworks/forums/dw_thread.jsp?forum=178&thread=33005&cat=10

I have client SSL code that works perfectly fine with the Sun JDK, but does not with the IBM JDK.

I am running IBM Java2-141 on a RedHat Linux box.

The SSL trace when running on the IBM JDK is:
code
provider: statically registered providers
provider: [0] com.ibm.jsse.IBMJSSEProvider
provider: [1] com.ibm.crypto.provider.IBMJCE
provider: [2] com.ibm.security.jgss.IBMJGSSProvider
provider: [3] com.ibm.security.cert.IBMCertPath
provider:
provider: loading provider http://com.ibm.jsse.IBMJSSEProvider into slot 0 ...
provider: slot 0 loaded with http://IBMJSSE version 1.41
provider: loading provider http://com.ibm.crypto.provider.IBMJCE into slot 1 ...
algorithm: request for CertificateFactory X.509 from null
algorithm: request for X.509 can be met by BootstrapProvider version 1.1
algorithm: X.509 created, class com.ibm.security.cert.CertificateFactoryImpl
algorithm: request for AlgorithmParameters 1.2.840.10040.4.1 from null
algorithm: request for 1.2.840.10040.4.1 can be met by BootstrapProvider version 1.1
algorithm: 1.2.840.10040.4.1 created, class com.ibm.security.bootstrap.DSAParameters
algorithm: request for AlgorithmParameters 1.2.840.10040.4.1 from null
algorithm: request for 1.2.840.10040.4.1 can be met by BootstrapProvider version 1.1
algorithm: 1.2.840.10040.4.1 created, class com.ibm.security.bootstrap.DSAParameters
algorithm: request for KeyFactory DSA from null
algorithm: request for DSA can be met by BootstrapProvider version 1.1
algorithm: DSA created, class com.ibm.security.bootstrap.DSAKeyFactory
algorithm: request for AlgorithmParameters 1.2.840.10040.4.1 from null
algorithm: request for 1.2.840.10040.4.1 can be met by BootstrapProvider version 1.1
algorithm: 1.2.840.10040.4.1 created, class com.ibm.security.bootstrap.DSAParameters
algorithm: request for AlgorithmParameters 1.2.840.10040.4.1 from null
algorithm: request for 1.2.840.10040.4.1 can be met by BootstrapProvider version 1.1
algorithm: 1.2.840.10040.4.1 created, class com.ibm.security.bootstrap.DSAParameters
algorithm: request for Signature SHA1withDSA from null
algorithm: request for SHA1withDSA can be met by BootstrapProvider version 1.1
algorithm: SHA1withDSA created, class com.ibm.security.bootstrap.DSASignature
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by BootstrapProvider version 1.1
algorithm: SHA1 created, class com.ibm.security.bootstrap.SHA
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by BootstrapProvider version 1.1
algorithm: SHA1 created, class com.ibm.security.bootstrap.SHA
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by BootstrapProvider version 1.1
algorithm: SHA1 created, class com.ibm.security.bootstrap.SHA
provider: slot 1 loaded with http://IBMJCE version 1.2
provider: loading provider http://com.ibm.security.jgss.IBMJGSSProvider into slot 2 ...
provider: slot 2 loaded with http://IBMJGSSProvider version 1.01
provider: loading provider http://com.ibm.security.cert.IBMCertPath into slot 3 ...
provider: slot 3 loaded with http://IBMCertPath version 1.0
algorithm: request for KeyStore JKS from null
algorithm: request for JKS can be met by IBMJCE version 1.2
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.security.bootstrap.SHA
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: JKS created, class com.ibm.crypto.provider.JavaKeyStore
algorithm: request for MessageDigest SHA from IBMJCE
algorithm: request for SHA can be met by IBMJCE version 1.2
algorithm: SHA created, class com.ibm.crypto.provider.SHA
algorithm: request for CertificateFactory X.509 from IBMJCE
algorithm: request for X.509 can be met by IBMJCE version 1.2
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: X.509 created, class com.ibm.crypto.provider.X509Factory
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: request for MessageDigest SHA from IBMJCE
algorithm: request for SHA can be met by IBMJCE version 1.2
algorithm: SHA created, class com.ibm.crypto.provider.SHA
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
algorithm: request for MessageDigest SHA1 from null
algorithm: request for SHA1 can be met by IBMJCE version 1.2
algorithm: SHA1 created, class com.ibm.crypto.provider.SHA
algorithm: request for KeyStore JKS from null
algorithm: request for JKS can be met by IBMJCE version 1.2
algorithm: JKS created, class com.ibm.crypto.provider.JavaKeyStore
algorithm: request for MessageDigest SHA from IBMJCE
algorithm: request for SHA can be met by IBMJCE version 1.2
algorithm: SHA created, class com.ibm.crypto.provider.SHA
algorithm: request for CertificateFactory X.509 from IBMJCE
algorithm: request for X.509 can be met by IBMJCE version 1.2
algorithm: X.509 created, class com.ibm.crypto.provider.X509Factory
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
JSSEContext: handleSession[Sockethttp://addr=/209.173.53.166,port=3121,localport=54234]
JSSEContext: confirmPeerCertificate[Sockethttp://addr=/209.173.53.166,port=3121,localport=54234]
algorithm: request for CertificateFactory X.509 from null
algorithm: request for X.509 can be met by IBMJCE version 1.2
algorithm: X.509 created, class com.ibm.crypto.provider.X509Factory
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
algorithm: request for KeyFactory RSA from null
algorithm: request for RSA can be met by IBMJCE version 1.2
algorithm: RSA created, class com.ibm.crypto.provider.RSAKeyFactory
X509TrustManagerImpl: checkServerTrusted
X509TrustManagerImpl: Certificate [
[
Version: V3
Subject: CN=XRP Server, OU=OTE, O=NEUSTAR, L=Sterling, ST=VA, C=US
Signature Algorithm: MD5withRSA, OID = 1.2.840.113549.1.1.4

Key: IBMJCE RSA Public Key:
modulus:
152621760974799600198574054100716896155008498856372487894482622577039630810298424175413175160287470837076229648044018037294419258010988166360664166069360572723733636971350586488161957122419095248375186664932263370324459277364077251796011490767184283194174531625113400639652518419163235080739957898252514365239
public exponent:
65537

Validity: [From: Thu Sep 18 16:55:50 IST 2003,
To: Fri Sep 17 16:55:50 IST 2004]
Issuer: CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 335

Certificate Extensions: 3
[1]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: f1 8c e6 cc e8 72 1a cb 34 df 78 b1 c4 7d f6 d1 .....r..4.x.....
0010: 25 e4 51 07 ..Q.
]

CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 0]

[2]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:false
PathLen: undefined
]

[3]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 01 06 ee 79 a4 83 3b b3 39 e0 5b db b4 ae 8e a5 ...y....9.......
0010: 3f 87 17 92 ....
]
]

]
Algorithm: MD5withRSA
Signature:
0000: 2c a1 c6 2b 2f fb 04 88 09 04 d5 a7 f0 c9 58 ee ..............X.
0010: 90 e4 72 9a aa fa 72 f6 ce 7f 82 14 0d 86 70 3c ..r...r.......p.
0020: 56 12 a0 81 b5 42 35 11 46 69 1e fb 1c 74 e4 c1 V....B5.Fi...t..
0030: 8c b1 25 86 d8 c2 ff 8c cb 69 d4 92 55 eb 77 fa .........i..U.w.
0040: ad d4 c4 4e c6 8f 4d 38 c5 ce ce 9f 87 ad 6b 5d ...N..M8......k.
0050: 82 fe f3 62 5c 8f 9a 8c 0a 0b 5c b4 2b 2a 6a 0d ...b..........j.
0060: fd 64 2a 2d d7 54 af 09 ea 1a e5 a8 83 b4 fe 0b .d...T..........
0070: 84 f1 00 29 e0 4f fd 76 cc 91 5e 14 65 d6 72 f2 .....O.v....e.r.

]
X509TrustManagerImpl: Certificate [
[
Version: V3
Subject: CN=OTE CA, OU=OTE, O=NeuStar, C=US
Signature Algorithm: MD5withRSA, OID = 1.2.840.113549.1.1.4

Key: IBMJCE RSA Public Key:
modulus:
150454590274280829684589305471851239347796901142772575896472336226133940327733435785705787399962825178765469833222766939246533167513490454595946546362321591543499586194728538884742983013218875526862337989102679480163480805671768902800835773362538056163576462434341090592943372233647261715508233614166585079813
public exponent:
65537

Validity: [From: Sat Sep 15 00:50:10 IST 2001,
To: Tue Sep 14 00:50:10 IST 2004]
Issuer: CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: [0]

Certificate Extensions: 4
[1]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: f1 8c e6 cc e8 72 1a cb 34 df 78 b1 c4 7d f6 d1 .....r..4.x.....
0010: 25 e4 51 07 ..Q.
]

CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 0]

[2]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:true
PathLen:2147483647
]

[3]: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
Key_CertSign
Crl_Sign
]

[4]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: f1 8c e6 cc e8 72 1a cb 34 df 78 b1 c4 7d f6 d1 .....r..4.x.....
0010: 25 e4 51 07 ..Q.
]
]

]
Algorithm: MD5withRSA
Signature:
0000: 12 58 ce 6c e7 be 55 7e 13 29 93 d3 9a 55 1c d1 .X.l..U......U..
0010: 46 84 08 59 6f 6b 11 6f 02 cc 8c 9f f1 c0 4c 51 F..Yok.o......LQ
0020: cd c5 11 26 ce c2 68 38 c6 dd 8e 14 1a c9 e1 2e ......h8........
0030: cf 22 9a 86 53 96 13 10 39 9e 4c 23 96 13 f8 19 ....S...9.L.....
0040: 72 11 05 a5 c2 d9 f4 9e 9a 73 87 e9 db b9 e4 3d r........s......
0050: a0 fb 1b 80 18 af 02 d3 f5 87 d5 af ee 2c dc d7 ................
0060: 30 37 46 c4 99 a1 d6 be 85 21 b6 9f 4f 0e 1a 0e 07F.........O...
0070: f4 0d e7 15 cc 6c af 8e a1 9c c4 45 5a 6d 43 f3 .....l.....EZmC.

]
Thu May 06 17:04:58 IST 2004 - NeulevelEppSession - Exception in connect - javax.net.ssl.SSLHandshakeException: handshake failure
javax.net.ssl.SSLHandshakeException: handshake failure
at com.ibm.jsse.bg.a(Unknown Source)
at com.ibm.jsse.a.a(Unknown Source)
at com.ibm.jsse.a.read(Unknown Source)
[/code]

The same code, when run with the Sun JDK gives the following SSL output:
code
***
found key for : ssl.keystore.sand.alias
chain [0] = [
[
Version: V3
Subject: CN=1000001000, OU=OTE, O=NeuLevel, C=US
Signature Algorithm: MD5withRSA, OID = 1.2.840.113549.1.1.4

Key: SunJSSE RSA public key:
public exponent:
010001
modulus:
c1ded480 ee31fcf1 2e31de23 129f88ac b2ada6d5 c84ae43d d691901b 34f6dbad
0497f70f 579d3cbe 6d6c06ea f8b72832 43024f63 631ccc6f 0135aa81 1acce8ac
a9d5edd7 4d31f6a9 b1f9a248 ea6dab14 953aad1b 6267357a d0820504 1fd62adb
1db53dc9 d5e73050 305717fe 1d13ce66 d9abfcf1 48cea5b9 13a7f4d1 e2fd3e69
Validity: [From: Tue Mar 16 13:01:01 GMT 2004,
To: Wed Mar 16 13:01:01 GMT 2005]
Issuer: CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 0181

Certificate Extensions: 3
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 5F 87 82 A9 FD D0 D3 E3 BE D6 B1 D3 FF 14 33 BD _.............3.
0010: 2A E5 A9 A2 *...
]
]

[2]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: F1 8C E6 CC E8 72 1A CB 34 DF 78 B1 C4 7D F6 D1 .....r..4.x.....
0010: 25 E4 51 07 %.Q.
]

CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 00]

[3]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:false
PathLen: undefined
]

]
Algorithm: MD5withRSA
Signature:
0000: 43 4D FF B2 51 42 4E 97 9E FF E7 D4 73 6E E4 4C CM..QBN.....sn.L
0010: 1F 71 80 6C F2 F9 7F 86 6E 00 BD C1 FD 12 D2 C2 .q.l....n.......
0020: C8 FD E5 25 EA 57 C9 62 AB A4 1A C9 38 54 75 D5 ...%.W.b....8Tu.
0030: 64 16 26 49 98 5A 62 81 F4 D6 40 4F AC E5 0F E8 d.&I.Zb...@O....
0040: 35 67 5C 9A 13 2B E2 94 F6 78 17 8A 82 1D 7B 94 5g\..+...x......
0050: 6F CB 8C 4A 76 D8 47 3A E5 99 0E A6 1E 4C 52 10 o..Jv.G:.....LR.
0060: 18 3D 40 98 15 B5 4B 9B 29 03 0B A5 5C C1 E9 B6 .=@...K.)...\...
0070: 86 26 F9 2F 0C 18 5A B0 12 1B 65 30 1A 68 29 48 .&./..Z...e0.h)H

]
chain [1] = [
[
Version: V3
Subject: CN=OTE CA, OU=OTE, O=NeuStar, C=US
Signature Algorithm: MD5withRSA, OID = 1.2.840.113549.1.1.4

Key: SunJSSE RSA public key:
public exponent:
010001
modulus:
d6412388 b2e09f3f c416e3e0 8762d1df 31159b29 8f9f1a2a 938f1360 80ed9155
49265c05 ac901d72 33f6adc1 cf560d60 9f38a5cb 026f63ec 37f7db64 4af609fe
7cd7fa4b 90078132 8147a7a7 0c0393cf 73378ae2 d19c85c5 fcc47637 50a0a875
2077c4ee 271d5d65 56fd1e4e b92d3cb8 9056c6c6 67c6228e 1322f3dd 761d7805
Validity: [From: Fri Sep 14 19:20:10 GMT 2001,
To: Mon Sep 13 19:20:10 GMT 2004]
Issuer: CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 00

Certificate Extensions: 4
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: F1 8C E6 CC E8 72 1A CB 34 DF 78 B1 C4 7D F6 D1 .....r..4.x.....
0010: 25 E4 51 07 %.Q.
]
]

[2]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: F1 8C E6 CC E8 72 1A CB 34 DF 78 B1 C4 7D F6 D1 .....r..4.x.....
0010: 25 E4 51 07 %.Q.
]

CN=OTE CA, OU=OTE, O=NeuStar, C=US
SerialNumber: 00]

[3]: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
Key_CertSign
Crl_Sign
]

[4]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:true
PathLen:2147483647
]

]
Algorithm: MD5withRSA
Signature:
0000: 12 58 CE 6C E7 BE 55 7E 13 29 93 D3 9A 55 1C D1 .X.l..U..)...U..
0010: 46 84 08 59 6F 6B 11 6F 02 CC 8C 9F F1 C0 4C 51 F..Yok.o......LQ
0020: CD C5 11 26 CE C2 68 38 C6 DD 8E 14 1A C9 E1 2E ...&..h8........
0030: CF 22 9A 86 53 96 13 10 39 9E 4C 23 96 13 F8 19 ."..S...9.L#....
0040: 72 11 05 A5 C2 D9 F4 9E 9A 73 87 E9 DB B9 E4 3D r........s.....=
0050: A0 FB 1B 80 18 AF 02 D3 F5 87 D5 AF EE 2C DC D7 .............,..
0060: 30 37 46 C4 99 A1 D6 BE 85 21 B6 9F 4F 0E 1A 0E 07F......!..O...
0070: F4 0D E7 15 CC 6C AF 8E A1 9C C4 45 5A 6D 43 F3 .....l.....EZmC.

]
***
adding as trusted cert:
Subject: CN=OTE CA, OU=OTE, O=NeuStar, C=US
Issuer: CN=OTE CA, OU=OTE, O=NeuStar, C=US
Algorithm: RSA; Serial number: 0x0
Valid from Fri Sep 14 19:20:10 GMT 2001 until Mon Sep 13 19:20:10 GMT 2004

adding as trusted cert:
Subject: CN=XRP Server, OU=OTE, O=NEUSTAR, L=Sterling, ST=VA, C=US
Issuer: CN=OTE CA, OU=OTE, O=NeuStar, C=US
Algorithm: RSA; Serial number: 0x14f
Valid from Thu Sep 18 11:25:50 GMT 2003 until Fri Sep 17 11:25:50 GMT 2004
[/code]
When running with the IBM JDK, for some reason it does not seem to be adding the trusted cert based on my trust store.

The algo for the factory has been changed from SunX509 to IbmX509.

Please let me know if there is something obvious that I am missing.
Spent a loooong time over this, but no go :(

Thanks!
Updated on 2012-01-17T10:15:56Z at 2012-01-17T10:15:56Z by NicolasB
  • SystemAdmin
    SystemAdmin
    2262 Posts

    Re: SSL handshake problems

    ‏2004-05-12T08:05:03Z  
    Nothing obvious springs to mind going from the trace. Are you using the same machine with both the passing and failing tests. Or to put it another way, could the problem be down to a network issue?

    I guess you used the system property "java.security.debug=all" to get the IBM trace. Did you try "javax.net.debug=true"?
  • NicolasB
    NicolasB
    4 Posts

    Re: SSL handshake problems

    ‏2012-01-06T10:53:02Z  
    Hello

    I am having the exact same problem (WAS 6 JRE versus Sun 1.4.2 JRE), meaning this is still not fixed.

    It does not happen with WAS 7 / WAS 8 JRE, but many of our production servers still run WAS 6...

    Did you found a solution since this post in 2004 ?
  • NicolasB
    NicolasB
    4 Posts

    Re: SSL handshake problems

    ‏2012-01-17T10:15:56Z  
    To whom may be interested : I solved this problem by using the IBMJSSE2 security provider, instead of IBMJSSE (which is the default).

    There was obviously a slight feature in the protocol implementation that was not correctly handled either by WAS or the SSL service provider...

    This article describes the differences between the different security providers in IBM's 1.4.2 JRE : http://www.ibm.com/developerworks/java/jdk/security/142/secguides/securityguide.win32.html#ibmjsse2
    while this one gives hint on how to use a specific one : http://www.ibm.com/developerworks/java/library/j-ibmsecurity/index.html#IBMJSSE2

    For your information, I did it at runtime to remove the need to update every server instance where the apps would be deployed :

    Provider ibmjsse2 = Security.getProvider("IBMJSSE2");
    if (ibmjsse2 != null) {
    log.info("IBMJSSE2 security provider has been detected : it will be used first");
    try {
    Security.removeProvider("IBMJSSE2");
    Security.insertProviderAt(ibmjsse2, 1);
    } catch (SecurityException se) {
    log.warning("Cannot move IBMJSSE2 to top priority", se);
    }
    }