IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • 14 replies
  • Latest Post - ‏2014-01-16T13:11:15Z by pablosanchez84
Rohit-Goyal
Rohit-Goyal
128 Posts

Pinned topic HTTPS Connection using dp:url-open

‏2013-05-17T06:41:01Z |

Hi,

I am trying to make HTTPS connection with backend using url-open. Problem is backend is using a Self-Signed certificate. We want to disable backend cert validation while establish connection. In other words, we don't want to create a specific SSL profile for this connection to work.

I tried same with DP utility (send-message) that available when you enable probe. There when I put HTTPS url and send the message, it worked. I didn't mention any SSL profile there.

Can someone tell what can be done to make a HTTPS connection to backend (using self-signed certificate) without creating SSL profile?

 

Thanks

RG

  • HermannSW
    HermannSW
    4514 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-17T10:18:28Z  

    > I tried same with DP utility (send-message)
    >

    If you do "View Page Source" you will see that the call is done via JavaScript.

     

    Not sure what your problem is with providing a "Forward (Client) Crypto Profile", that allow to access any self signed SSL backend service.

    First curl sends data to "beast" SSL backend (identity service).

    Next coproc2 executes stylesheet "url-open.xsl" with same input on "firestar" box.
    That stylesheet goes against "beast" backend with url-open.
    "Forward" crypto profile is like curl's "-k" option:

    $ curl -k --data-binary @ab.xml https://beast:2051 ; echo
    <?xml version="1.0" encoding="UTF-8"?>
    <a>1<b>2</b>3</a>
    $
    $ coproc2 url-open.xsl ab.xml http://firestar:2223 ; echo
    <a>1<b>2</b>3</a>
    $
    $ cat url-open.xsl
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <xsl:output omit-xml-declaration="yes" />
        
      <xsl:variable name="httpHeaders" select="/.."/>


      <xsl:template match="/">
        <xsl:variable name="result">
          <dp:url-open
             target="https://beast.boeblingen.de.ibm.com:2051"
             http-headers="$httpHeaders/*"
             ssl-proxy="firestar"
          >
            <xsl:copy-of select="."/>
          </dp:url-open>
        </xsl:variable>

        <xsl:copy-of select="$result"/>
      </xsl:template>

    </xsl:stylesheet>
    $

     

    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

  • Rohit-Goyal
    Rohit-Goyal
    128 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-18T02:34:04Z  
    • HermannSW
    • ‏2013-05-17T10:18:28Z

    > I tried same with DP utility (send-message)
    >

    If you do "View Page Source" you will see that the call is done via JavaScript.

     

    Not sure what your problem is with providing a "Forward (Client) Crypto Profile", that allow to access any self signed SSL backend service.

    First curl sends data to "beast" SSL backend (identity service).

    Next coproc2 executes stylesheet "url-open.xsl" with same input on "firestar" box.
    That stylesheet goes against "beast" backend with url-open.
    "Forward" crypto profile is like curl's "-k" option:

    $ curl -k --data-binary @ab.xml https://beast:2051 ; echo
    <?xml version="1.0" encoding="UTF-8"?>
    <a>1<b>2</b>3</a>
    $
    $ coproc2 url-open.xsl ab.xml http://firestar:2223 ; echo
    <a>1<b>2</b>3</a>
    $
    $ cat url-open.xsl
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <xsl:output omit-xml-declaration="yes" />
        
      <xsl:variable name="httpHeaders" select="/.."/>


      <xsl:template match="/">
        <xsl:variable name="result">
          <dp:url-open
             target="https://beast.boeblingen.de.ibm.com:2051"
             http-headers="$httpHeaders/*"
             ssl-proxy="firestar"
          >
            <xsl:copy-of select="."/>
          </dp:url-open>
        </xsl:variable>

        <xsl:copy-of select="$result"/>
      </xsl:template>

    </xsl:stylesheet>
    $

     

    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    Thanks Herman.

    So conclusion is we can't ignore Server Cert validation while making a connection with HTTPS backend?

    Rohit

  • HermannSW
    HermannSW
    4514 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-18T09:34:10Z  

    Thanks Herman.

    So conclusion is we can't ignore Server Cert validation while making a connection with HTTPS backend?

    Rohit

    Hi,

    I am not a security guy, but to my knowledge there is no server cert validation happening with a Forward Crypto Profile.

    The server validates that the client cert you present with Forward Crypto Profile is fine.

    It is the same as using curl's -k option:

    $ curl -h | grep insecure
     -k/--insecure      Allow connections to SSL sites without certs (H)
    $

    Server cert validation happens to my knowledge only with mutual authentication.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

  • swlinn
    swlinn
    1346 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-19T21:08:08Z  
    • HermannSW
    • ‏2013-05-18T09:34:10Z

    Hi,

    I am not a security guy, but to my knowledge there is no server cert validation happening with a Forward Crypto Profile.

    The server validates that the client cert you present with Forward Crypto Profile is fine.

    It is the same as using curl's -k option:

    $ curl -h | grep insecure
     -k/--insecure      Allow connections to SSL sites without certs (H)
    $

    Server cert validation happens to my knowledge only with mutual authentication.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    By default, your backside (forward) Crypto Cert will validate the server certificate, so you need to have a validation credential with either the certificate or immediate issuer (which in your case you don't have since the cert is self signed).  There is an option on both the SSL Proxy Profile and Crypto Profile about allowing connections to an insecure SSL Server which I don't know will apply in your case, but by selecting it you've now made a unique SSL Proxy / Crypto Profile, so why not specify the validation credential and do this in a secure manner?

    Regards,
    Steve

  • Rohit-Goyal
    Rohit-Goyal
    128 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-20T13:35:40Z  
    • swlinn
    • ‏2013-05-19T21:08:08Z

    By default, your backside (forward) Crypto Cert will validate the server certificate, so you need to have a validation credential with either the certificate or immediate issuer (which in your case you don't have since the cert is self signed).  There is an option on both the SSL Proxy Profile and Crypto Profile about allowing connections to an insecure SSL Server which I don't know will apply in your case, but by selecting it you've now made a unique SSL Proxy / Crypto Profile, so why not specify the validation credential and do this in a secure manner?

    Regards,
    Steve

    Thanks Steve, Herman

    I ended up creating a forward profile with val cred having the certificate. It worked fine.

    I was interested to understand if DataPower, like most of HTTP clients (SOAPUI, Curl), provide option to create a insecure connection to a HTTPS backend.

    Rohit

  • kenhygh
    kenhygh
    1475 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-20T13:40:14Z  

    Thanks Steve, Herman

    I ended up creating a forward profile with val cred having the certificate. It worked fine.

    I was interested to understand if DataPower, like most of HTTP clients (SOAPUI, Curl), provide option to create a insecure connection to a HTTPS backend.

    Rohit

    Rohit,

    I *think* I've done this in the past by not putting a cert in the valcred.

  • Rohit-Goyal
    Rohit-Goyal
    128 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-21T10:56:40Z  
    • kenhygh
    • ‏2013-05-20T13:40:14Z

    Rohit,

    I *think* I've done this in the past by not putting a cert in the valcred.

    Ken, I tried that. It didn't work. ValCred need Cert to establish connection. 

    Is there any special setting that I need to do?

    Rohit

  • kenhygh
    kenhygh
    1475 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2013-05-21T14:56:22Z  

    Ken, I tried that. It didn't work. ValCred need Cert to establish connection. 

    Is there any special setting that I need to do?

    Rohit

    oops, it's leave off the ValCred, not the cert, according to some doc I have access to.

    So try it with an SSL proxy profile without a ValCred.

     

  • pablosanchez84
    pablosanchez84
    31 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2014-01-15T16:05:27Z  
    • kenhygh
    • ‏2013-05-21T14:56:22Z

    oops, it's leave off the ValCred, not the cert, according to some doc I have access to.

    So try it with an SSL proxy profile without a ValCred.

     

    Ken, I found this by mistake while configuring a new SSL Profile with a WSRR instance.

    Tried to look everywhere but couldn't find any assertion like "A SSL Pro. Prof. in "forward" mode with empty Ident/ValCred will act as insecure client". Good to know you have this documented internally. At least I know it's not a bug (I almost opened a PMR).

    Thanks, 

  • pablosanchez84
    pablosanchez84
    31 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2014-01-15T16:20:46Z  

    Ken, I found this by mistake while configuring a new SSL Profile with a WSRR instance.

    Tried to look everywhere but couldn't find any assertion like "A SSL Pro. Prof. in "forward" mode with empty Ident/ValCred will act as insecure client". Good to know you have this documented internally. At least I know it's not a bug (I almost opened a PMR).

    Thanks, 

    Well, I think I take that back... The DP SOA Handbook says:

    When DataPower acts as an SSL client, a validation credential is usually needed (if no validation credential is present we will validate using the certificates in the pubcert:directory) and an identification credential is required if mutual authentication is in use. The SSL direction would be set to Forward,and a "forward"Crypto Profile would be defined.

    In my case, the WSRR instance is using a Self-Signed cert. Just wondering which cert from pubcert is being used to establish handshake. Any ideas?

  • Rohit-Goyal
    Rohit-Goyal
    128 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2014-01-16T02:52:39Z  

    Well, I think I take that back... The DP SOA Handbook says:

    When DataPower acts as an SSL client, a validation credential is usually needed (if no validation credential is present we will validate using the certificates in the pubcert:directory) and an identification credential is required if mutual authentication is in use. The SSL direction would be set to Forward,and a "forward"Crypto Profile would be defined.

    In my case, the WSRR instance is using a Self-Signed cert. Just wondering which cert from pubcert is being used to establish handshake. Any ideas?

    Let me give some updates to it. 

    Usecase was to try calling XML Interface of DataPower using dp:url-open. Initially when we were using DataPower self signed cert, I was required to add cert into forward crypto profile to make it work.

    Later we changed all DataPower WebUI and XML Interface cert with a good certificate verified by my company's own CA. I tried to remove Id and Val Cred from Crypto profile and it worked. My understanding is, in case when Id and Val cred are not available, DataPower tries to verify the cert using available certs in Cert:/// and PubCert:// folder. 

    - Rohit

  • pablosanchez84
    pablosanchez84
    31 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2014-01-16T10:31:16Z  

    Let me give some updates to it. 

    Usecase was to try calling XML Interface of DataPower using dp:url-open. Initially when we were using DataPower self signed cert, I was required to add cert into forward crypto profile to make it work.

    Later we changed all DataPower WebUI and XML Interface cert with a good certificate verified by my company's own CA. I tried to remove Id and Val Cred from Crypto profile and it worked. My understanding is, in case when Id and Val cred are not available, DataPower tries to verify the cert using available certs in Cert:/// and PubCert:// folder. 

    - Rohit

    Thanks Rohit. That's make complete sense.

    But in my case, how can a self-signed cert. (WSRR) be validated by an empty Forward Crypto Profile?
    I'm just trying to understand whether or not DP is establishing an insecure connection.

  • UlrikAndersen
    UlrikAndersen
    184 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2014-01-16T12:33:21Z  

    Thanks Rohit. That's make complete sense.

    But in my case, how can a self-signed cert. (WSRR) be validated by an empty Forward Crypto Profile?
    I'm just trying to understand whether or not DP is establishing an insecure connection.

    If by "empty" you mean "no validation credential", the device will accept ANY certificate from the server side. This is not best practice.

  • pablosanchez84
    pablosanchez84
    31 Posts

    Re: HTTPS Connection using dp:url-open

    ‏2014-01-16T13:11:15Z  

    If by "empty" you mean "no validation credential", the device will accept ANY certificate from the server side. This is not best practice.

    Thanks Ulrik. I did a few packet capture and the data is indeed encrypted.

    Yes, I understand about the risks and will not keep with it this way, was just trying to understand behavior.

    Thanks again.