Topic
  • 12 replies
  • Latest Post - ‏2012-11-06T20:09:47Z by triniman10
dpdeveloper
dpdeveloper
111 Posts

Pinned topic DP stripping MIME boundary headers

‏2011-07-13T22:02:27Z |
Hello,

One of our backend services responds with MIMEBoundary's. We have a MPGW, where response type is set to non-xml. In this case datapower is stripping of the MIMEBoundary headers and only processing the actual pay load.

If we set the response type to passthrough, then the response comes as unaltered.

Response from the service:
--MIMEBoundary_1111111111111111
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <0.11111111111111111@apache.org>

<?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><test>valid</test></soapenv:Body></soapenv:Envelope>
MIMEBoundary_011111111111111111

Response from DP
<?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><test>valid</test></soapenv:Body></soapenv:Envelope>.

Is this is a bug? and is there any workaround for this other than setting the response to passthrough.
Updated on 2012-11-06T20:09:47Z at 2012-11-06T20:09:47Z by triniman10
  • dpdeveloper
    dpdeveloper
    111 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-13T22:17:44Z  
    Eventhough this property is on, still its stripping. If I enable the probe, the mime headers are not even coming into the response rule.

    **Back Side MIME Header Processing**

    The body of a message (the payload independent of protocol headers) can sometimes contain MIME headers before any preamble and before the first MIME boundary in the body of the message. These MIME headers can contain important information that is not available in the protocol headers, such as a string that identifies the MIME boundary. If this property is set to on, these MIME headers are processed by the service.

    When set to on and there are no MIME headers in the message, the device continues to try and parse the message, using the protocol header information, if available.

    When set to off and MIME headers are present in the body of the message, these MIME headers are considered part of the preamble and not used to parse the message. If protocol headers, such as HTTP, indicate MIME boundaries, the device can parse and process individual attachments. If such information is not available, no attachments can be parsed from the body of the message.
  • msiebler
    msiebler
    140 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-14T18:46:18Z  
    Eventhough this property is on, still its stripping. If I enable the probe, the mime headers are not even coming into the response rule.

    **Back Side MIME Header Processing**

    The body of a message (the payload independent of protocol headers) can sometimes contain MIME headers before any preamble and before the first MIME boundary in the body of the message. These MIME headers can contain important information that is not available in the protocol headers, such as a string that identifies the MIME boundary. If this property is set to on, these MIME headers are processed by the service.

    When set to on and there are no MIME headers in the message, the device continues to try and parse the message, using the protocol header information, if available.

    When set to off and MIME headers are present in the body of the message, these MIME headers are considered part of the preamble and not used to parse the message. If protocol headers, such as HTTP, indicate MIME boundaries, the device can parse and process individual attachments. If such information is not available, no attachments can be parsed from the body of the message.
    Does the MIME message have only 1 part ? In that case the device will assume that the mime boundaries are not needed ; since they are not separating anything.
    You could force the device to add them back in by adding some kind of small dummy attachment to the root part.
  • dpdeveloper
    dpdeveloper
    111 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-14T22:09:58Z  
    • msiebler
    • ‏2011-07-14T18:46:18Z
    Does the MIME message have only 1 part ? In that case the device will assume that the mime boundaries are not needed ; since they are not separating anything.
    You could force the device to add them back in by adding some kind of small dummy attachment to the root part.
    It has both start and end boundary, so I guess it has both parts. How can enforce dp to add them back. do I have to use any binary transformation.
  • HermannSW
    HermannSW
    4876 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-15T08:14:26Z  
    It has both start and end boundary, so I guess it has both parts. How can enforce dp to add them back. do I have to use any binary transformation.
    DataPower can process attachments, for SOAP and XML.
    You said that you have Non-XML response type which may be the problem.

    The message you pasted is nearly unreadable.
    Please attach an unmodified backend message (not gone through an Editor, please).

    Even if processing of your mime message may not be possible natively I am sure we can find a solution.
    The reason for this is that DataPower does not allow to handle binary file-upload HTTP posts.
    I provided swaform tool to handle that, see Enhancing “near” SOAP With Attachment data slide:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019118&aid=1#page=16

     
    Hermann<myXsltBlog/>
  • dpdeveloper
    dpdeveloper
    111 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-18T20:24:58Z  
    • HermannSW
    • ‏2011-07-15T08:14:26Z
    DataPower can process attachments, for SOAP and XML.
    You said that you have Non-XML response type which may be the problem.

    The message you pasted is nearly unreadable.
    Please attach an unmodified backend message (not gone through an Editor, please).

    Even if processing of your mime message may not be possible natively I am sure we can find a solution.
    The reason for this is that DataPower does not allow to handle binary file-upload HTTP posts.
    I provided swaform tool to handle that, see Enhancing “near” SOAP With Attachment data slide:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019118&aid=1#page=16

     
    Hermann<myXsltBlog/>
    The response is set to non-xml because through same the same gateway we can expect some non-xml responses also.

    the exact response if we set to passthrough ( removed some info)

    --MIMEBoundary_06266d59021d86252889f35e9f1e2b9339b51707a6a2c479
    Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
    Content-Transfer-Encoding: binary
    Content-ID: <0.16266d59021d86252889f35e9f1e2b9339b51707a6a2c479@apache.org>

    <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header><msg:Context xmlns:msg="http://xxxx/entity/message/2003/" soapenv:actor="http://schemas.xmlsoap.org/soap/actor/next" soapenv:mustUnderstand="0"><msg:messageId>4488cd7c-b179-11e0-8944-8053e89b4ff6</msg:messageId><msg:refToMessageId>1</msg:refToMessageId><msg:sessionSequenceNumber>1</msg:sessionSequenceNumber><msg:creationTimestamp>2011-07-18T13:05:23.121Z</msg:creationTimestamp></msg:Context><</soapenv:Header><soapenv:Body><Response xmlns="http://xxxxxx/2010/"><FaultList xmlns="http://xxxxxxx/entity/message/2003/"><Fault><faultCode><value xmlns="">1005</value></faultCode><faultDetail><technicalText xmlns="">xam/xset inaccessible</technicalText></faultDetail></Fault></FaultList></Response></soapenv:Body></soapenv:Envelope>
    MIMEBoundary_06266d59021d86252889f35e9f1e2b9339b51707a6a2c479
    Response if set to non-xml.

    <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header><msg:Context xmlns:msg="http://xxxx/entity/message/2003/" soapenv:actor="http://schemas.xmlsoap.org/soap/actor/next" soapenv:mustUnderstand="0"><msg:messageId>4488cd7c-b179-11e0-8944-8053e89b4ff6</msg:messageId><msg:refToMessageId>1</msg:refToMessageId><msg:sessionSequenceNumber>1</msg:sessionSequenceNumber><msg:creationTimestamp>2011-07-18T13:05:23.121Z</msg:creationTimestamp></msg:Context><</soapenv:Header><soapenv:Body><Response xmlns="http://xxxxxx/2010/"><FaultList xmlns="http://xxxxxxx/entity/message/2003/"><Fault><faultCode><value xmlns="">1005</value></faultCode><faultDetail><technicalText xmlns="">xam/xset inaccessible</technicalText></faultDetail></Fault></FaultList></Response></soapenv:Body></soapenv:Envelope>

    and becuase of DP stripping the mime headers, some client are not able to digest the response.

    Is there any workaround to replace this headers back?
  • HermannSW
    HermannSW
    4876 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-19T05:39:52Z  
    The response is set to non-xml because through same the same gateway we can expect some non-xml responses also.

    the exact response if we set to passthrough ( removed some info)

    --MIMEBoundary_06266d59021d86252889f35e9f1e2b9339b51707a6a2c479
    Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
    Content-Transfer-Encoding: binary
    Content-ID: <0.16266d59021d86252889f35e9f1e2b9339b51707a6a2c479@apache.org>

    <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header><msg:Context xmlns:msg="http://xxxx/entity/message/2003/" soapenv:actor="http://schemas.xmlsoap.org/soap/actor/next" soapenv:mustUnderstand="0"><msg:messageId>4488cd7c-b179-11e0-8944-8053e89b4ff6</msg:messageId><msg:refToMessageId>1</msg:refToMessageId><msg:sessionSequenceNumber>1</msg:sessionSequenceNumber><msg:creationTimestamp>2011-07-18T13:05:23.121Z</msg:creationTimestamp></msg:Context><</soapenv:Header><soapenv:Body><Response xmlns="http://xxxxxx/2010/"><FaultList xmlns="http://xxxxxxx/entity/message/2003/"><Fault><faultCode><value xmlns="">1005</value></faultCode><faultDetail><technicalText xmlns="">xam/xset inaccessible</technicalText></faultDetail></Fault></FaultList></Response></soapenv:Body></soapenv:Envelope>
    MIMEBoundary_06266d59021d86252889f35e9f1e2b9339b51707a6a2c479
    Response if set to non-xml.

    <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header><msg:Context xmlns:msg="http://xxxx/entity/message/2003/" soapenv:actor="http://schemas.xmlsoap.org/soap/actor/next" soapenv:mustUnderstand="0"><msg:messageId>4488cd7c-b179-11e0-8944-8053e89b4ff6</msg:messageId><msg:refToMessageId>1</msg:refToMessageId><msg:sessionSequenceNumber>1</msg:sessionSequenceNumber><msg:creationTimestamp>2011-07-18T13:05:23.121Z</msg:creationTimestamp></msg:Context><</soapenv:Header><soapenv:Body><Response xmlns="http://xxxxxx/2010/"><FaultList xmlns="http://xxxxxxx/entity/message/2003/"><Fault><faultCode><value xmlns="">1005</value></faultCode><faultDetail><technicalText xmlns="">xam/xset inaccessible</technicalText></faultDetail></Fault></FaultList></Response></soapenv:Body></soapenv:Envelope>

    and becuase of DP stripping the mime headers, some client are not able to digest the response.

    Is there any workaround to replace this headers back?
    Hi,

    your files are not useful because the forum software scrambles them.
    Please attach them, do not copy in.

     
    Hermann<myXsltBlog/>
  • dpdeveloper
    dpdeveloper
    111 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-20T15:57:58Z  
    • HermannSW
    • ‏2011-07-19T05:39:52Z
    Hi,

    your files are not useful because the forum software scrambles them.
    Please attach them, do not copy in.

     
    Hermann<myXsltBlog/>
    Hermann,

    Please find the attached response which is the actual response when set to passthrough.
  • HermannSW
    HermannSW
    4876 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-20T20:17:19Z  
    Hermann,

    Please find the attached response which is the actual response when set to passthrough.
    Thanks for providing the file.

    Now we can see that you have a MIME message consisting of only one part.
    In this case DataPower works as designed in removing the boundaries (compliant processors should be able to deal with both).
    A change to this bevavior would need an Enhancement Request (ER), please contact your sales rep.
    You already found a workaround in having response type as pass-thru.

    A second workaround would be to add a dummy attachment resulting in more than one part, then everything works fine. But your client needs to be able to deal with this.

    A third workaround would be real Non-XML processing preserving the MIME boundaries.
    But 2nd and 3rd workaround are only needed if you do have a need in processing the backend response.
    If there is no need then path-thru is most simple and best solution.

     
    Hermann<myXsltBlog/>
  • dpdeveloper
    dpdeveloper
    111 Posts

    Re: DP stripping MIME boundary headers

    ‏2011-07-21T17:33:13Z  
    • HermannSW
    • ‏2011-07-20T20:17:19Z
    Thanks for providing the file.

    Now we can see that you have a MIME message consisting of only one part.
    In this case DataPower works as designed in removing the boundaries (compliant processors should be able to deal with both).
    A change to this bevavior would need an Enhancement Request (ER), please contact your sales rep.
    You already found a workaround in having response type as pass-thru.

    A second workaround would be to add a dummy attachment resulting in more than one part, then everything works fine. But your client needs to be able to deal with this.

    A third workaround would be real Non-XML processing preserving the MIME boundaries.
    But 2nd and 3rd workaround are only needed if you do have a need in processing the backend response.
    If there is no need then path-thru is most simple and best solution.

     
    Hermann<myXsltBlog/>
    Thanks Hermann. We are in the process of filing an ER.
  • triniman10
    triniman10
    29 Posts

    Re: DP stripping MIME boundary headers

    ‏2012-11-01T21:21:35Z  
    • HermannSW
    • ‏2011-07-20T20:17:19Z
    Thanks for providing the file.

    Now we can see that you have a MIME message consisting of only one part.
    In this case DataPower works as designed in removing the boundaries (compliant processors should be able to deal with both).
    A change to this bevavior would need an Enhancement Request (ER), please contact your sales rep.
    You already found a workaround in having response type as pass-thru.

    A second workaround would be to add a dummy attachment resulting in more than one part, then everything works fine. But your client needs to be able to deal with this.

    A third workaround would be real Non-XML processing preserving the MIME boundaries.
    But 2nd and 3rd workaround are only needed if you do have a need in processing the backend response.
    If there is no need then path-thru is most simple and best solution.

     
    Hermann<myXsltBlog/>
    Hi Hermann,
    We are experiencing the same issue however due to the processing we perform thru stylesheets in the processing rule pass thru will not be an option for us. Is there a reason why the default behavior for processing a MIME message with one part to strip the MIME boundary?

    v/r
    Burt
  • HermannSW
    HermannSW
    4876 Posts

    Re: DP stripping MIME boundary headers

    ‏2012-11-06T18:12:01Z  
    Hi Hermann,
    We are experiencing the same issue however due to the processing we perform thru stylesheets in the processing rule pass thru will not be an option for us. Is there a reason why the default behavior for processing a MIME message with one part to strip the MIME boundary?

    v/r
    Burt
    > Is there a reason why the default behavior for processing a MIME message with one part to strip the MIME boundary?
    >
    I thinks this is the problem -- if a MIME message has just one part, and you process it (XML), then it is not a MIME message but just normal XML.

    An XSLT processor can produce XML/SOAP, HTML and TEXT as output.
    So the input and output do not need to have anything in common, not event the content-type.

    DataPower does not strip the MIME boundary, it just does not generate a MIME boundary if the result (XML/SOAP, HTML, TEXT) does not have attachments.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • triniman10
    triniman10
    29 Posts

    Re: DP stripping MIME boundary headers

    ‏2012-11-06T20:09:47Z  
    • HermannSW
    • ‏2012-11-06T18:12:01Z
    > Is there a reason why the default behavior for processing a MIME message with one part to strip the MIME boundary?
    >
    I thinks this is the problem -- if a MIME message has just one part, and you process it (XML), then it is not a MIME message but just normal XML.

    An XSLT processor can produce XML/SOAP, HTML and TEXT as output.
    So the input and output do not need to have anything in common, not event the content-type.

    DataPower does not strip the MIME boundary, it just does not generate a MIME boundary if the result (XML/SOAP, HTML, TEXT) does not have attachments.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Thanks Hermann as that does make sense for that behavior.