`dp:url-open`

Uses a specified protocol to send data to or receive data from an arbitrary URL.

Namespace declaration

xmlns:dp="http://www.datapower.com/extensions"
extension-element-prefixes="dp"

Syntax

<dp:url-open
  target="URL"
  response="xml | binaryNode | ignore | responsecode | responsecode-binary | responsecode-ignore"
  resolve-mode="xml | swa"
  base-uri-node="nodeset"
  data-type="xml | base64 | filename"
  http-headers="nodeset"
  http-version="0 | 1 | 2"
  http2-required="true | false"
  content-type="contentType"
  ssl-proxy="client:profile"
  timeout="seconds"
  http-method="get | patch | post | put | delete | head"
  options="options">
</dp:url-open>

Attributes

target="URL"

The xs:string that identifies the target URL of the message destination or the message source. The dp:url-open extension supports the following protocols.

AMQP, see url-open, AMQP for information.
FTP, see url-open, FTP for information.
HTTP and HTTPS
ICAP, see url-open, ICAP for information.
IBM® IMS Connect, see url-open, IMS Connect for information.
IBM MQ, see url-open, IBM MQ for information.
IBM MQ File Transfer Edition (FTE), see url-open, IBM MQ FTE for information.
IBM WebSphere® eXtreme Scale, see url-open, eXtreme Scale for information.
IBM WebSphere JMS, see url-open, WebSphere JMS for information.
Kafka, see url-open, Kafka for information.
NFS, see url-open, NFS for information.
SFTP, see url-open, SFTP for information.
SMTP, see url-open, SMTP for information.
SNMP, see url-open, SNMP for information.
SQL, see url-open, SQL for information.
TCP, see url-open, TCP for information.
Length-prefixed TCP, see url-open, length-prefixed TCP for information.
TIBCO EMS (Enterprise Message Services), see url-open, TIBCO EMS for information.

The optional xs:string that specifies how the DataPower® Gateway handles the response, if any, from the target URL. The default value is xml.

xml

Indicates that the DataPower Gateway parses the response as XML.

binaryNode (deprecated)

Deprecated - Indicates that the DataPower Gateway treats any received response from the target URL as non-parsed binary data. This keyword is deprecated. Use responsecode-binary.

ignore

Indicates that the DataPower Gateway ignores any response that is received from the target URL.

responsecode

responsecode-ignore

Enable testing of received protocol status codes.

responsecode

For testing HTTP transmissions.

All successful HTTP transactions return the following element, when you set the response attribute to responsecode:
```
<url-open>
  <responsecode>200</responsecode>
</url-open>
```
All failed HTTP transactions return the following element, when you set the response attribute to responsecode:
```
<url-open>
  <responsecode>500</responsecode>
</url-open>
```
All successful IBM MQ transactions return the following element, when you set the response attribute to responsecode:
```
<url-open>
  <responsecode>0</responsecode>
</url-open>
```
All failed IBM MQ transactions typically return one of the following elements when you set the response attribute to responsecode.

2009

Cannot access queue

2059

Cannot establish connection
```
<url-open>
  <responsecode>2059</responsecode>
</url-open>
```
All successful FTP transactions return the following element, when you set the response attribute to responsecode.
```
<url-open/>
```
All failed FTP transactions return the following element, when you set the response attribute to responsecode.
```
<url-open>
  <response> ... returned data ... </response>
</url-open>
```

responsecode-ignore

For testing PUT operations.

All successful HTTP transactions return the following element, when you set the response attribute to responsecode-ignore.
```
<url-open>
  <responsecode>200</responsecode>
</url-open>
```
All failed HTTP transactions return the following element, when you set the response attribute to responsecode-ignore.
```
<url-open>
  <responsecode>500</responsecode>
</url-open>
```
All successful IBM MQ transactions return the following element, when you set the response attribute to responsecode-ignore.
```
<url-open>
  <responsecode>0</responsecode>
</url-open>
```
All failed IBM MQ transactions typically return one of the following elements, when you set the response attribute to responsecode-ignore.

2009

Cannot access queue

2059

Cannot establish connection
```
<url-open>
  <responsecode>2059</responsecode>
</url-open>
```
All successful FTP transactions return the following element, when you set the response attribute to responsecode-ignore.
```
<url-open/>
```
All failed FTP transactions return the following element, when you set the response attribute to responsecode-ignore.
```
<url-open>
  <statuscode>n</statuscode>
</url-open>
```

responsecode-binary

The responsecode-binary keyword specifies that the DataPower Gateway inserts the payload into the <binary> element regardless of the response code value. The <binary> element makes the payload available as base-64-encoded information for error and message handling.

HTTP transactions return the following element for a 200 response code.

<result>
  <binary>***BINARY NODE***</binary>
  <responsecode>200</responsecode>
  <reasonphrase>OK</reasonphrase>
  <httpversion>...</httpversion>
  <headers> ... </headers>
</result>

HTTP transactions return the following element for a 400 response code.

<result>
  <binary>***BINARY NODE***</binary>
  <responsecode>400</responsecode>
  <reasonphrase>Bad Request</reasonphrase>
  <httpversion>...</httpversion>
  <headers> ... </headers>
</result>

HTTP transactions that encounter no server or server down conditions return the following element.

<result>
  <errorstatuscode>5</errorstatuscode>
  <errorstring>Remote error on URL</errorstring>
</result>

resolve-mode="xml | swa"

The optional xs:string that is meaningful when the target URL points to an attached document. The default value is xml.

swa: Indicates that the target URL is a SOAP with Attachments document. When set to swa, the extension element attempts to resolve the URL as an attachment. For example, the element returns any attachment whose Content-Location header matches the target URL. If the extension element does not locate a SOAP with Attachments document or cannot find a URL-to-Content-Location match, the extension element stops processing. When processing stops, the extension element ignores all other specified attributes.
xml: Indicates a non-SWA operational environment.

base-uri-node="nodeset"

The node set that is similar to the base URI argument for the document() function. If the target URL is relative, the extension element uses the first node in the specified node set to determine the base URI. The base URI is determined according to the XML base mechanism. If you do not specify a node set, <url-open/> uses the current node in the input document to determine the base URI.

data-type="xml | base64 | filename"

The xs:string that describes the input data to the extension element. The default value is xml.

xml

Indicates that the input data is XML.

base64

Indicates that the input data is base-64 encoded and the extension element decodes the data before it resumes further processing.

filename

Indicates that a local file contains the input data. Both the content and the extension element identify the target file. The following example uses local:///myFile as the input data:

<dp:url-open target="URL" data-type="filename">local:///my
File</dp:url-open>

http-headers="nodeset"

The nodeset that specifies HTTP headers to append to this connection. The node set that is passed to the extension element can contain one or more <header> elements with a name attribute that specifies the name of the header. The content of the header element specifies the value of the header.

The following example defines the SOAPAction header with the ClientRequest value.

<xsl:variable name="headerValues">
  <header name="SOAPAction">ClientRequest</header>
</xsl:variable>

http-version="0 | 1 | 2"

The optional xs:integer that sets the HTTP version. The default value is 1.

0: Indicates HTTP/1.0.
1: Indicates HTTP/1.1.
2: Indicates HTTP/2.

http2-required="true | false"

When HTTP/2, the optional xs:boolean that controls whether the connection requires HTTP/2. The default value is false, which means that HTTP/2 is not required. To require HTTP/2, set the value to true.

For HTTPS, HTTP/2 is enforceable. The server connection is secured with a TLS client profile.
For HTTP, HTTP/2 is not enforceable. The server connection is insecure. The server might not honor the upgrade request and process the original request.

content-type="contentType"

The optional xs:string that specifies the value to use for the HTTP Content-Type header.

ssl-proxy="client:profile"

The optional xs:string that identifies the name of an existing TLS profile that this extension element uses to establish a secured connection to the target URL. If not specified, the extension element uses the current mappings in the user agent to establish the secured connection.

For an IBM MQ and SQL connection, the extension element does not support this attribute.

timeout="seconds"

The optional xs:string that specifies the time for the DataPower Gateway to establish a connection to the target server or for an established connection with the target server to be idle. Enter a value of range 1 - 86400. The value of -1 uses the default value that is set by the user agent. The default value in the user agent is 300.

Note: By default, persistent connections are supported with an idle timeout set to 15 seconds.

For HTTP GET and HEAD requests, the service attempts the connection again after the specified value. Therefore, the actual timeout is twice the specified value.

The optional xs:string that specifies the HTTP request method for this request.

get: Uses the HTTP GET method. If the request contains content, this attribute is ignored. The extension element uses the HTTP POST method.
patch: Uses the HTTP PATCH method.
post: Uses the HTTP POST method.
put: Uses the HTTP PUT method.
delete: Uses the HTTP DELETE method.
head: Uses the HTTP HEAD method.

options="options"

The optional xs:string that specifies more connection options. The following option is supported.

donot-follow-redirect: By default, the DataPower Gateway attempt to resolve HTTP redirects. Specify this option when you want to prevent the resolution of HTTP redirects.

Guidelines

The DataPower Gateway serializes and sends all specified values in the dp:url-open element to the target URL with UTF-8 as the encoding.

If you specify xml for the response attribute, the DataPower Gateway parses the response from the target URL and inserts the content in the output for the dp:url-open element.
If you specify responsecode-binary for the response attribute, the DataPower Gateway inserts the payload into the <binary></binary> element regardless of the response code value.
If you specify ignore for the response attribute, the DataPower Gateway discards any response from the target URL and does not produce output.

The semantics for send and receive depend on the type of URL. For HTTP, send corresponds to a client POST request and receive corresponds to the server response.

If you do not specify content in the dp:url-open element, the DataPower Gateway does an HTTP GET request instead of an HTTP POST request. This behavior can be changed by specifying a value for the http-method attribute. The http-method attribute can be used with the HTTP or HTTPS protocols only.

Some protocols might not support the retrieval of data.

If you specify a BasicAuth header with the http-headers attribute, it has a higher priority than a BasicAuthPolicy configured with the user agent.

The dp:url-open element passes the target, data-type, content-type, and ssl-proxy attributes (all of type xs:string) as AVTs. It also passes the response and resolve-mode attributes (both of type xs:string) as literals and the base-uri-node and http-headers attributes (both node sets) as XPath expressions.

Received data format.

If the response attribute characterizes the response data from the target URL as xml or omitted, the element outputs the returned data as parsed XML.

If the response attribute characterizes the response data from the target URL as responsecode, the element outputs the returned data in the following format.

<url-open>
  <statuscode>0</statuscode>
  <responsecode>0</responsecode>
  <errorcode>0</errorcode>
  <errorstring>error</errorstring>
  <contenttype>text/xml</contenttype>
  <headers>
    <header name="contenttype">text/xml</header>
      ...
    <header name="foo">bar</header>
  </headers>
  <response> ... returned data ... </response>
</url-open>

If the response attribute characterizes the response data from the target URL as responsecode-ignore (meaning to ignore any response data), the element outputs the returned data in the following format.

<url-open>
  <statuscode>0</statuscode>
  <responsecode>0</responsecode>
  <errorcode>0</errorcode>
  <errorstring>error</errorstring>
  <contenttype>text/xml</contenttype>
  <headers>
    <header name="contenttype">text/xml</header>
      ...
    <header name="foo">bar</header>
  </headers>
</url-open>

If the response attribute characterizes the response data from the target URL as binaryNode or ignore, the element fetches binary input without an XML parse. The element outputs the returned data in the following format.
```
<result>
  <statuscode>0</statuscode>
  <binary> ... binary data ... </binary>
</result>
```

The following table lists the values and their meaning for the statuscode element in the response.

Table 1. Mapping of `statuscode` value to meaning.
Code	Meaning
`0`	Success.
`1`	For internal use.
`2`	Unsupported function.
`3`	Resource not found. Can attempt connection with a different opener.
`4`	Generic error.
`5`	Connection error.
`6`	Error during DNS lookup.
`7`	Error with TLS connection.
`8`	Cannot read HTTP headers.
`9`	Triggered loop-detection.
`10`	Cannot access the document in the document cache.
`11`	Exceeded the allowable number of redirect requests.
`12`	Redirect does not have a `Location` header
`13`	Redirect has bad body length.
`14`	Cannot create the connection for the redirect.
`15`	Unhandled HTTP response code, which is any response code that does not indicate success or redirect.
`16`	Protocol framing error. (SMTP, FTP)
`17`	Protocol response error. (SMTP, FTP)
`18`	Protocol state error. (SMTP, FTP)
`19`	Connection attempt to port timed out.
`20`	Cannot connect on port. Another attempt is acceptable.
`21`	Redirected to an unsupported URI schema.
`22`	Invalid local address.
`23`	Target not found.

Examples

Send a serialized copy of the content of $some-nodes to http://www.datapower.com/application.jsp. The element parses the response and saves the output in the $jsp-response variable.
```
<xsl:variable name="jsp-response">
  <dp:url-open target="http://www.datapower.com/application.jsp">
    <xsl:copy-of select="$some-nodes">
  </dp:url-open>
</xsl:variable>
```
Do not send any content to the target URL. Similar to a GET and PUT action, the element parses the foo.xml response and inserts the file into the output. This example is similar to a document() call, but in this example the element inserts the response directly into the output.
```
<dp:url-open target="http://www.datapower.com/foo.xml"/>
```
Specify a relative URL, so the element uses the base URI of the /doc node in the input document. The base URI defines a location from which the element can open files. For example, if the URI were http://www.yahoo.com, the element fetches http://www.yahoo.com/foo.xml
```
<dp:url-open target="foo.xml" base-uri-node="/doc"/>
```
Post some data to the draw-a-gif.jsp application. The application returns a gif file, which the element ignores and does not attempt to parse as XML.
```
<dp:url-open target="http://www.datapower.com/draw-a-gif.jsp"
    response="ignore">
  ...
</dp:url-open>
```
Append the HTTP headers that are defined by the httpHeaders variable to the connection. The node set that is passed to the extension element contains two <header> elements with name attributes that specify the name of the HTTP header. The content of the header elements specifies the values of the HTTP headers.
```
<xsl:variable name="httpHeaders">
  <header name="SOAPAction">doit</header>
  <header name="Stuff">andjunk</header>
</xsl:variable>

<dp:url-open target="http://127.0.0.1:9070/foo" response="xml"
    http-headers="$httpHeaders">
  <xsl:copy-of select="$call"/>
</dp:url-open>
```

Use an HTTP PUT request to create a book on the target service.

<xsl:variable name="url" select="concat
    ('"title="http://127.0.0.1:13450/library/','book099.xml')"/>"
     class="link">http://127.0.0.1:13450/library/','book099.xml')"/>

<xsl:variable name="book">
  <xsl:element name="author">Jones, Tim</xsl:element>
  <xsl:element name="title">Developers Guide</xsl:element>
  <xsl:element name="genre">Computer</xsl:element>
  <xsl:element name="price">14.95</xsl:element>
  <xsl:element name="publish_date">2008-10-16</xsl:element>
  <xsl:element name="description">Creating applications</xsl:element>
</xsl:variable>

<xsl:template match="/">
  <dp:url-open target="{$url}" response="responsecode-ignore"
      http-method="put">
    <xsl:copy-of select="$book"/>
  </dp:url-open>
  <xsl:copy-of select="."/>
</xsl:template>

Send material to a virus-scan service. The element scans the service response, which is written to the $attachment variable, for a 403 response code. If the element detects a 403 response code, indicating the presence of a virus, it rejects the message. Receipt of a 200 or 204 response code results in message acceptance.

Note the use of AVT syntax with the target attribute.

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:dp="http://www.datapower.com/extensions"
    xmlns:dpconfig="http://www.datapower.com/param/config"
    extension-element-prefixes="dp"
    exclude-result-prefixes="dp dpconfig"
    version="1.0">

  <xsl:output method="xml"/>
      <xsl:variable name="location" select="content-location=pic"/>
        
      <!-- how to specify binary attachments? -->
      <xsl:variable name="binarydata">
        <dp:url-open target="{concat($location,'?Encode=base64')}"/>
      </xsl:variable>

      <xsl:variable name="httpHeaders">
        <header name="Host">127.0.0.1</header>
        <header name="Allow">204</header>
      </xsl:variable>

    <!-- Reject by default -->
      <dp:reject>Could not scan</dp:reject>
      <xsl:variable name="attachment">
        <dp:url-open target="icap://x.xx.xx.xx:nn/AVSCAN?action=SCAN"
            response="responsecode" data-type="base64"
            http-headers="$httpHeaders">
          <xsl:value-of select="$binarydata/base64"/>
        </dp:url-open>
      </xsl:variable>

      <xsl:if test="$attachment/url-open/responsecode = '403'">
      <dp:reject override="true">Virus Found</dp:reject>
      </xsl:if>
      <xsl:if test="$attachment/url-open/responsecode = '200'">
        <dp:accept/>
      </xsl:if>
      <xsl:if test="$attachment/url-open/responsecode = '204'">
        <dp:accept/>
      </xsl:if>

Send a mail message that contains log information to a recipient on an SMTP mail server.

<xsl:variable name="log-message">
  <MyError>
    <Date>
      <xsl:value-of select="date:date()" />
    </Date>
    <Time>
      <xsl:value-of select="date:time()" />
    </Time>
    <Domain>
      <xsl:value-of select="dp:variable('var://service/domain-name')" />
    </Domain>
    <TransactionID>
      <xsl:value-of select="dp:variable('var://service/transaction-id')" />
    </TransactionID>
    <ErrorCode>
      <xsl:value-of select="dp:variable('var://service/error-code')" />
    </ErrorCode>
    <ErrorMessage>
     <xsl:value-of select="dp:variable('var://service/error-message')" />
    </ErrorMessage>
    <Payload>
      <!-- show request sent -->
      <xsl:copy-of select="/*" />
    </Payload>
  </MyError>
</xsl:variable>

<!-- send log to remote location  -->
<dp:url-open target="dpsmtp://9.56.228.182/?To=joedoe@us.ibm.com
 &From=drecluse@us.ibm.com&Subject=DP-Error" response="responsecode">
   <xsl:copy-of select="$log-message"/>
</dp:url-open>

Send an SQL request to an SQL data source. Cambridge_DB2 is the name of the SQL data source object.

<xsl:variable name="selectStmt">
  SELECT name FROM table where id=6
</xsl:variable>
<dp:url-open target="sql//:Cambridge_DB2/static?$selectStmt"/>
</xsl:variable>

Send a message to an EMS request queue. The stylesheet reflects the names and values of the EMS headers in the http-headers attribute.

<xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:dp="http://www.datapower.com/extensions"
    xmlns:dpconfig="http://www.datapower.com/param/config"
    extension-element-prefixes="dp"
    exclude-result-prefixes="dp dpconfig">

<xsl:output method="xml"/>
<xsl:template match="/">
  <xsl:copy-of select="."/>

	<xsl:variable name="headers">
    <header name="DP_EMSMessageType">text</header>
    <header name="UBER">VERY</header>
    <header name="UnTO">NUNCA</header>
  </xsl:variable>

  <dp:url-open target="dptibems://tibems/?RequestQueue=Vecchio" http-headers="$headers">
    TESTING TESTING2
  </dp:url-open>
</xsl:template>

Retrieve a message from an EMS reply queue.

...
<xsl:variable name="msgContents">
  <dp:url-open target="dptibems://tibems/?ReplyQueue="Bernini">
  </dp:url-open>
</xsl:variable>
...

Send a message to a JMS request queue. The stylesheet reflects the names and values of the JMS headers in the http-headers attribute.

<xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:dp="http://www.datapower.com/extensions"
    xmlns:dpconfig="http://www.datapower.com/param/config"
    extension-element-prefixes="dp"
    exclude-result-prefixes="dp dpconfig">

<xsl:output method="xml"/>

<xsl:template match="/">
  <xsl:copy-of select="."/>
	<xsl:variable name="headers">
    <header name="DP_JMSMessageType">text</header>
    <header name="UBER">VERY</header>
    <header name="UnTO">NUNCA</header>
  </xsl:variable>
  <dp:url-open target="dpwasjms://wasjms/?RequestQueue=Vecchio"
     http-headers="$headers">
    TESTING TESTING2
  </dp:url-open>
</xsl:template>

Retrieve a message from a JMS reply queue.

...
<xsl:variable name="msgContents">
  <dp:url-open target="dpwasjms://wasjms/?ReplyQueue="Bernini">
  </dp:url-open>
</xsl:variable>
...