HTTP Adapter commands

The following table lists valid commands for the HTTP Adapter, their syntax, and whether they are supported (✓) for use with data sources, targets, or both.

Name Syntax Source Target
Client Certificate -CERT {label}
From -FROM address
Header -HDR [+][I]
HTTP(S) -URL {HTTP|HTTPS}://[user[:pass]@]
[host[:port]][/dir[/file]]
Ignore -IGNORE error_code[:error_code...]
Inline Output -INLINE  
Listen -LSN seconds  
Method -METHOD method_name
Modified Since -MODIFIED http-date
Proxy URL -PROXY {HTTP|HTTPS}://[user[:pass]@]
[host[:port]][/dir[/file]]
Redirections Allowed -REDIRECT limit
Session -SESSION session_id [*]
Server Name Indication -SNI
SOCKS URL -SOCKS {socks4|socks5}://[user[:pass]@]
[host[:port]]
SSL Encryption Strength -STR {WEAK|STRONG|ANY}
SSL Protocol -SPROTO {SSLv2|SSLv3|SSLv23|TLSv1}
Status Code -STATUS_CODE {map_return_code=HTTP_status_code[,map_return_code=HTTP_status_code...] | map_return_code:map_return_code[:map_return_code...]=HTTP_status_code}  
Status Line -STATUSLINE  
Status Page -STATUS_PAGE file_name[*].file_type  
Status Page Default -STATUS_PAGE_DEFAULT file_name.file_type  
Timeout -TIMEOUT time_in_seconds
Trace -T[E][+][S|V] [full_path]
Tunneling -TUNNELING
Type -TYPE content_type [charset=encoding]

Client Certificate

Specifies the label of the client certificate in the key store. This command is applicable only when the SSL_CLIENT secure_mode setting in the config.yaml file is set, and SSL/TLS encryption is in effect.

The corresponding adapter command is -CERT {label}.

label: The name of the entry in the certificate store, which is specified by the key_store field in the config.yaml file.

Note: The certificate store contains all personal and trusted certificates in a PKCS12 format file. To see the examples on how to import private keys and certificate chains into this encrypted file, refer to the readme.txt file located in the <Design Studio installation folder>\examples\secureadapt\ssl folder.

From

Specifies the email address of a contact person. The adapter sends alerts to the contact person in case of a problem using requests.

The corresponding adapter command is -FROM.

-FROMaddress

address: Specifies the e-mail address of a contact person

Header

This command is used to send message headers from data in a map file or to return message headers to the map file. In addition to generating new headers, you can append user-supplied headers, eliminating duplicating all adapter-generated headers to add a new header.

Note: The header and body data must be separated by <CR><LF><CR><LF> when sent to the HTTP Adapter. The Header and Headers groups are in the http.mtt example type tree provides a clear illustration of this concept. Each header is terminated by <CR><LF>, and the Headers group is terminated by another <CR><LF>.

The corresponding adapter command is -HDR.

For example:

-HDR[+][I]

+-> Appends user-supplied headers. It eliminates the need to duplicate the existing adapter-generated headers by adding a new one.

I-> In a request-reply scenario, indicates headers are provided in the input request but should not be returned in the response.

See also: HTTP Header Adapter Command (-HDR) Example.

HTTP(S) URL

This property establishes a connection between the adapter and the HTTP(S) server. Specify the URL to connect to the HTTP(S) server. You must also specify the name of the file you want to transfer.

The corresponding adapter command is -URL.

For example:

-URL {HTTP|HTTPS}://[user[:pass]@][host[:port]][/dir[/file]]

HTTP: The HTTP application protocol.

HTTPS: The HTTPS application protocol.

Note: On Linux platforms only, new attempts to negotiate an SSL/TLS connection will fail if there are more than 1024 open file descriptors in the currently running process.

user: Specifies the user name required to connect to the HTTP(S) server. Unless you specify a password, this must be followed by @ if you specify the host.

:pass: Specifies the password required that authenticates the user name. This parameter must be followed by @ if you also specify the host.

host: The HTTP(S) server name or address.

:port: The HTTP(S) server port name or number.

/dir: The directory path for the file.

/file: The file that the HTTP adapter retrieves.

Note: The user and pass arguments extend what the RFC supports for a URL path.

The URL does not need to be completely `URL-encoded'. As shown in the following table, you only need to use escape characters for tokens specifically used in a URL.

Replace the tokens with escape characters respectively:
Table 1. The following tokens are replaced with escape characters, respectively.
Tokens Escape characters
% (percent) %25
@ (at) %40
/ (forward slash) %2F
: (colon) %3A
; (semicolon) %3B
& (ampersand) Escape is not needed; use it as is.
# (pound) Escape is not needed; use it as is.
8-bit or control characters Escape is not needed; use it as is.

Any URL that contains spaces must be enclosed in quotes. For example:

-URL HTTP://Home Machine/HTTP Fil

Ignore

This property is used to ignore specific error codes. The corresponding adapter command is -IGNORE.

-IGNORE error_code[:error_code]

error_code: Specifies the HTTP error codes that are to be ignored by the adapter. A colon (:) must separate multiple error codes.

For example, the results for the following command -I 404:410 is:
  • The adapter ignores errors of type 404 (not found).
  • The adapter ignores errors of type 410 (gone).

Inline Output

This property is used to append data to the end of a URL. The corresponding adapter command is -INLINE.

You can use this command to submit a long query string to a form that uses method=GET. In the example below, the GET function has three arguments. The third argument is optional.

GET ( "HTTP", "-URL HTTP://[www.yahoo.com/cgi-bin/search?] - INLINE", "widgets")

Result: Gets the URL http://www.yahoo.com/cgi-bin/search?widgets.

KEEPALIVE

This command limits the connection reuse time of an HTTP adapter to prevent the reuse of closed connections. For example, when the connection exceeds the HTTP server KeepAlive Timeout setting.

A map reuses a connection under two conditions:
  • A running map reuses an active HTTP connection when it issues multiple HTTP requests with the same command-line connection options within the same map instance.
  • A map also reuses an idle HTTP connection when it runs multiple times in different map instances.

The corresponding adapter command is -KEEPALIVE.

-KEEPALIVEtimeout

timeout: The number of seconds that can elapse between consecutive reuses of a connection. After the timeout is exceeded, the HTTP adapter does not reuse the connection. Valid values are 0 - 99999. A 0 value specifies no connection reuse.

Example

A map issues multiple HTTP requests and runs multiple times. The HTTP server has a KeepAliveTimeout of 5 seconds. The config.yaml file has an IdleHTTP=10 /runtime/Connections Manager/HTTP/idle setting.

The map processing sequence is as follows:
  1. A new instance of Map 1 starts.
  2. Map 1, Input Card 1, iteration 1.
  3. Map 1, Output Card 1, iteration 1 runs.
    1. The HTTP adapter establishes Connection 1 or reuses a connection from a prior map with similar HTTP settings.
    2. The map issues an HTTP Post operation.
  4. Map 1, Input Card 1, iteration 2.
  5. Map 1, Output Card 1, iteration 2 runs.
    1. The HTTP adapter reuses Connection 1.
    2. The map issues an HTTP Post operation.
  6. Map 1 is completed.
  7. Map 1 runs again with a new instance number and repeats step 1.
Without the -KEEPALIVE command, consider these potential failure scenarios:
  • More than 5 seconds elapse between HTTP POST operations (steps 3b and 5b):
    • The HTTP server KeepAliveTimeout closes the connection.
    • The HTTP adapter attempts to use the active (but closed) connection in step 5b, and the map fails.
  • Less than 10 seconds elapse between step 5b in the first map instance and step 3a in the new map instance:
    • The HTTP adapter reuses the idle connection.
    • Step 3b activates the reused connection.
    • If more than 5 seconds elapse between HTTP POST operations (steps 3b and 5b), the HTTP server KeepAliveTimeout closes the connection. The map fails when the HTTP adapter attempts to use the active (but closed) connection in step 5a.
The -KEEPALIVE 5 adapter command prevents the HTTP adapter from reusing a connection if more than 5 seconds elapsed since the last time the map used the connection. Consider these successful scenarios:
  • More than 5 seconds elapse between HTTP POST operations (steps 3b and 5b):
    • The HTTP server KeepAliveTimeout closes the active connection.
    • The -KEEPALIVE 5 adapter command prevents the HTTP adapter from reusing the active (but closed) connection.
    • The HTTP adapter establishes a new connection, and the map completes successfully.
  • The connection’s idle time between map instances (steps 6 and 7) is between 5 - 9 seconds:
    • The idle time is less than 10 seconds, so the idle connection qualifies for reuse.
    • The time between requests exceeds the server KeepAliveTimeout value. The HTTP server closes the connection.
    • The -KEEPALIVE 5 adapter command prevents the HTTP adapter from reusing the idle (but closed) connection in step 3a.
    • The HTTP adapter establishes a new connection, and the map completes successfully.

Listen

Specifies a time (in seconds) to wait for messages to arrive. Suppose the URL does not become available in the specified listen time. In that case, you will receive an error message, Source not available.

The corresponding adapter command is -LSN.

-LSNseconds

seconds: The number of seconds to wait for messages to arrive.

The command -LSN 0 means do not wait at all. If no message is immediately available, the adapter returns a warning indicating no messages were found.

Method

Specifies the transfer method to use in an HTTP request. The HTTP request method can be any request the HTTP(S) server can process. Typical method names are PUT, GET, and POST. If the transfer is not specified, the default value for -METHOD is GET for input cards (to retrieve data) and POST for output cards (to store data).

The corresponding adapter command is -M or (-METHOD).

-METHODmethod_name

method_name: Transfer method used in an HTTP request.

Modified Since

This command generates the If-Modified-Since header information used during a GET request. The HTTP Adapter then adds the date to the request.

The corresponding adapter command is -MOD 0r (-MODIFIED).

-MODIFIEDhttp-date

http-date: Dates supported by RFC 1123 and RFC 850. The date can also be in the form of {wkday date time}, where:
  • wkday is Mon | Tue | Wed | Thu | Fri | Sat | Sun
  • date is month day (for example, JUN 06)
  • time is 00:00:00 to 23:59:59

Proxy URL

This property specifies the URL to connect to a proxy server and a file name for transferring the file.

The corresponding adapter command is -PROXY

For example:

-PROXY {HTTP|HTTPS}://[user[:pass]@][host[:port]][/dir[/file]]

HTTP: The HTTP application protocol.

HTTPS: The HTTPS application protocol.

user: Specifies the user name required to connect to the proxy server. Unless you specify a password, this must be followed by @ if you specify a host.

:pass: Specifies the password required that authenticates the user name. This parameter must be followed by @ if you specify the host.

host: The proxy server name or address.

:port: The proxy server port name or number.

/dir: The directory path for the file.

/file: The file that the HTTP Adapter retrieves.

Redirections Allowed

This property specifies the maximum number of redirections or file-moved messages allowed before the transfer request fails. The request fails if the adapter cannot locate the file within the limit. Use a value of 0 to prevent the adapter from following redirections.

The corresponding adapter command is -REDIRECT.

-REDIRECTlimit

Session

This property is used to coordinate the request and response of an HTTP. The corresponding adapter command is -SESSION.

-SESSION session_id[*]

session_id: Specifies the session identifier used for matching purposes. When receiving the HTTP request in a map, this contains the agent-assigned identifier. This identifier must be returned in the HTTP response.

*-> Permits all HTTP session IDs to be passed from the input card to the output card. You must enter a space between the -SESSION command and the asterisk (*) option for it to process correctly.

Generally, this command is written as:

-url localhost:8080/example -session * -type text/html -tv+

This command specifies the following:
  • Specifies the entry to search for the HTTP server called localhost on port 8080 and request the /example page.
  • Allows all session id's to be passed from the input card to the output card
  • Specifies text/html as the MIME Content-Type for the file used in the adapter request
  • Enables a verbose trace file called m4http.mtr in append mode

SOCKS URL

This property is used to connect to a SOCKS proxy server. The corresponding adapter command is -SOCKS

Generally, the command is written as:

-SOCKS {socks4|socks5}://[user[:pass]@][host[:port]]

socks4: The SOCKS protocol version 4.

socks5: The SOCKS protocol version 5.

user: Specifies the user name required to connect to the SOCKS server.

pass: Specifies the password required that authenticates the user name.

host: Specifies the server name or address of SOCKS.

port: Specifies the server port name or number of SOCKS.

Examples to connect to a SOCKS:
  • To connect to a SOCKS server named myhost using SOCKS protocol version 4, use the command:

    -SOCKS socks4://myhost

  • To connect to a SOCKS server named hmss with a user name of jbond and password of 007 using SOCKS protocol version 5, use the command:

    -SOCKS socks5://jbond:007@hmss

SSL Encryption Strength

Specifies the encryption strength for FTP connections. It can be set to either WEAK, which uses only weak algorithms, or STRONG, which uses only strong algorithms. If not specified, any available algorithm is used. If STRONG is unavailable, then WEAK is used. However, if the export version of the libraries is used, only weak algorithms are available, and this command is ignored.

This command is applicable only when the SSL secure_mode setting is in the config.yaml configuration file (/runtime/secure_mode) is set to 0, and RSA-based SSL encryption is in effect.

The corresponding adapter command is -STR.

-STR{WEAK|STRONG|ANY}

WEAK-> Use strong (non-exportable) encryption only.

STRONG-> Use exportable encryption only.

ANY-> Use any available encryption.

SSL Protocol

Specifies setting the Secure Socket Layer (SSL) protocol level. SSL enables the adapter to process FTPS URLs. The corresponding adapter command is -SPROTO.

-SPROTO{SSLv2|SSLv3|SSLv23|TLSv1}.

SSLv2: Specifies SSL protocol version 2.

SSLv3: Specifies SSL protocol version 3.

SSLv23: Specifies SSL protocol version 2, version 3, TLSv1, TLSv1.1 or TLSv1.2. Whenever the protocol method of SSLv23 is specified, the protocol of first choice is TLSv1.2, with a fallback to TLSv1.1, then to TLSv1, then to SSLv3, and finally to SSLv2, depending on the GSKit module that is loaded.

TLSv1: Specifies Transport Layer Security (TLS) protocol version 1.

If -SPROTO is not specified, the SSL Protocol defaults to SSLv23.

This command is applicable only when the SSL secure_mode setting is in the config.yaml configuration file (/runtime/secure_mode) is set to 0 and RSA-based SSL encryption is in effect. An SSL secure_mode setting of 1 or 2 supersedes this command.

Status Code

Returns an HTTP status code through the HTTP listener to the external HTTP client that requested a particular URL. The HTTP status code is associated with one or more map return codes. You can customize the HTTP status codes or use the default codes defined in the /runtime/M4HTTP/StatusCode path of the config.yaml configuration file. The corresponding adapter command is -STATUS_CODE.

Unable to insert image

  • map_return_code: The numeric code indicating the map execution result. See the Map Designer description of map execution and warning messages for a list of execution codes.
  • HTTP_status_code: An official HTTP status code maintained by the Internet Assigned Numbers Authority (IANA).

To specify multiple p return code/HTTP status code combinations on a single -STATUS_CODE keyword, separate each combination with a comma (,) character. For example:

-STATUS_CODE30 = 400 Bad Request, 12 = 404 Not Found

To link multiple map return codes to a single HTTP status code, delineate each map code with a colon (:) character. For example:

-STATUS_CODE8:9:30 = 400 Bad Request"

Status Line

Allows the initial HTTP response line to be returned to the map. The response line will be pre-pended to the regular data returned by the HTTP Adapter.

If the -STATUSLINE command is specified on the command line, the HTTP Adapter behaves like the Header (-HDR) command was also specified.

Status line is an optional command. The corresponding adapter command is -STATUSLINE.

For example, if you specify: -URL http://www.google.com -STATUSLINE

The command would return the complete HTTP response from Google, including HTTP headers and the HTTP Response status as the first line.

Status Page

Returns a text file containing the execution results of a session-specific map to an external HTTP client. The map can dynamically construct the file using predefined variables that resolve at map run time. The corresponding adapter command is -SP or -STATUS_PAGE.

Read syntax diagramSkip visual syntax diagram-STATUS_PAGE-SPfile_name.file_typefile_name* .file_type

file_name.file_type

file_name*.file_type

The file name of the status page file that the HTTP listener returns to the HTTP client. If the file name is not qualified, the map directory is used as the file path.

Optionally, you can generate a session-specific file name by appending an asterisk (*) wildcard character at the end of the file name. The wildcard character resolves to the Launcher session ID and timestamp.

Status page variables
With predefined status page variables, a map can dynamically create the status page the HTTP listener returns to the external HTTP client. When the map runs, the variables in the status page resolve to map execution details such as error code and map name. The variables are as follows:
  • %TX_ADAPTER_STATUS_CODE%: The selected status code from the “-status_code” option.
  • %TX_MAP_ERROR_CODE%: The map error code.
  • %TX_MAP_ERROR_MSG%: The map error message.
  • %TX_MAP_NAME%: The map name
  • %TX_MAP_INSTANCE%: The map instance number
  • %TX_MAP_SESSSION_ID%: The unique ID assigned by the HTTP listener to this instance of the triggered input card
  • %TX_TIMESTAMP%: The timestamp when the HTTP listener sends the response to the external HTTP client
Status page files for multi-threaded maps

To support multi-threaded maps, append an asterisk (*) character to the file name of the HTTP adapter status page. The asterisk is a wildcard character that generates a unique file name for the status page to ensure that other instances of the same map do not overwrite the file contents.

The wildcard character resolves to session_ID, timestamp. The session_ID is a unique alphanumeric string that the HTTP listener associates with an instance of a triggered input card.

For example, a multi-threaded map's input card can include the following command:

-status_page http_response_*.html

When the map runs, the wildcard character resolves to a unique file name, such as:

http_response_NEED-SESSION-ID,NEED-TIMESTAMP-FORMAT.html

Status Page Default

Specifies a backup status page to be returned to an external HTTP client when the page specified by the -STATUS_PAGE option fails. The /runtime/M4HTTP/StatusPage path of the config.yaml configuration file defines a single default status page.

The corresponding adapter command is -STATUS_PAGE_DEFAULT. This option allows you to create a custom default page for each watch.

-STATUS_PAGE_DEFAULT is optional. If you omit it, the HTTP listener returns the default status page defined in the/runtime/M4HTTP/StatusPage path of the config.yaml configuration file.

Read syntax diagramSkip visual syntax diagram-STATUS_PAGE_DEFAULT-SPDfile_name.file_type

file_name.file_type

file_name*.file_type

The file name of the status page file that the HTTP listener returns to the HTTP client. If the file name is not qualified, the map directory is used as the file path.

Timeout

This property sets the socket timeouts to a specified number of seconds. The default timeout is 300 seconds.

The corresponding adapter command is -TIMEOUT.

-TIMEOUT time_in_seconds

time_in_seconds: Specifies the number of seconds for the sockets timeout to be set. The default timeout is set to 300 seconds.

Trace

Produces a diagnostics file that contains detailed information about HTTP Adapter activity. The corresponding adapter command is -T or (-TRACE).

The default filename is m4http.mtr, created in the map directory unless otherwise specified.

Generally, the command is written as:

-T[E][+][S|V] [full_path]

E: Produces a trace file containing only the adapter errors during map execution.

+: Appends trace information to the existing log file.

S: Summary mode. Default value. Record only minimal information in the log file.

V: Verbose mode. Record all activity occurring while the adapter retrieves data in the log file. If not specified, summary mode is assumed.

full_path: Creates a trace file with the specified name in the directory.

Note: Because summary mode is the default, using -TRACES provides the same output as -TRACE.
Note: You can dynamically override the adapter command line trace options using the Management Console. See "Dynamic Adapter Tracing" in the Launcher documentation for detailed information.
Tunneling

Establishes a clean TCP connection to the proxy server and then upgrades to SSL using the CONNECT method. This command should only be used in cases where the specified proxy URL is non-secure (http), and the request URL is secure (https). The corresponding adapter command is -TUNNELING.

Generally, the command is written as:

-url <https://secure-server:8080/example> -proxy <http://user@proxy-server:2888> -tunneling

This command specifies:

  • The entry to search for the HTTPS server called a secure-server on port 8080 and request the /example page.
  • The proxy server called a proxy-server with a plain (non-secure) HTTP connection.
  • The tunneling mechanism should be used to upgrade the non-secure proxy connection to a secure one for accessing the given URL.

Type

Specifies the type of content that is contained in the body of the HTTP request. The default type is application/octet-stream. See "Known Limitations" for additional information. The corresponding adapter command is -TY or (-TYPE).

When you send a SOAP-based HTTP request, encoding the HTTP request's content might require you to specify the charset parameter of the -TYPE command. The default encoding is ISO-8859-1, as originally specified by RFC 2616. The SOAP adapter requires a UTF-8 encoded response from the HTTP server. However, the request can be in any Western or UTF-8 encoded format.

-TYPE content_type [charset=encoding]

  • content_type: The default content type is application/octet stream. You might use other types. However, do not use compound types such as multipart.
  • charset: Specifies the encoding of the content in the HTTP request's body. The charset keyword is optional. If you omit it, the default encoding is ISO-8859-1.

When you specify the charset keyword on a SOAP adapter command line, enclose both the content_type and the charset keyword and value pair within two sets of double quotation marks (""content_type charset=encoding"") to avoid parsing errors.

The following command line notifies the HTTP Server that the content is UTF-8 data:

-TYPE "text/xml; charset=utf-8"

The following command line uses a SOAP adapter GET request and notifies the HTTP Server that the content is UTF-8 data. Note that both the content_type and the charset keyword and value pair are enclosed within two sets of double quotation marks:

=GET("SOAP", "-T -SA http://localhost/WebService/test -TRANSPORT 'HTTP(-HDRI+ -T -TYPE ""text/xml; charset=utf-8"" -METHOD POST -URL http://localhost/WebService/test)' ", PACKAGE(SoapRequest))