HTTP Adapter commands
The following table lists valid commands for the HTTP Adapter, their syntax, and whether they are supported (✓) for use with data sources, targets, or both.
Name | Syntax | Source | Target |
---|---|---|---|
Client Certificate | -CERT {label} | ✓ | ✓ |
From | -FROM address | ✓ | ✓ |
Header | -HDR [+][I] | ✓ | ✓ |
HTTP(S) | -URL
{HTTP|HTTPS}://[user[:pass]@] [host[:port]][/dir[/file]] |
✓ | ✓ |
Ignore | -IGNORE error_code[:error_code...] | ✓ | ✓ |
Inline Output | -INLINE | ✓ | |
Listen | -LSN seconds | ✓ | |
Method | -METHOD method_name | ✓ | ✓ |
Modified Since | -MODIFIED http-date | ✓ | ✓ |
Proxy URL | -PROXY
{HTTP|HTTPS}://[user[:pass]@] [host[:port]][/dir[/file]] |
✓ | ✓ |
Redirections Allowed | -REDIRECT limit | ✓ | ✓ |
Session | -SESSION session_id [*] | ✓ | ✓ |
Server Name Indication | -SNI | ✓ | ✓ |
SOCKS URL | -SOCKS
{socks4|socks5}://[user[:pass]@] [host[:port]] |
✓ | ✓ |
SSL Encryption Strength | -STR {WEAK|STRONG|ANY} | ✓ | ✓ |
SSL Protocol | -SPROTO {SSLv2|SSLv3|SSLv23|TLSv1} | ✓ | ✓ |
Status Code | -STATUS_CODE {map_return_code=HTTP_status_code[,map_return_code=HTTP_status_code...] | map_return_code:map_return_code[:map_return_code...]=HTTP_status_code} | ✓ | |
Status Line | -STATUSLINE | ✓ | |
Status Page | -STATUS_PAGE file_name[*].file_type | ✓ | |
Status Page Default | -STATUS_PAGE_DEFAULT file_name.file_type | ✓ | |
Timeout | -TIMEOUT time_in_seconds | ✓ | ✓ |
Trace | -T[E][+][S|V] [full_path] | ✓ | ✓ |
Tunneling | -TUNNELING | ✓ | ✓ |
Type | -TYPE content_type [charset=encoding] | ✓ | ✓ |
Client Certificate
Specifies the label of the client certificate in the key store. This command is applicable only when the SSL_CLIENT secure_mode setting in the config.yaml file is set, and SSL/TLS encryption is in effect.
The corresponding adapter command is -CERT {label}.
label: The name of the entry in the certificate store, which is specified by the key_store field in the config.yaml file.
From
Specifies the email address of a contact person. The adapter sends alerts to the contact person in case of a problem using requests.
The corresponding adapter command is -FROM.
-FROMaddress
address: Specifies the e-mail address of a contact person
Header
This command is used to send message headers from data in a map file or to return message headers to the map file. In addition to generating new headers, you can append user-supplied headers, eliminating duplicating all adapter-generated headers to add a new header.
The corresponding adapter command is -HDR.
For example:
-HDR[+][I]
+-> Appends user-supplied headers. It eliminates the need to duplicate the existing adapter-generated headers by adding a new one.
I-> In a request-reply scenario, indicates headers are provided in the input request but should not be returned in the response.
See also: HTTP Header Adapter Command (-HDR) Example.
HTTP(S) URL
This property establishes a connection between the adapter and the HTTP(S) server. Specify the URL to connect to the HTTP(S) server. You must also specify the name of the file you want to transfer.
The corresponding adapter command is -URL.
For example:
-URL {HTTP|HTTPS}://[user[:pass]@][host[:port]][/dir[/file]]
HTTP: The HTTP application protocol.
HTTPS: The HTTPS application protocol.
user: Specifies the user name required to connect to the HTTP(S) server. Unless you specify a password, this must be followed by @ if you specify the host.
:pass: Specifies the password required that authenticates the user name. This parameter must be followed by @ if you also specify the host.
host: The HTTP(S) server name or address.
:port: The HTTP(S) server port name or number.
/dir: The directory path for the file.
/file: The file that the HTTP adapter retrieves.
The URL does not need to be completely `URL-encoded'. As shown in the following table, you only need to use escape characters for tokens specifically used in a URL.
Tokens | Escape characters |
---|---|
% (percent) | %25 |
@ (at) | %40 |
/ (forward slash) | %2F |
: (colon) | %3A |
; (semicolon) | %3B |
& (ampersand) | Escape is not needed; use it as is. |
# (pound) | Escape is not needed; use it as is. |
8-bit or control characters | Escape is not needed; use it as is. |
Any URL that contains spaces must be enclosed in quotes. For example:
-URL HTTP://Home Machine/HTTP Fil
Ignore
This property is used to ignore specific error codes. The corresponding adapter command is -IGNORE.
-IGNORE error_code[:error_code]
error_code: Specifies the HTTP error codes that are to be ignored by the adapter. A colon (:) must separate multiple error codes.
- The adapter ignores errors of type 404 (not found).
- The adapter ignores errors of type 410 (gone).
Inline Output
This property is used to append data to the end of a URL. The corresponding adapter command is -INLINE.
You can use this command to submit a long query string to a form that uses method=GET. In the example below, the GET function has three arguments. The third argument is optional.
GET ( "HTTP", "-URL HTTP://[www.yahoo.com/cgi-bin/search?] - INLINE", "widgets")
Result: Gets the URL http://www.yahoo.com/cgi-bin/search?widgets.
KEEPALIVE
This command limits the connection reuse time of an HTTP adapter to prevent the reuse of closed connections. For example, when the connection exceeds the HTTP server KeepAlive Timeout setting.
- A running map reuses an active HTTP connection when it issues multiple HTTP requests with the same command-line connection options within the same map instance.
- A map also reuses an idle HTTP connection when it runs multiple times in different map instances.
The corresponding adapter command is -KEEPALIVE.
-KEEPALIVEtimeout
timeout: The number of seconds that can elapse between consecutive reuses of a connection. After the timeout is exceeded, the HTTP adapter does not reuse the connection. Valid values are 0 - 99999. A 0 value specifies no connection reuse.
Example
A map issues multiple HTTP requests and runs multiple times. The HTTP server has a KeepAliveTimeout of 5 seconds. The config.yaml file has an IdleHTTP=10 /runtime/Connections Manager/HTTP/idle setting.
- A new instance of Map 1 starts.
- Map 1, Input Card 1, iteration 1.
- Map 1, Output Card 1, iteration 1 runs.
- The HTTP adapter establishes Connection 1 or reuses a connection from a prior map with similar HTTP settings.
- The map issues an HTTP Post operation.
- Map 1, Input Card 1, iteration 2.
- Map 1, Output Card 1, iteration 2 runs.
- The HTTP adapter reuses Connection 1.
- The map issues an HTTP Post operation.
- Map 1 is completed.
- Map 1 runs again with a new instance number and repeats step 1.
- More than 5 seconds elapse between HTTP POST operations (steps 3b and
5b):
- The HTTP server KeepAliveTimeout closes the connection.
- The HTTP adapter attempts to use the active (but closed) connection in step 5b, and the map fails.
- Less than 10 seconds elapse between step 5b in the first map instance and step 3a in the
new map instance:
- The HTTP adapter reuses the idle connection.
- Step 3b activates the reused connection.
- If more than 5 seconds elapse between HTTP POST operations (steps 3b and 5b), the HTTP server KeepAliveTimeout closes the connection. The map fails when the HTTP adapter attempts to use the active (but closed) connection in step 5a.
- More than 5 seconds elapse between HTTP POST operations (steps 3b and 5b):
- The HTTP server KeepAliveTimeout closes the active connection.
- The -KEEPALIVE 5 adapter command prevents the HTTP adapter from reusing the active (but closed) connection.
- The HTTP adapter establishes a new connection, and the map completes successfully.
- The connection’s idle time between map instances (steps 6 and 7) is between 5 - 9 seconds:
- The idle time is less than 10 seconds, so the idle connection qualifies for reuse.
- The time between requests exceeds the server KeepAliveTimeout value. The HTTP server closes the connection.
- The -KEEPALIVE 5 adapter command prevents the HTTP adapter from reusing the idle (but closed) connection in step 3a.
- The HTTP adapter establishes a new connection, and the map completes successfully.
Listen
Specifies a time (in seconds) to wait for messages to arrive. Suppose the URL does not become available in the specified listen time. In that case, you will receive an error message, Source not available.
The corresponding adapter command is -LSN.
-LSNseconds
seconds: The number of seconds to wait for messages to arrive.
The command -LSN 0 means do not wait at all. If no message is immediately available, the adapter returns a warning indicating no messages were found.
Method
Specifies the transfer method to use in an HTTP request. The HTTP request method can be any request the HTTP(S) server can process. Typical method names are PUT, GET, and POST. If the transfer is not specified, the default value for -METHOD is GET for input cards (to retrieve data) and POST for output cards (to store data).
The corresponding adapter command is -M or (-METHOD).
-METHODmethod_name
method_name: Transfer method used in an HTTP request.
Modified Since
This command generates the If-Modified-Since header information used during a GET request. The HTTP Adapter then adds the date to the request.
The corresponding adapter command is -MOD 0r (-MODIFIED).
-MODIFIEDhttp-date
- wkday is Mon | Tue | Wed | Thu | Fri | Sat | Sun
- date is month day (for example, JUN 06)
- time is 00:00:00 to 23:59:59
Proxy URL
This property specifies the URL to connect to a proxy server and a file name for transferring the file.
The corresponding adapter command is -PROXY
For example:
-PROXY {HTTP|HTTPS}://[user[:pass]@][host[:port]][/dir[/file]]
HTTP: The HTTP application protocol.
HTTPS: The HTTPS application protocol.
user: Specifies the user name required to connect to the proxy server. Unless you specify a password, this must be followed by @ if you specify a host.
:pass: Specifies the password required that authenticates the user name. This parameter must be followed by @ if you specify the host.
host: The proxy server name or address.
:port: The proxy server port name or number.
/dir: The directory path for the file.
/file: The file that the HTTP Adapter retrieves.
Redirections Allowed
This property specifies the maximum number of redirections or file-moved messages allowed before the transfer request fails. The request fails if the adapter cannot locate the file within the limit. Use a value of 0 to prevent the adapter from following redirections.
The corresponding adapter command is -REDIRECT.
-REDIRECTlimit
Session
This property is used to coordinate the request and response of an HTTP. The corresponding adapter command is -SESSION.
-SESSION session_id[*]
session_id: Specifies the session identifier used for matching purposes. When receiving the HTTP request in a map, this contains the agent-assigned identifier. This identifier must be returned in the HTTP response.
*-> Permits all HTTP session IDs to be passed from the input card to the output card. You must enter a space between the -SESSION command and the asterisk (*) option for it to process correctly.
Generally, this command is written as:
-url localhost:8080/example -session * -type text/html -tv+
- Specifies the entry to search for the HTTP server called localhost on port 8080 and request the /example page.
- Allows all session id's to be passed from the input card to the output card
- Specifies text/html as the MIME Content-Type for the file used in the adapter request
- Enables a verbose trace file called m4http.mtr in append mode
SOCKS URL
This property is used to connect to a SOCKS proxy server. The corresponding adapter command is -SOCKS
Generally, the command is written as:
-SOCKS {socks4|socks5}://[user[:pass]@][host[:port]]
socks4: The SOCKS protocol version 4.
socks5: The SOCKS protocol version 5.
user: Specifies the user name required to connect to the SOCKS server.
pass: Specifies the password required that authenticates the user name.
host: Specifies the server name or address of SOCKS.
port: Specifies the server port name or number of SOCKS.
- To connect to a SOCKS server named myhost using SOCKS protocol
version 4, use the command:
-SOCKS socks4://myhost
- To connect to a SOCKS server named hmss with a user name of
jbond and password of 007 using SOCKS protocol version 5, use the command:
-SOCKS socks5://jbond:007@hmss
SSL Encryption Strength
Specifies the encryption strength for FTP connections. It can be set to either WEAK, which uses only weak algorithms, or STRONG, which uses only strong algorithms. If not specified, any available algorithm is used. If STRONG is unavailable, then WEAK is used. However, if the export version of the libraries is used, only weak algorithms are available, and this command is ignored.
This command is applicable only when the SSL secure_mode setting is in the config.yaml configuration file (/runtime/secure_mode) is set to 0, and RSA-based SSL encryption is in effect.
The corresponding adapter command is -STR.
-STR{WEAK|STRONG|ANY}
WEAK-> Use strong (non-exportable) encryption only.
STRONG-> Use exportable encryption only.
ANY-> Use any available encryption.
SSL Protocol
Specifies setting the Secure Socket Layer (SSL) protocol level. SSL enables the adapter to process FTPS URLs. The corresponding adapter command is -SPROTO.
-SPROTO{SSLv2|SSLv3|SSLv23|TLSv1}.
SSLv2: Specifies SSL protocol version 2.
SSLv3: Specifies SSL protocol version 3.
SSLv23: Specifies SSL protocol version 2, version 3, TLSv1, TLSv1.1 or TLSv1.2. Whenever the protocol method of SSLv23 is specified, the protocol of first choice is TLSv1.2, with a fallback to TLSv1.1, then to TLSv1, then to SSLv3, and finally to SSLv2, depending on the GSKit module that is loaded.
TLSv1: Specifies Transport Layer Security (TLS) protocol version 1.
If -SPROTO is not specified, the SSL Protocol defaults to SSLv23.
This command is applicable only when the SSL secure_mode setting is in the config.yaml configuration file (/runtime/secure_mode) is set to 0 and RSA-based SSL encryption is in effect. An SSL secure_mode setting of 1 or 2 supersedes this command.
Status Code
Returns an HTTP status code through the HTTP listener to the external HTTP client that requested a particular URL. The HTTP status code is associated with one or more map return codes. You can customize the HTTP status codes or use the default codes defined in the /runtime/M4HTTP/StatusCode path of the config.yaml configuration file. The corresponding adapter command is -STATUS_CODE.
Unable to insert image
- map_return_code: The numeric code indicating the map execution result. See the Map Designer description of map execution and warning messages for a list of execution codes.
- HTTP_status_code: An official HTTP status code maintained by the Internet Assigned Numbers Authority (IANA).
To specify multiple p return code/HTTP status code combinations on a single -STATUS_CODE keyword, separate each combination with a comma (,) character. For example:
-STATUS_CODE “30 = 400 Bad Request, 12 = 404 Not Found”
To link multiple map return codes to a single HTTP status code, delineate each map code with a colon (:) character. For example:
-STATUS_CODE “8:9:30 = 400 Bad Request"
Status Line
Allows the initial HTTP response line to be returned to the map. The response line will be pre-pended to the regular data returned by the HTTP Adapter.
If the -STATUSLINE command is specified on the command line, the HTTP Adapter behaves like the Header (-HDR) command was also specified.
Status line is an optional command. The corresponding adapter command is -STATUSLINE.
For example, if you specify: -URL http://www.google.com -STATUSLINE
The command would return the complete HTTP response from Google, including HTTP headers and the HTTP Response status as the first line.
Status Page
Returns a text file containing the execution results of a session-specific map to an external HTTP client. The map can dynamically construct the file using predefined variables that resolve at map run time. The corresponding adapter command is -SP or -STATUS_PAGE.
file_name.file_type
file_name*.file_type
The file name of the status page file that the HTTP listener returns to the HTTP client. If the file name is not qualified, the map directory is used as the file path.
Optionally, you can generate a session-specific file name by appending an asterisk (*) wildcard character at the end of the file name. The wildcard character resolves to the Launcher session ID and timestamp.
- Status page variables
-
With predefined status page variables, a map can dynamically create the status page the HTTP listener returns to the external HTTP client. When the map runs, the variables in the status page resolve to map execution details such as error code and map name. The variables are as follows:
- %TX_ADAPTER_STATUS_CODE%: The selected status code from the “-status_code” option.
- %TX_MAP_ERROR_CODE%: The map error code.
- %TX_MAP_ERROR_MSG%: The map error message.
- %TX_MAP_NAME%: The map name
- %TX_MAP_INSTANCE%: The map instance number
- %TX_MAP_SESSSION_ID%: The unique ID assigned by the HTTP listener to this instance of the triggered input card
- %TX_TIMESTAMP%: The timestamp when the HTTP listener sends the response to the external HTTP client
- Status page files for multi-threaded maps
-
To support multi-threaded maps, append an asterisk (*) character to the file name of the HTTP adapter status page. The asterisk is a wildcard character that generates a unique file name for the status page to ensure that other instances of the same map do not overwrite the file contents.
The wildcard character resolves to
session_ID, timestamp
. The session_ID is a unique alphanumeric string that the HTTP listener associates with an instance of a triggered input card.For example, a multi-threaded map's input card can include the following command:
-status_page http_response_*.html
When the map runs, the wildcard character resolves to a unique file name, such as:
http_response_NEED-SESSION-ID,NEED-TIMESTAMP-FORMAT.html
Status Page Default
Specifies a backup status page to be returned to an external HTTP client when the page specified by the -STATUS_PAGE option fails. The /runtime/M4HTTP/StatusPage path of the config.yaml configuration file defines a single default status page.
The corresponding adapter command is -STATUS_PAGE_DEFAULT. This option allows you to create a custom default page for each watch.
-STATUS_PAGE_DEFAULT is optional. If you omit it, the HTTP listener returns the default status page defined in the/runtime/M4HTTP/StatusPage path of the config.yaml configuration file.
file_name.file_type
file_name*.file_type
The file name of the status page file that the HTTP listener returns to the HTTP client. If the file name is not qualified, the map directory is used as the file path.
Timeout
This property sets the socket timeouts to a specified number of seconds. The default timeout is 300 seconds.
The corresponding adapter command is -TIMEOUT.
-TIMEOUT time_in_seconds
time_in_seconds: Specifies the number of seconds for the sockets timeout to be set. The default timeout is set to 300 seconds.
Produces a diagnostics file that contains detailed information about HTTP Adapter activity. The corresponding adapter command is -T or (-TRACE).
The default filename is m4http.mtr, created in the map directory unless otherwise specified.
Generally, the command is written as:
-T[E][+][S|V] [full_path]
E: Produces a trace file containing only the adapter errors during map execution.
+: Appends trace information to the existing log file.
S: Summary mode. Default value. Record only minimal information in the log file.
V: Verbose mode. Record all activity occurring while the adapter retrieves data in the log file. If not specified, summary mode is assumed.
full_path: Creates a trace file with the specified name in the directory.
Establishes a clean TCP connection to the proxy server and then upgrades to SSL using the CONNECT method. This command should only be used in cases where the specified proxy URL is non-secure (http), and the request URL is secure (https). The corresponding adapter command is -TUNNELING.
Generally, the command is written as:
-url <https://secure-server:8080/example> -proxy <http://user@proxy-server:2888> -tunneling
This command specifies:
- The entry to search for the HTTPS server called a secure-server on port 8080 and request the /example page.
- The proxy server called a proxy-server with a plain (non-secure) HTTP connection.
- The tunneling mechanism should be used to upgrade the non-secure proxy connection to a secure one for accessing the given URL.
Type
Specifies the type of content that is contained in the body of the HTTP request. The default type is application/octet-stream. See "Known Limitations" for additional information. The corresponding adapter command is -TY or (-TYPE).
When you send a SOAP-based HTTP request, encoding the HTTP request's content might require you to specify the charset parameter of the -TYPE command. The default encoding is ISO-8859-1, as originally specified by RFC 2616. The SOAP adapter requires a UTF-8 encoded response from the HTTP server. However, the request can be in any Western or UTF-8 encoded format.
-TYPE content_type [charset=encoding]
- content_type: The default content type is application/octet stream. You might use other types. However, do not use compound types such as multipart.
- charset: Specifies the encoding of the content in the HTTP request's body. The charset keyword is optional. If you omit it, the default encoding is ISO-8859-1.
When you specify the charset keyword on a SOAP adapter command line,
enclose both the content_type and the charset keyword and
value pair within two sets of double quotation marks (""content_type
charset=encoding""
) to avoid parsing errors.
The following command line notifies the HTTP Server that the content is UTF-8 data:
-TYPE "text/xml; charset=utf-8"
The following command line uses a SOAP adapter GET request and notifies the HTTP Server that the content is UTF-8 data. Note that both the content_type and the charset keyword and value pair are enclosed within two sets of double quotation marks:
=GET("SOAP", "-T -SA http://localhost/WebService/test -TRANSPORT 'HTTP(-HDRI+ -T -TYPE ""text/xml; charset=utf-8"" -METHOD POST -URL http://localhost/WebService/test)' ", PACKAGE(SoapRequest))