HTTP Monitor

The HTTP monitor checks the availability and response time of web servers.

It can monitor individual web pages, including that uses CGI, which would normally require the user to enter data into fields. It can also monitor the download time for elements such as images on a web page.

Table 1. HTTP monitor file summary 
Monitor files Name or location
Monitor executable nco_m_http
Properties file $ISHOME/etc/props/http.props
Rules file $ISHOME/etc/rules/http.rules
Log file $ISHOME/log/http.log
Guidelines for configuring HTTP monitor
The HTTP and HTTPS monitors check the availability and response time of web servers. Use the HTTP monitor in the following situations:
  • The target website is static.

    For dynamic websites, use the TRANSX monitor.

  • The target website is served over the HTTP protocol.

    For websites that deliver content over the HTTPS protocol, select the HTTPS monitor.

  • To perform monitoring across multiple platforms.
  • Where speed is a determining factor (the HTTP monitor provides high performance).
HTTP request types

The HTTP monitor emulates a web browser that supports the HTTP/1.0 protocol. To test the web server, the monitor sends it a request for a web page using any of the following HTTP request types:

  • HEAD

    The HEAD command attempts to access a web page and return the HTTP header. Issuing the HEAD command is a fast way to check that a web page is accessible.

  • GET

    The GET command attempts to access the web page and return the whole page, including the HTTP header. It doesn’t attempt to return files associated with the page, such as images.

  • GETALL

    The GETALL command attempts to access the web page and return the whole page including the HTTP header, background, images, applets, frames, cascading stylesheet (CSS) files, and scripts. Like the HEAD and GET commands, this command also checks that a web page is accessible, but because the GETALL command returns the whole page and all its associated files, it may give a more realistic indication of the time taken to access the page. The monitor also uses multiple threads during a GETALL command to more accurately match the behavior of web browsers.

  • POST

    The POST command attempts to access a web page that contains an HTTP form and complete the fields of that form. Add body text for the POST request to the Body tab in the Internet Service Monitoring configuration, or use the @Body group in the Internet Service Monitoring configuration or ismbatch. Alternatively, you can use the FORM parameters. You can’t use both Body text and FORM parameters in the POST request.

Using a proxy server

You can test the availability of web pages through a proxy server. When you configure the monitor to use a proxy, it sends HTTP requests through the proxy. If required, you can bypass the proxy cache. You configure the parameters for the proxy server on the Proxy Details tab. The HTTP monitor supports authenticated access to proxy servers. This authentication is independent of any authentication required by the target web page.

Proxy server elements

In earlier versions, when you configured a profile element to use a proxy server, by default the HTTP monitor inserted the name of the proxy server and port into the $server and $port elements, instead of the name and port of the intended destination server. To preserve the value of the intended destination server name and port in earlier versions, set the generateProxyTokens property to 1, or start the monitor with the -generateproxytokens command line parameter.

In addition to preserving the values of the $server and $port elements when this property or command line parameter is set, the monitor generates the $proxyServer, $proxyPort, $proxyAuthType, $proxyUsername, and $proxyCache elements.

Authentication

If the web page that you want to monitor, or the proxy server you want to test, requires authentication specify credentials for accessing the page in the authenticationtype, username, and passwordparameter fields on the Advanced or Proxy Details tab.

To disable authentication, set authenticationtype to NONE.

To select basic authentication:

  1. Set the authenticationtype to BASIC.
  2. Set username and password to those required by the web page or proxy server.

To select NTLM:

  1. Set authenticationtype to NTLMv1 or NTLMv2.
  2. Set username and password to those required by the web page or proxy server.
Note:

The monitor limits the length of HTTP requests to 4096 characters. If the length of the additional form data results in a request length that exceeds this limit, the monitor doesn’t include the additional form data in the request.

Configuring HTTP Monitor Service Test

Use the HTTP monitor configuration parameters to define HTTP service tests.

Table 2. HTTP monitor configuration
Field Description
server The host name of the server to be monitored. Example is www.mycompany.com
page The URL of the page to be monitored. Example is index.html
description A text field for providing descriptive information on the element. Example ismonitoring via a proxy server
port

The port on the HTTP server to use.

Default: 80

localip Specifies the IP address of the network interface that the monitor uses for the test. If this field is empty, the monitor uses the interface specified by the IpAddress property.
version The HTTP protocol version to be used:
  • 1.0
  • 1.1

Default: 1.0

command

The HTTP request type:

  • HEAD
  • GET
  • GETALL
  • POST

Default: GET

formname When used in a transaction, the HTTP monitor scans the specified form for default values. Any values found are automatically completed the next HTTP step in the transaction.
authenticationtype

Specifies the challenge -response authentication mechanism for authenticating network users:

  • NONE - No authentication
  • BASIC
  • NTLMv1 - Windows NTLM version 1 challenge/response authentication
  • NTLMv2 - Windows NTLM version 2

Default: NONE

username The username (account name) for the monitor to use to log in to the server.
password The password corresponding to the username for the monitor to use to log into the server.
timeout

The time, in seconds, to wait for the server to respond.

Default: 30

poll

The time, in seconds, between each poll.

Default: 300

.
failureretests

The number of times to retest before indicating a failure.

Default: 0

retestinterval

The time, in seconds, to wait between each failure retest.

Default: 10

verifycertificate It’s by default disabled.
Proxy details
server The host name of the proxy server.
port

The port on the proxy server to use.

Default: 8080

authenticationtype

The server authentication type for the proxy server. For more information, see previous authenticationtype.

Default: NONE

username Used by the monitor together with the password to log into the proxy server.
password Used by the monitor together with the username to log into the proxy server and the label is password.
useproxy Configures the monitor to perform the request using a proxy server.
  • proxy (use true in ismbatch)
  • noproxy (use false in ismbatch)

Default: noproxy

hostnamelookuppreference Determines which IP version, IPv6 or IPv4, is applied to the supplied host name. The options are:
  • default sets the monitor to use monitor-wide properties settings. This is the default.
  • 4Then6 selects IPv4 and then IPv6. Uses IPv4 addresses if they are available. If no IPv4 addresses are found, IPv6 addresses are used.
  • 6Then4 selects IPv6 and then IPv4. Uses IPv6 addresses if they are available. If no IPv6 addresses are found, IPv4 addresses are used.
  • 4Only selects IPv4 only. Uses IPv4 addresses only. If there are no IPv4 addresses, the poll returns an error.
  • 6Only selects IPv6 only. Uses IPv6 addresses only. If there are no IPv6 addresses, the poll returns an error.
  • 6Or4 selects either IPv4 or IPv6. Uses the first address returned from the host name.
nocache It’s by default set to cache.
Regular Expression

You can perform a regular expression search on the information being downloaded by entering up to 50 different regular expressions. The HTTP monitor attempts to match the contents retrieved to each of the regular expressions. If a match for a specified regular expression is found, the matched lines (or as much as can fit in the monitor’s internal buffer) are returned in the corresponding $regexpMatchn element. If the regular expression matches more than once in the information downloaded, only the first match is returned. The status of each regular expression test is indicated by the $regexpStatusn elements. You can use the regular expression matches and their status information as criteria for service level classifications. SeeTable 3 for information about regular expression syntax,

Head and Form parameter

The HTTP monitor can send extra data in the header fields and message body of HTTP requests.

You configure the parameters for this extra data on the Parameters tab. The parameters are Name, Value and Type and they operate in the following way:

  • Name-value pairs of type HEAD specify additional header fields, such as User-Agent and Referer, included in all HTTP requests sent by the monitor. Header fields may be specified for any type of HTTP method (GET, GETALL, HEAD or POST).

    For ITCAM for Transactions V7.4.0.1 and later, the default user agent head parameter, Mozilla/5.0 (ISM-MONITOR) is added for every new HTTP or HTTPS element. The default user-agent header is so that the HTTP and HTTPS monitors can be used for websites that switch content based on the browser client.

  • Name-value pairs of type FORM specify extra data included in the message body of HTTP POSTrequests sent by the monitor. If the target page contains a form matching the name specified in the formname field, the monitor treats any name-value pairs in the form as if they were configured in the profile element.
Note:

The monitor limits the length of HTTP requests to 4096 characters. If the length of the additional form data results in a request length that exceeds this limit, the monitor doesn’t include the additional form data in the request.

Monitor elements

In addition to the test results common to all elements, the HTTP monitor generates a set of test results containing data specific to HTTP service tests. Elements indicated by an asterisk (*) are available as attribute. The names of the attributes are shown within brackets. Absence of an asterisk indicates there’s no equivalent attribute. Attributes shown in bracket but without an element indicates that they are only available as attribute, there’s no equivalent element.

Table 3. HTTP monitor elements
Element Description
$bytesPerSec*(BytesPerSec) The average number of bytes transferred each second.
$bytesTransfered*
(BytesTransferred)
The number of bytes uploaded or downloaded.
$checksum The Checksum element doesn’t normally provide meaningful values for service
level classifications because checksum values aren’t known when the profile
element is created (the monitor calculates checksum values while
tests are in progress). The $checksum and $previousChecksum monitor
elements are intended for alert enrichment using the monitor's rules file.
$command The HTTP command issued by the monitor. For example, HEAD, GET, GETALL, or POST.
$connectTime*(ConnectTime) The time taken to connect to the server.
$downloadTime*(DownloadTime) The time taken to download the file.
(Elements) The number of page elements received.
$formname The name of the form used in a POST action.
$lastStatus*(PageStatus) If a profile element retrieves multiple pages, this element contains the result
string of the last page retrieved. This value is the same as that of $urlResultn
where n is equal to the value of $pageCount.
$lastModified The value of the Last-Modified HTTP header field of the first page retrieved.
$page*(Page) The page accessed on the HTTP server.
$pageCount The total number of resources downloaded during a GETALL test, excluding the
test page itself. If the tested page doesn’t refer to any other resources, this
element isn’t generated.
$port*(Port) The port used to access the HTTP server. If the test used a proxy server, this
is the value of the port on the proxy server to which the request was
submitted. To preserve the port of the intended destination server, set the
generateProxyTokens property to 1, or start the monitor with the
-generateproxytokens command line parameter
$previousChecksum The PreviousChecksum element doesn’t normally provide meaningful values
for service level classifications because checksaren’tes aren’t known
when the profile element is created (the monitor calculates checksum values
while tests are in progress). The $previousChecksum and $checksum monitor
elements are intended for alert enrichment using the monitor's rules file.
$proxyAuthType The server authentication type for the proxy server.
$proxyCache The value true indicates that the proxy server retrieved the web page from the server, rather than from its own cache.
$proxyPort The port number of the proxy server to which the request was submitted.
$proxyServer The host name of the proxy server.
$proxyUsername Used by the monitor together with the password to log in to the proxy server.
$regexpMatchn The contents of the line matching the regular expression.
$regexpn The regular expression.
$regexpMatchn The contents of the line matching the regular expression.
$regexpStatusn The status of the regular expression match:
NONE - No regular expression checking is configured
MATCHED - A match was found for the regular expression
FAILED - A match wasn’t found for the regular expression
$responsetime*
(ResponseTime)
The time taken after a connection is created, until the first byte of the page is received.
$timeSinceModification The time that has elapsed since the page was last modified. This is the
difference between the time of the test and the value of the Last-Modified
HTTP header field of the first page retrieved.
$urlDownloadTimesn*
(UrlDownloadTime)
URL download time of each element in a GETALL request. Each element is numbered, starting with 000 ($urlDownloadTime000, $urlDownloadTime001, $urlDownloadTime002, and so on).
$urln*(Url) URL of each page in a GETALL test. Each page is numbered, starting with 000 ($url000, $url001, $url002, and so on).
$urlResultn*
(UrlResultString)
Result string for each page downloaded in a GETALL request. Each result is numbered, starting with 000 ($urlResult000, $urlResult001, $urlResult002,
and so on).
$username The name used to access pages that require the user to be authenticated.
Status message

The HTTP monitor provides status messages in the ResultMessage attribute when using IBM® Application Performance Management. These messages indicate the result of the test.

Table 4. Table 4. HTTP monitor status messages 
Message Description
OK The request from the monitor succeeded.
Form not found The requested page cannot be found.
Initialise Fetch Page Failed Not enough memory to allocate space for the HTTP page fetching mechanism. The previous line message should contain further information.
Connection Failed The monitor failed to connect for reasons other than the link is down, connection is reset, link is unreachable, connection timed out, connection terminated, or host is down. See the HTTP monitor log file for more information.
Form parameters and regular expression matching

Monitor the operation of the form http://support.mycompany.com/cgi-bin/search.cgi by sending HTTP POST requests with the form parameter search=ism, and use a regular expression to match the string Your search was successful in the response. If this string is returned in the response, classify the service level as Good, and Failed otherwise.

Create a new HTTP profile element and set the fields as shown in the following table.

Table 5. HTTP form profile element example
Profile element configuration field Value
server support.mycompany.com
page /cgi-bin/search.cgi
description Example - form parameters and regular expressions
Regular expression details
match 1 Your search was successful
Service level classification details
statement Regexp Status 1 = MATCHED then status GOOD
Head and Form details
name search
value ism
type FORM
Properties

The properties and command-line options specific to the HTTP monitor are described in the following table.

Table 6. HTTP monitor properties

Property name

Property parameter

Description

AllowDuplicateDownload 0|1
Forces pages to be downloaded each time they are found.
  • 0 - disabled (downloaded only once)
  • 1 - enabled

ForceHTMLParse

0|1
Forces pages that don’t have content-type text/html to be parsed as HTML.
  • 0 - disabled
  • 1 - enabled

GenerateProxyTokens

0|1
Specifies whether the monitor generates additional elements containing information about the proxy server if a proxy server is used in a test.
  • 0 - disabled
  • 1 - enabled (additional elements $server and $port contain values for the proxy server)

GETALLThreadNum

1|2|3|4|5

Specifies the number of separate threads to use during a GETALL request.

GetLinkTags

0|1
Activates download of linked stylesheets for GETALL requests:
  • 0 - disabled
  • 1 - enabled (if the target page contains a link tag with attribute value rel=stylesheet, the monitor attempts to download the resource referred to by the link tag's href attribute)

HostnameLookupPreference

string

Determines which IP version, IPv6 or IPv4, is applied to the supplied host name. The possible values are:
  • 4Then6 selects IPv4 and then IPv6. Uses IPv4 addresses if they are available. If no IPv4 addresses are found, IPv6 addresses are used.
  • 6Then4 selects IPv6 and then IPv4. Uses IPv6 addresses if they are available. If no IPv6 addresses are found, IPv4 addresses are used.
  • 4Only selects IPv4 only. Uses IPv4 addresses only. If there are no IPv4 addresses, the poll returns an error.
  • 6Only selects IPv6 only. Uses IPv6 addresses only. If there are no IPv6 addresses, the poll returns an error.
  • 6Or4 selects either IPv4 or IPv6. Uses the first address returned from the host name.

Default: 4Then6

Ipv6Address

integer

The local address to bind to as an origin for HTTP requests when using HTTP IPv6.

Default: no address

NoParseExtensions

string

A comma-separated list of file extensions indicating file types that the monitor won’t parse and instead only download.

OutputDirectory

string

Specifies the output directory to use if OutputResult is true (set to 1).

Default: $ISHOME/var

OutputResult

0|1
Specifies whether the monitor saves the data it receives from the service.
  • 0 - disabled
  • 1 - enabled

RelativeRedirects

0|1
Allows Location fields in HTTP 301 and HTTP 302 status codes to contain relative URLs instead of absolute URLs.
  • 0 - absolute URLs
  • 1 - relative URLs

RFCPOST

0|1
Specifies that the monitor should follow RFC1945 and RFC2616 and send a second POST after a redirect. Many web servers don’t expect a POST after a redirect and most browsers don’t follow the RFCs.
  • 0 - disabled
  • 1 - enabled