Rate limiting

Web reverse proxies allow incoming requests to be limited based on a user-defined set of criteria and policy.

Rate limiting achieves the following protections:

Brute force attacks on sensitive information such as passwords or PINs
Denial of Service attacks on a server or the Web Reverse Proxy

Rate limiting is performed by taking the incoming request and identifying the parts of the request that makes it unique to a client. Information such as the IP address that made the connection, a session cookie, other header information or the URL and HTTP method that were used can all be included to identify a client. When duplicate requests come in they fill up a counter; when full causes the reverse proxy to return an error to the client or drop the incoming connection.

Rate limiting can be deployed with two types of end users in consideration. The malicious end user who tries to cause damage to the service or steal a credential with brute force. A valid user who needs to be stopped from making too many requests in order to stop service degradation or overloading a back end server. There are two approaches to handle a malicious user:

The first approach is to handle them in the most efficient manner by closing the connection without performing any further request processing.
The second approach is to mislead the malicious user by returning false information, making it unclear to the user, if the operation was successful or even processed.

However, for a valid end user who is intended to use the service, a page that provides more information is more suitable.

The web reverse proxy that performs the rate limiting might not be the perimeter device in a connection by a client, and thus the incoming IP is not from the end user. In this instance, the device that is at the network perimeter includes the client address information in a header such as X-Forwarded-For, which can then be included by the reverse proxy when identifying the request.

Rate limiting is configured by authoring a policy file. This policy file contains the following items:

Requests that the policy applies to. The request is identified by the HTTP method and URL.
The information in the request that identifies it. This includes; cookies, headers, query string parameters, the client IP address and credential attributes if they are available.
The threshold for how many identical requests a client can make before they need to be rate limited and how long a client remains rate limited.
How to respond when a client is rate limited.

Once a rate limiting policy is created, the reverse proxy configuration file must be updated to attach it to an instance. After the configuration file is attached to the instance, restart the instance. Once a policy is added to the reverse proxy configuration file and the instance is reset, any changes to the policy file is dynamically reloaded once they are deployed, without any additional restart of the reverse proxy.

Rate limiting can occur in two places during the reverse proxy processing flow. When the rate limiting policy does not include credential attributes as a value to use in the request then rate limiting occurs very early in processing a request. This includes authentication and authorization. When the credential attributes are configured, then the rate limiting occurs once the users session is verified. This is after HTTP transformations (when the transformation is not attached with POP) and authentication, but prior to authorization checks. If rate limiting is applied early in the processing and the URL is re-written, then rate limiting is not applied again.

On every request all rate limiting polices are applied, such that if a request satisfies multiple policies they are all applied until one results in a reaction.

Rate limiting policies can be layered. An example of applying multiple policies is how you can rate limit a user account on a device. This can be done with two rate limiting policies. The first policy limits a user/s ability to log in based on client IP, and the POST to the login form. Then you can use a second policy to rate limit the use of a PDS-SESSION-ID cookie, with the reaction of invoking the logout page. This way, when a user abuses the session, they are logged out and must log in again. With the first policy in place, logging in is also a limited operation.

Trace for rate limiting is enabled by using the trace string pdweb.http.ratelimit

Logging

The log configuration entry within the policy YAML file can be used to enable logging of reactions. If enabled, WebSEAL writes a message to the WebSEAL message log file on the first reaction since the last reset for the policy. The reaction data is formatted as JSON and includes details of the request, along with details of the criteria that caused the reaction to take place. The policy name is obtained from the name configuration entry within the policy YAML file. An example log message is provided in the following section.

2023-02-17-08:28:51.814+10:00I----- 0x38CF0475 webseald ERROR wwa http RateLimit.cpp 683 0x7fa9746f4700 A rate limiting policy has been triggered: {"policy":"my-policy","url":"/","method":"GET","criteria":{"headers":{"host":"www.ibm.com"}},"ip":"10.10.10.200"}

HTTP Response Headers

The add-response-headers configuration entry, within the [rate-limiting] stanza of the WebSEAL configuration file controls whether HTTP headers that containing rate limiting information is included in the HTTP response. If enabled, the following headers are added to each HTTP response that matches a configured rate limiting policy.

X-Rate-Limit-Policy: The name of the rate limiting policy that is closest to being exceeded. The policy name is obtained from the name configuration entry within the policy YAML file.
X-Rate-Limit-Remaining: The number of requests that are left for the rate limiting policy in the current rate-limit window.
X-Rate-Limit-Reset: The time (Coordinated Universal Time Epoch time) at which the rate limiting policy resets.