Enable Bulkhead

Edit online

Bulkhead configuration allows you to specify the maximum number of concurrent requests processed by an API. You can configure this setting individually for an API or globally for all APIs.

When the number of concurrent requests to an API exceeds the specified limit, the excess requests are rejected. In such scenarios, the policy violation events are generated to report the violations occurred for an API. If you have 100 violations, then 100 policy violation events are generated.

As per the order of policies, the Bulkhead limit policy is applied first. That is, if you have applied multiple policies including the Bulkhead limit policy, then the Bulkhead limit policy is applied first.

Why do we configure bulkhead limit for APIs?

In an environment where multiple APIs are running, there are chances of one API making maximum use of the system resources due to the number of invocations. This in turn suppresses the performance of other APIs. Consider the following example. The maximum thread pool size of your system is 100, and it is shared among five APIs. When there are 100 invocations to one API, then that API uses up all the available threads to process its requests. The others APIs must wait till there are available threads.

bulkhead before graphics

To prevent this, you can specify the maximum number of concurrent requests that an API can process. This number depends on the maximum thread pool size of your system. Consider the same scenario after specifying the bulkhead limit for the APIs. In our example, the thread pool size is 100 and if you configure 20 as the maximum concurrent limit for each API, then each of the APIs can process a maximum of 20 requests, exceeding which the requests are rejected. This ensures optimal usage of the system resources.

bulkhead after graphics

As part of bulkhead configuration, you can specify the following details.

  • Bulkhead limit for an API at the API level. Specifies the maximum concurrent request limit for an API, exceeding which the requests are rejected. This number does not include the callbacks that an API receives. So, you can separately specify the maximum concurrent callbacks that an API can handle. The 503 Service unavailable error code is sent to the client service when the requests are rejected. You can configure the required error code and status phrase using the extended settings. You can customize the required status code and message using the extended settings pg.bulkhead.statusCode and pg.bulkhead.statusMessage respectively.
  • Bulkhead limit for all APIs (Global policy). Specifies the maximum concurrent request limit for all APIs. Similar to the API level configuration, you can provide the maximum concurrent callbacks and your choice to retry the rejected requests.

    When you have configured global-level bulkhead limit for APIs, and if you configure a different bulkhead limit at an API-level, then the limit configured at the global level takes precedence. To override this, you must exclude the required APIs from the global-policy using filters. You can apply the required API filters when creating or editing the bulkhead global policy. For information about creating global policies and applying API filters, see Creating a global policy.

  • Add Retry-After response header. Specifies whether the Retry-After header must be included in the response sent to the client when requests are rejected. If you select to add, you must also specify the duration (in seconds) that the system must wait before sending the consecutive requests. You can configure this setting to keep the client informed on the waiting duration before processing the consecutive requests.

Bulkhead feature considerations

Impact of the bulkhead policy on other features that you must remember when using the policy.

  • Applicable API types.

    Bulkhead policy is applicable for REST, SOAP, OData, and GraphQL APIs. However, you can specify the number of maximum concurrent callbacks only for the REST APIs.

  • Bulkhead limit configuration for APIs in a mashup.

    API-level bulkhead limit is not applied to the participating APIs. Instead, the bulkhead limit configured at the mashup API-level takes effect.

  • Bulkhead limit configured for APIs in cluster

    In cluster environments, you can specify the bulkhead limit in each node. webMethods API Gateway checks the number of concurrent requests per instance and allows or rejects the requests based on the bulkhead limit configured for an API in the corresponding node.

  • Overall thread pool violation

    When you enable the Bulkhead policy and if the overall thread pool is violated, then the requests exceeding the thread pool count are all rejected, and the Connection timeout message is displayed.

  • Conditional Routing policy

    When the Bulkhead policy is enabled with the Conditional Routing policy, then bulkhead limit configured at API-level for the participating APIs is applicable.

The table lists the properties that you can specify for the Enable bulkhead policy.

Field Description
Maximum concurrent calls Specify the maximum number of concurrent requests that the API can process, exceeding which the requests are rejected.
Enable bulkhead for callbacks Select this option to specify the maximum number of concurrent callbacks for the API. If you select this, the Maximum concurrent callbacks field appears.
Note: This setting is applicable only for REST APIs. Hence, this field is not displayed when you perform the bulkhead limit configuration for any other APIs.
Maximum concurrent callbacks Specify the maximum number of concurrent callbacks that the API can process, exceeding which the callbacks are rejected. This field appears only when you have selected the Enable bulkhead for callbacks check box.
Add Retry-After response header Select this option to include the Retry-after header in the response sent to the client when requests are rejected. This is to keep the client informed about the waiting duration before sending any consecutive requests.
Note: This option is to provide an approximate waiting duration before sending any consecutive requests; and it does not guarantee that the requests sent after the specified time are processed. The maximum number of concurrent requests that are processed is purely dependant on the configured bulkhead limit.
Retry after (value, in seconds) Specify the duration that the client must wait, after requests are rejected by webMethods API Gateway, to send consecutive requests. This field appears only when you have selected the Retry after (value, in seconds) check box.