Enable Bulkhead

Edit online

Bulkhead configuration allows you to specify the maximum number of concurrent requests processed by an API. You can configure this setting individually for an API or globally for all APIs.

When the number of concurrent requests to an API exceeds the specified limit, the excess requests are rejected. In such scenarios, the policy violation events are generated to report the violations occurred for an API. If there are 100 violations, then 100 policy violation events are generated.

As per the order of policies, the Bulkhead limit policy is applied first. That is, if you have applied multiple policies including the Bulkhead limit policy, then the Bulkhead limit policy is applied first.

Why do we configure bulkhead limit for APIs?

Bulkhead scenario
In an environment where multiple APIs are running, there are chances of one API making maximum use of the system resources due to the number of invocations. This in turn suppresses the performance of other APIs. Consider the following example. The maximum thread pool size of your system is 100, and it is shared among five APIs. When there are 100 invocations to one API, then that API uses up all the available threads to process its requests. The others APIs must wait till there are available threads.
To prevent this, you can specify the maximum number of concurrent requests that an API can process. This number depends on the maximum thread pool size of your system. Consider the same scenario after specifying the bulkhead limit for the APIs. In our example, the thread pool size is 100 and if you configure 20 as the maximum concurrent limit for each API, then each of the APIs can process a maximum of 20 requests, exceeding which the requests are rejected.

This ensures optimal usage of the system resources.

As part of bulkhead configuration, you can specify
  • Bulkhead limit for an API at the API level. Specifies the maximum concurrent request limit for an API, exceeding which the requests are rejected. This number does not include the callbacks that an API receives. So, you can separately specify the maximum concurrent callbacks that an API can handle. The 503 Service unavailable error code is sent to the client service when the requests are rejected. You can configure the required error code and status phrase using the extended settings. You can customize the required status code and message using the extended settings pg.bulkhead.statusCode and pg.bulkhead.statusMessage respectively.
  • Bulkhead limit for all APIs (Global policy). Specifies the maximum concurrent request limit for all APIs. Similar to the API level configuration, you can provide the maximum concurrent callbacks and your choice to retry the rejected requests.

    When you have configured global-level bulkhead limit for APIs, and if you configure a different bulkhead limit at an API-level, then the limit configured at the global level takes precedence. To override this, you must exclude the required APIs from the global-policy using filters. You can apply the required API filters when creating or editing the bulkhead global policy. For information about creating global policies and applying API filters, see Creating a Global Policy.

  • Add Retry-After response header. Specifies whether the Retry-After header must be included in the response sent to the client when requests are rejected. If you select to add, you must also specify the duration (in seconds) that the system must wait before sending the consecutive requests. You can configure this setting to keep the client informed on the waiting duration before processing the consecutive requests.