Configuring rate limits for IBM Master Data Management API operations

Configure rate limits for IBM Master Data Management API operations by modifying NGINX configuration files and updating the ZenExtension custom resource definition.

Roles and permissions: Cluster administrator To complete this task, you must be a cluster administrator with access to the Red Hat® OpenShift® Console, permission to execute commands in NGINX pods, and permission to modify custom resource definitions in OpenShift.

Important: By default, no rate limits are configured. If you want to implement rate limiting for the IBM Master Data Management API, complete the steps documented in this topic.

About this task

Rate limits control the number of API requests that can be processed within a specified time period. Setting rate limits enables you to manage the network traffic for your APIs and for specific operations within your APIs.

If rate limits are configured, IBM Master Data Management uses three rate limiting tiers:

Short (default, if configured): 100 requests per second. For most API operations.
Medium: 20 requests per second. For resource-intensive operations.
High: 6 requests per minute. For operations that require significant processing time.

When you configure rate limiting, most APIs are in the short tier by default. You can assign IBM Master Data Management API operations to the medium or high tier as needed, according to their resource requirements.

To set up API rate limiting:

Procedure

Log in to the OpenShift Console.
- Log in through the web UI by opening the OpenShift Console URL in a web browser.
- Log in by using the OpenShift CLI with the following command:
```
oc login <OPENSHIFT_API_URL> --username=<USERNAME> --password=<PASSWORD>
```
Identify the NGINX pods.
Run the following command to list the NGINX pods:
```
oc get pods | grep nginx
```
There are at least two NGINX pods: the main NGINX pod and the tester pod.
Access the shell of the main NGINX pod.
Run the following command:
```
oc exec -it <MAIN_NGINX_POD_NAME> -- /bin/bash
```
Open the /etc/nginx/nginx.conf file for editing.

Add the rate limit configuration to the http section of the nginx.conf file.

Insert the following configuration:

map_hash_bucket_size 128;

#
# Default/short rate limit for all APIs
#
# Uses tenant + HTTP method + URI as the key,
# so each tenant and API combination gets its own counter.
#

map "$request_method:$uri" $short_rate_limit {
    ~^[A-Z]+:/mdm/  $request_method:$uri;
    default         "";
}

limit_req_zone $short_rate_limit zone=mdm_short_limit:10m rate=100r/s;

#
# Medium rate limit map
#
# Only APIs configured here will use the medium limiter.
#

map "$request_method:$uri" $medium_rate_limit {

    "POST:/mdm/v1/probabilistic_search" "ps";
    "POST:/mdm/v1/data_entitlements/roles/batch_assign" "ba";
    "PUT:/mdm/v1/composite_service" "cs";
    "POST:/mdm/v1/ongoing_sync" "os";
    "POST:/mdm/v1/search" "sr";
    "POST:/mdm/v1/delete_linkage_rules" "dlr";
    "PUT:/mdm/v1/linkage_rules" "lr";
    "POST:/mdm/v1/data_quality/potential_matches/batch/tags" "pmt";
    "POST:/mdm/v1/data_quality/remediation_workflows" "rw";
    "POST:/mdm/v1/data_quality/remediation_workflows/[^/]+/resolve$ "rwr";
    "POST:/mdm/v1/configuration_export" "ce";
    "GET:/mdm/v1/pair_decisions/dynamic_metrics" "dm";
    "GET:/mdm/v1/pair_decisions/statistics" "st";
    "POST:/mdm/v1/suggest_data_mappings" "sdm";

    default "";
}

#
# High rate limit map
#
# Only APIs configured here will use the high limiter.
#

map "$request_method:$uri" $high_rate_limit {

    "POST:/mdm/v1/schema_update" "su";
    "PUT:/mdm/v1/instance_metadata" "im";
    "POST:/mdm/v1/configuration_metadata/processes" "cmp";
    "POST:/mdm/v1/weight_tuning_job" "wtj";
    "GET:/mdm/v1/pair_decisions/pair_review_histogram_data" "prh";
    "POST:/mdm/v1/snapshots" "ss";
    "PUT:/mdm/v1/snapshots/compare" "ssc";

    default "";
}

#
# Medium limit zone definition
#

limit_req_zone $medium_rate_limit zone=mdm_medium_limit:10m rate=20r/s;

#
# High limit zone definition
#

limit_req_zone $high_rate_limit zone=mdm_high_limit:10m rate=6r/m;

Save the changes to the nginx.conf file.
Reload the NGINX configuration.
Run the following command:
```
nginx -s reload
```
Exit the NGINX pod shell.
Repeat steps 3 through 8 for the tester NGINX pod.
Update the mdm-frontdoor-extension custom resource definition.
1. From the OpenShift Console, navigate to Administration > CustomResourceDefinitions.
2. Search for and select ZenExtension.
3. Locate the mdm-frontdoor-extension resource.
4. Add the following entries to all location blocks in the configuration:
```
limit_req zone=mdm_short_limit burst=10 nodelay;
limit_req zone=mdm_medium_limit burst=10 nodelay;
limit_req zone=mdm_high_limit burst=10 nodelay;
```
5. Save the changes to the custom resource definition.

Results

Rate limits are now configured for IBM Master Data Management API operations. The system enforces the following limits:

Short (100 requests per second)
Medium (20 requests per second)
High (6 requests per minute)