How payload size and transformation affect performance for the API provider
When data passes through a IBM® z/OS® Connect server, the server needs to transform each byte in the payload between JSON and the language structure. Therefore, the larger the JSON payload, the more CPU processing time is needed. Although this data transformation is zIIP eligible, unnecessary CPU usage might slow down your overall performance.
When you create a service archive (.sar) file, you can use the API toolkit to reduce the size of your JSON payload by excluding unused fields or renaming fields, to improve performance. These features are not available with the build toolkit.
If your payload contains arrays, you can optimize the JSON payload by using
DEPENDING ON (ODO) in COBOL, and
REFER in PL/I, or use array counters, or
both. This method can reduce the number of array elements in the JSON payload.
ODO (COBOL) and
REFER (PL/I) can also reduce the size of
the language structure, which might further improve z/OS
subsystem performance. For example, by reducing IMS message sizes.
Exclude unused fields
When creating IBM z/OS Connect services using the API toolkit, fields that are not needed for your service should be excluded from the JSON payload by clearing the checkbox for those fields in the Include column in the Service Interface Editor. This method reduces the size of the JSON payload and makes the schemas more accurate as they do not describe properties that are not relevant.Example 1
In this example, the request and response use the same copybook, but different fields. When the same copybook is used for both the request and the response, define separate request and response service interfaces if you plan to exclude or rename different fields in the request versus the response.
By default, all fields are included for both the request and the response. So for an HTTP PUT or POST, all fields in the request and response are subject to data transformation. For an HTTP GET, all fields in the response are subject to data transformation.
Transforming unnecessary fields in this JSON payload is inefficient and degrades performance. Therefore, clear the Include checkbox of fields that are not needed for the request, and similarly for the response, as shown in Figure 2. In this example, the request does not require all the response fields to be checked, as the COBOL application does not require these fields as input to the program. This reduces the number of bytes from 30,376 to 112, and reduces the JSON payload size from approximately 64,496 to 122.
To rename fields, use the Interface Rename feature of the API toolkit. Renaming these fields to names shorter than the default saves CPU processing cost by reducing the amount of JSON parsing or generation that is needed, and also provides field names that are more familiar to the JSON programmer.
This saving is especially noticeable when large arrays are used in the data structure, as shown in the Figure 6.
The size of the JSON payload can also be reduced by using array sizes determined by using the following techniques:
OCCURS DEPENDING ON (ODO)for COBOL copybooks, or
REFERfor PL/I includes.
- Array counters.
- ODO or REFER
- In Figure 7, CARMODEL_INFO_DATA has a maximum of 100 elements in the array. However, a user might want to request only the first 10 cars listed in a database, rather than have the maximum set of 100 cars returned. Therefore, by specifying 10 in the request, only 10 elements of the array will be included in the JSON response payload, instead of 100 elements, considerably reducing the size of the payload to be transformed.
- Array counters
- You can use array counters in two ways,
- To optimize the JSON payload for fixed-length arrays.
- To optimize the JSON payload for nested variable-length arrays (ODO or REFER).
- Array counters for a fixed-length or variable-length array.
- Consider the following snippet from a fixed array copybook
After you import this copybook into the API toolkit, depending on the COBOL (or PL/I) application, it might be possible to specify an array counter to limit the number of array elements.
15 CARMODEL-INFO-TABLE. 20 CARMODEL-INFO-DATA OCCURS 100 TIMES. 25 CARMODEL-1-REF PIC X(12). 25 CARMODEL-2-MODEL PIC X(12). 25 CARMODEL-3-COST PIC X(12). 25 CARMODEL-4-LEATHERSEATS PIC X(12). :: :: ::
In this example, the number of entries in the array field CARMODEL_INFO_DATA, which is an ODO/REFER array subject, is controlled by array counter REQ_MAX_NBR_OF_REC_REQUESTED. Specifying an array counter allows you to set a minimum number of entries so that the array appears variable-length to an API client.
- Array counters with nested ODO or REFER.
- Nested ODO or REFER statements can generate very large JSON payloads. The use of array counters
can reduce the size of these JSON payloads considerably, and improve overall performance.
The following example demonstrates the value of an array counter in the context of a nested ODO array.The field CARMODEL-NBR-OF-COMMENTS is used to maintain the count of the ODO array CARMODEL-COMMENTS by using the arrays counter support. This allows each CARMODEL-INFO-DATA to have a different number of comments in the request or response JSON within the range of 1 to (
REQ-MAX-NBR-OF-COMMENTS <= 20) times.
05 REQ-NBR-OF-REC-RETURNED PIC 9(09) COMP. 05 REQ-MAX-NBR-OF-COMMENTS PIC 9(09) COMP. :: :: :: 15 CARMODEL-INFO-TABLE. 20 CARMODEL-INFO-DATA OCCURS 1 TO 100 TIMES DEPENDING ON REQ-NBR-OF-REC-RETURNED. 25 CARMODEL-NBR-OF-COMMENTS PIC 9(09) COMP. 25 CARMODEL-COMMENTS OCCURS 1 TO 20 TIMES DEPENDING ON REQ-MAX-NBR-OF-COMMENTS. 30 CARMODEL-COMMENT-A. 30 CARMODEL-COMMENT-B. 30 CARMODEL-COMMENT-C.
As the arrays are nested, the size of the JSON payload can become very large. Therefore, the use of array counters can reduce the size of the JSON payload, and improve performance as less transformation is needed.Figure 9 shows the two arrays, with the second ODO highlighted, and shows the array counter that is assigned to it.
For request service interfaces, ODO/REFER array objects are automatically set by IBM z/OS Connect so that the application receives only as many array entries in the ODO/REFER array subject as are present in the JSON. For nested ODO/REFER array subjects, respective ODO array objects are automatically set to the maximum number of entries across all occurrences of the nested array subject in the request JSON.
For response service interfaces, ODO/REFER array objects must be set by the application so that IBM z/OS Connect processes only the requested number of array entries in the respective ODO/REFER array subjects. For nested ODO/REFER array subjects, respective ODO array objects must be set by the application to the maximum number of entries across all occurrences of the array subject in the response data.
For more information about array counters, see Defining array counters.
You can use these techniques separately or concurrently to ensure that IBM z/OS Connect processes only the payload data that is relevant to your application and eliminate unnecessary transformation.