Examples of using REST stage in DataStage

In this document, you can find four examples of using REST stage in DataStage®:

Make a single call
Make calls to Next-link (pagination)
Make calls to offset (pagination)
Make calls to multiple requests

Few more examples are provided to help you quickly build and configure your API calls. These examples include demonstrations of how to make calls involving pagination, use variables, chain requests, and obtain authentication tokens. To use one, click Import example in the header of the REST stage editor and select an example. After import, set variable values and add output columns as needed to handle the data for your specific example. You can find there following examples:

Get projects
Get DataStage flows
Get DataStage flows by project ID
Get DataStage flows by project name
Get DataStage data assets by project name
Get DataStage jobs by project name

Example 1: Make a single call

Make a single call by using REST stage and Peek stage in DataStage :

Create or open a project. Drag the REST stage and Peek stage to your canvas. Link both stages by making REST stage as a source.
Double-click the REST stage. In Stage tab go to Requests tab, set the endpoint method to GET, and paste URL address to your chosen API in the frame.
Go to Settings tab under Requests and make sure to enable SSL and accept self-signed certificates.
Go to the Output tab. Click Add column twice.
Change the first column name to code. Under Derivation click Edit expression icon to open an expression builder. Under Built-in variables find and double-click code. You would see following expression:
```
RESPONSE_BODY["code"]
```
Click Apply and return. Change Data type to VARCHAR.
Change the second column name to json. Go to the expression builder, under Functions find, and double-click toJson(object). You would see following expression:
```
JSON.toJson(%object%)
```
Select %object%, under Built-in-variables find and double-click RESPONSE_BODY. Click Apply and return.
Click Compile and next click Run.

Example 2: Make calls to Next-link (pagination)

Make calls to Next-link by using REST stage and Peek stage in DataStage:

Create or open a project. Drag the REST stage and Peek stage to your canvas. Link both stages by making REST stage as a source.
Double-click the REST stage. In Stage tab go to Requests tab, set the endpoint method to GET, and paste URL address to your chosen API in the frame.
Go to Settings tab under Requests and make sure to enable SSL and accept self-signed certificates.
Go to Control tab and enable pagination. Check the Use expression and click the icon button to open expression builder. In Pagination templates double-click Next-link. You should see following expression:
```
ITERATION == 1 ? %first_url% : %next_url%
```
Select %first_url% and paste the URL. Select %next_url% and change it with RESPONSE_BODY.next. You should end with following expression:
```
ITERATION == 1 ? 'https://api.example.com/items' : RESPONSE_BODY.next
```
Click Apply and return. In Pagination condition provide following expression:
```
RESPONSE_BODY.next != null
```
Go to the Output tab. Click Add column twice.
Change the first column name to code. Under Derivation click Edit expression icon to open an expression builder. Under Built-in variables find and double-click code. You would see following expression:
```
RESPONSE_BODY["code"]
```
Click Apply and return. Change Data type to VARCHAR.
Change the second column name to json. Go to the expression builder, under Functions find, and double-click toJson(object). You would see following expression:
```
JSON.toJson(%object%)
```
Select %object%, under Built-in-variables find and double-click RESPONSE_BODY. Click Apply and return.
Click Compile and next click Run.

Example 3: Make calls to offset (pagination)

Make calls to offset by using REST stage and Peek stage in DataStage:

Create or open a project. Drag the REST stage and Peek stage to your canvas. Link both stages by making REST stage as a source.
Double-click the REST stage. In Stage tab go to Requests tab, set the endpoint method to GET, and paste URL address to your chosen API in the frame.
Go to Settings tab under Requests and make sure to enable SSL and accept self-signed certificates.
Go to Control tab and enable pagination. Check the Use expression and click the icon button to open expression builder. In Pagination templates double-click Offset. You should see following expression:
```
'%url%?limit=' + %limit% + '&offset=' + (ITERATION - 1) * %limit%
```
Select '%url%?limit=' and paste the API URL. Change the %limit% to the value of your choice. You should end with following expression:
```
'https://api.example.com/items' + 100 + '&offset=' + (ITERATION - 1) * 100
```
Click Apply and return. In Pagination condition provide following expression:
```
RESPONSE_BODY['resources'].size() == 100
```
Go to the Output tab. Click Add column twice.
Change the first column name to code. Under Derivation click Edit expression icon to open an expression builder. Under Built-in variables find and double-click code. You would see following expression:
```
RESPONSE_BODY["code"]
```
Click Apply and return. Change Data type to VARCHAR.
Change the second column name to json. Go to the expression builder, under Functions find, and double-click toJson(object). You would see following expression:
```
JSON.toJson(%object%)
```
Select %object%, under Built-in-variables find and double-click RESPONSE_BODY. Click Apply and return.
Click Compile and next click Run.

Example 4: Make calls to multiple requests

You can use REST stage to make API calls to multiple request with parameters support in DataStage. This example is build from three requests:

Create or open a project. Drag the REST stage and Peek stage to your canvas. Link both stages by making REST stage as a source.

From main project menu click Add parameters and Create parameter. In this example, you should add following parameters with chosen API URL as a default value:

Table 1. Used parameters
Name	Type	Default value (optional)
console_url	String	https://console.com
platform_url	String	https://platform.com
username	String	cpdadmin
password	Encrypted	password123
project_limit	Integer	30

Click Return to canvas.

Double-click the REST stage. In Stage tab go to Requests tab, set the endpoint method to POST, and enable the Use expression option. Open the expression builder, in which type:
```
console_url + '/idprovider/v1/auth/identitytoken'
```
Go to the Request tab > Custom headers and click Add header twice. Use following parameters:
Table 2. Headers

Key Default Value

Content-Type 'x-www-form-urlencoded

charset UTF-8
Switch tab to the Body and select raw option. Select application/x-www-form-urlencoded as Content type, Text as the source and in Text tab select the Use expression option. Under that click the button to open the expression builder. Type the following expression:
```
'grant-type=password&username-${username}&password-${password}&scope=openid'
```
Click Apply and return.
Go to Settings tab under Requests and make sure to enable SSL and accept self-signed certificates.
Go to the Control tab under Requests and make sure to disable the Write output record per iteration option.
Add another request by clicking the plus icon. Set the endpoint method to GET, and enable the Use expression option. Open the expression builder, in which type:
```
platform_url + '/v1/preauth/validateAuth'
```

Table 2. Headers
Key	Default Value
Content-Type	'x-www-form-urlencoded
charset	UTF-8

Go to the Request tab > Custom headers and click Add header twice. Use following parameters:

Table 3. Headers
Key	Derivation (by using expression builder)
username	Click Parameters and click `username`.
aim-token	Write the following expression: `RESPONSE_BODIES[0]['access_token']`.

Click Apply and return.

Go to the Settings tab under Requests and make sure to enable SSL and accept self-signed certificates.
Go to the Control tab under Requests and make sure to disable the Write output record per iteration option.
Add another request by clicking the plus icon. Set the endpoint method to GET, and enable the Use expression option. Open the expression builder, in which type:
```
platform_url + '/v2/projects?limit=' + project_limit + '&skip=' + (iteration - 1) * project_limit
```
Click Authentication tab and change authentication method to Bearer token. Open expression builder and fill it with following statement:
```
RESPONSE_BODIES[1]['accessToken']
```
Go to the Request tab > Custom headers and click Add header. Under Key type Content-Type, under Derivation type application/json.
Click Apply and return.
Go to the Settings tab under Requests and make sure to enable SSL and accept self-signed certificates.
Go to the Control tab under Requests and make sure to enable pagination and in expression builder type:
```
RESPONSE_BODIES[2]['resources'].size() == project_limit
```
Make sure, that in this request you have Write output record per iteration option enabled.
Go to the Output tab. Click Add column twice.
Change the first column name to code. Under Derivation click Edit expression icon to open an expression builder. Under Built-in variables find and double-click code. You would see following expression:
```
RESPONSE_STATUS["code"]
```
Click Apply and return. Change Data type to VARCHAR.
Change the second column name to json. Go to the expression builder, under Functions find, and double-click toJson(object). You would see following expression:
```
JSON.toJson(%object%)
```
Select %object%, under Built-in-variables find and double-click RESPONSE_BODIES. Click Apply and return.
Click Compile and next click Run.