Creating a service instance for Data Gate programmatically

After you install Data Gate, you must create at least one Data Gate service instance in the operands project. If you are a IBM® Software Hub user, you can use the /v3/service_instances REST API call to programmatically create service instances.

Who needs to complete this task?: To create a service instance programmatically by using the /v3/service_instances REST API call, you must have the Create service instances (can_provision) permission in IBM Software Hub.

When do you need to complete this task?

Complete this task only if you want to create a service instance programmatically by using the /v3/service_instances REST API call.

Alternative methods for creating a service instance

From the web client. For more information, see Creating a service instance for Data Gate from the web client.
By using the cpd-cli service-instance create command. For more information, see Creating a service instance for Data Gate with the cpd-cli service-instance create command.

Information you need to complete this task

Review the following information before you create a service instance for Data Gate:

Version requirements: All of the components that are associated with an instance of IBM Software Hub must be installed or created at the same release. For example, if Data Gate is installed at Version 5.4.0, you must create the service instance at Version 5.4.0.

Important: Data Gate uses a different version number from IBM Software Hub. This topic includes a table that shows the Data Gate version for each refresh of IBM Software Hub. Use this table to find the correct version based on the version of IBM Software Hub that is installed.

Environment variables

The commands in this task use environment variables so that you can run the commands exactly as written.

If you don't have the script that defines the environment variables, see Setting up installation environment variables.
To use the environment variables from the script, you must source the environment variables before you run the commands in this task. For example, run:
```
source ./cpd_vars.sh
```

Before you begin

This task assumes that the following prerequisites are met:

Prerequisite	Where to find more information
Data Gate is installed.	If this task is not complete, see Installing Data Gate.
Create a target database for the Data Gate instance: Create a Db2 database for transactional workloads. Create a Db2 Warehouse database for analytic workloads. Ensure that you have the display name and deployment ID of the database that you created. You will need this information to create the Data Gate service instance.	If this task is not complete, see the appropriate topic for the type of database that you need to create: Creating a service instance for Db2 Creating a service instance for Db2 Warehouse
Configure network access	If you (or your network administrator) have not yet configured network access between Db2 for z/OS and the Data Gate instance, follow the process specified in Configuring network access between Data Gate and IBM Z.
You generated an API key. The API key must be associated with a user who has the Create service instances (`can_provision`) permission in IBM Software Hub.	If this task is not complete, see Generating an API authorization token.
Activate Db2 Connect license on the source Db2 for z/OS subsystem	If you (or your database administrator) have not yet activated a valid Db2 Connect license on the Db2 for z/OS subsystem that acts as data source for the new Data Gate instance, follow the process specified in Activating the Db2 Connect Unlimited license on a Db2 for z/OS subsystem.

Procedure

Complete the following tasks to create a service instance:

Creating a service instance
Validating that the service instance was created
What to do next

Creating a service instance

To create a service instance:

Change to the directory on your workstation where you want to create the JSON file that defines the service instance payload.
Create a secret that the service instance can use to connect to the target database that you created:
1. Set the CPD_ROUTE environment variable:
```
export CPD_ROUTE=$(oc get route cpd -n ${PROJECT_CPD_INST_OPERANDS} -o jsonpath={".spec.host"})
```
  The command uses the PROJECT_CPD_INST_OPERANDS variable, which is already defined in your installation environment variables script.
2. Set the API_KEY environment variable to the API key that you created:
```
export API_KEY=<your_api_key>
```
3. Set the DATABASE_ID environment variable to the deployment ID of the database that you created for this service instance:
```
export DATABASE_ID=<database-id>
```
  Run the following command to get a list of the databases that are deployed on this instance of IBM Software Hub:
```
oc get db2u -n ${PROJECT_CPD_INST_OPERANDS}
```
  Use the value from the NAME column.
  
  Db2 database
  
  The ID has the format db2oltp-<unique-id>.
  
  Db2 Warehouse database
  
  The ID has the format db2wh-<unique-id>.
4. Create the secret.
  The command that you run depends on whether the instance of IBM Software Hub where you want to create the service instance uses a self-signed certificate or a certificate signed by a trusted certificate authority.
  The instance uses a certificate signed by a trusted certificate authority
```
curl -X POST https://${CPD_ROUTE}/data-gate/api/${PROJECT_CPD_INST_OPERANDS}/${DATABASE_ID}/secret \
   -H "Authorization: ZenApiKey ${API_KEY}" \
   -H 'cache-control: no-cache' \
   -H 'content-type: application/json'
```
  The instance uses a self-signed certificate (default)
```
curl -k -X POST https://${CPD_ROUTE}/data-gate/api/${PROJECT_CPD_INST_OPERANDS}/${DATABASE_ID}/secret \
   -H "Authorization: ZenApiKey ${API_KEY}" \
   -H 'cache-control: no-cache' \
   -H 'content-type: application/json'
```
  - If the request was successful, the command returns the following response message:
```
{"result":"success","message":"create secret successfully"}
```
  - If the request was not successful, use the response to determine the reason.
5. Confirm that the secret was created:
```
oc get secret -n ${PROJECT_CPD_INST_OPERANDS} | grep dg-${DATABASE_ID}-adminusertoken
```
  If the secret exists, the command will return information about the secret.
Set the environment variables that are used to populate the JSON payload for the service instance:
1. Set the INSTANCE_NAME environment variable to the unique name that you want to use as the display name for the service instance:
```
export INSTANCE_NAME=<display-name>
```
  This name is displayed on the Instances page of the IBM Software Hub web client.
  
  The display name is a string and can contain only alphanumeric characters (a-z, A-Z, 0-9) and dashes (-). The name must start and end with alphanumeric characters.
2. Set the INSTANCE_VERSION environment variable to the version that corresponds to the version of IBM Software Hub on your cluster:
```
export INSTANCE_VERSION=<version>
```
  Use the following table to determine the appropriate value:
  
  IBM Software Hub version Service instance version
  
  5.4.0 15.4.0
3. Set the INSTANCE_CPU environment variable to the amount of CPU to allocate to the service instance:
```
export INSTANCE_CPU=<integer>
```
  Specify an integer between 2 and 32.
  
  Size the instance based on your workload. For more information about the number of CPU to allocate to the service instance, see the component scaling guidance PDF, which you can download from the IBM Entitled Registry.
4. To set and use the additional environment variables defined in the following script based on the value that you set for the INSTANCE_CPU environment variable, you must source the following script:
```
#!/bin/bash

# export INSTANCE_CPU=$1

case $INSTANCE_CPU in
    2)
        stunnel_cpu=0.3
        server_cpu=0.65
        apply_cpu=0.55
        api_cpu=0.3
        ui_cpu=0.1
        ;;
    3)
        stunnel_cpu=0.5
        server_cpu=0.95
        apply_cpu=0.95
        api_cpu=0.4
        ui_cpu=0.1
        ;;
    4)
        stunnel_cpu=0.7
        server_cpu=1.35
        apply_cpu=1.35
        api_cpu=0.4
        ui_cpu=0.1
        ;;
    5)
        stunnel_cpu=1.0
        server_cpu=1.65
        apply_cpu=1.65
        api_cpu=0.4
        ui_cpu=0.2
        ;;
    6)
        stunnel_cpu=1.2
        server_cpu=2.05
        apply_cpu=2.05
        api_cpu=0.4
        ui_cpu=0.2
        ;;
    7)
        stunnel_cpu=1.4
        server_cpu=2.45
        apply_cpu=2.45
        api_cpu=0.4
        ui_cpu=0.2
        ;;
    8)
        stunnel_cpu=1.6
        server_cpu=2.85
        apply_cpu=2.85
        api_cpu=0.4
        ui_cpu=0.2
        ;;
    9)
        stunnel_cpu=1.8
        server_cpu=3.25
        apply_cpu=3.25
        api_cpu=0.4
        ui_cpu=0.2
        ;;
    10)
        stunnel_cpu=2.0
        server_cpu=3.65
        apply_cpu=3.65
        api_cpu=0.4
        ui_cpu=0.2
        ;;
    *)
        stunnel_cpu=2.0
        server_cpu=`echo "scale=2;3.65+($INSTANCE_CPU-10)/2" | bc`
        apply_cpu=$server_cpu
        api_cpu=0.4
        ui_cpu=0.2
        ;;
esac

stunnel_cpu=`echo "${stunnel_cpu}*1000" | bc`
server_cpu=`echo "${server_cpu}*1000" | bc`
apply_cpu=`echo "${apply_cpu}*1000" | bc`
api_cpu=`echo "${api_cpu}*1000" | bc`
ui_cpu=`echo "${ui_cpu}*1000" | bc`

export STUNNEL_CPU=${stunnel_cpu%.*}
export SERVER_CPU=${server_cpu%.*}
export APPLY_CPU=${apply_cpu%.*}
export API_CPU=${api_cpu%.*}
export UI_CPU=${ui_cpu%.*}

# echo "STUNNEL_CPU: ${STUNNEL_CPU}"
# echo "SERVER_CPU: ${SERVER_CPU}"
# echo "APPLY_CPU: ${APPLY_CPU}"
# echo "API_CPU: ${API_CPU}"
# echo "UI_CPU: ${UI_CPU}"
```
5. Set the INSTANCE_MEMORY environment variable to the amount of memory to allocate to the service instance:
```
export INSTANCE_MEMORY=<integer>
```
  Specify a value between 13 Gi and 32 Gi. Specify the value as an integer. Omit the unit of measurement.
  
  Size the instance based on your workload. For more information about the amount of memory to allocate to the service instance, see the component scaling guidance PDF, which you can download from the IBM Entitled Registry.
6. Set the following environment variables based on the value that you set for the INSTANCE_MEMORY environment variable:
  1. Set the SERVER_MEMORY environment variable to the following equation:
```
export SERVER_MEMORY=$(bc <<< "scale=1; 3.5 + (${INSTANCE_MEMORY} - 13) / 2")
```
  2. Set the APPLY_MEMORY environment variable to the following equation:
```
export APPLY_MEMORY=$(bc <<< "scale=1; 7 + (${INSTANCE_MEMORY} - 13) / 2")
```
7. Set the DATABASE_NAME environment variable to the display name of the database that you created for this service instance:
```
export DATABASE_NAME="<display-name>"
```
  This name is displayed on the Databases page of the IBM Software Hub web client.
  
  The display name is a string and can contain alphanumeric characters (a-z, A-Z, 0-9), spaces ( ), dashes (-), underscores (_), and periods (.). Make sure that you surround the display name with quotation marks, as shown in the preceding export command.
8. Set the DATABASE_ENGN_SVC environment variable to the name of the engn-svc service that is associated with the database:
  Db2 database
```
export DATABASE_ENGN_SVC=c-${DATABASE_ID}-db2u-engn-svc
```
  Db2 Warehouse SMP database
```
export DATABASE_ENGN_SVC=c-${DATABASE_ID}-db2u-engn-svc
```
  Db2 Warehouse MPP database
```
export DATABASE_ENGN_SVC=c-${DATABASE_ID}-db2u-head-engn-svc
```
9. Set the DB_ENCODING environment variable.
  Db2 database
```
export DB_ENCODING=UNICODE
```
  Db2 Warehouse database
  Decide whether you want to use this instance of Data Gate for query acceleration. If you want to use this instance for query acceleration, set the encoding to EBCDIC. Otherwise, set the encoding to UNICODE.
  EBCDIC
  
  export DB_ENCODING=EBCDIC
  
  UNICODE
  
  export DB_ENCODING=UNICODE
10. Set the INSTANCE_ROUTE environment variable.
  1. Get the IBM Software Hub route:
```
oc get route cpd --namespace=${PROJECT_CPD_INST_OPERANDS}
```
    Assuming that you use the default route, the route has the format:
```
https://cpd-namespace.apps.OCP-default-domain
```
  2. In a text editor, remove the following elements from the IBM Software Hub route:
```
https://cpd-namespace.
```
    The remaining portion of the route has the following format:
```
apps.OCP-default-domain
```
  3. Set the ROUTE_SUBDOMAIN environment variable to the remaining portion of the route:
```
export ROUTE_SUBDOMAIN=<remaining-portion-of-the-route>
```
11. Set the ROUTE_PREFIX environment variable to a unique identifier for this Data Gate instance:
```
export ROUTE_PREFIX=<unique-identifier>
```
  The identifier can contain lowercase alphanumeric characters (a-z, 0-9) and dashes (-). The identifier must start and end with an alphanumeric character.
12. Set the ROUTE_PORT environment variable to the port that you opened to accept incoming communication from the remote Db2 for z/OS cluster:
```
export ROUTE_PORT=<port-number>
```
13. Set the NODE_AFFINITY environment variable based on whether you want the service instance to run on the same node as the target database:
  Run the service instance on the same node as the target database
```
export NODE_AFFINITY=true
```
  Run the service instance on any node
```
export NODE_AFFINITY=false
```
14. Specify the appropriate environment variables based on whether you want to provision new storage or use existing storage when you create the service instance:
  Provision new storage
  Set the INSTANCE_STORAGE environment variable to the amount of storage to allocate to the service instance:
```
export INSTANCE_STORAGE=<size>
```
  50 Gi is sufficient for most environments. Specify the value as an integer. Omit the unit of measurement.
  Use existing storage
  Set the INSTANCE_PVC_NAME environment variable to the name of the persistent volume claim that you want to use to allocate storage to the service instance:
```
export INSTANCE_PVC_NAME=<existing-pvc-name>
```
  The PVC should point to file storage.

IBM Software Hub version	Service instance version
5.4.0	15.4.0

Create the data-gate-instance.json payload file.

The command that you run depends on how you want to provision storage for the service instance

Use storage classes to automatically provision storage

cat << EOF > ./data-gate-instance.json
{
    "addon_type": "dg",
    "display_name": "${INSTANCE_NAME}",
    "namespace": "${PROJECT_CPD_INST_OPERANDS}",
    "addon_version": "${INSTANCE_VERSION}",
    "preExistingOwner": false,
    "create_arguments": {
        "resources" : {
            "cpu" : "${INSTANCE_CPU}",
            "memory" : "${INSTANCE_MEMORY}"
        },
        "metadata": {
            "arch" : "${IMAGE_ARCH}",
            "baseNamespace" : "${PROJECT_CPD_INST_OPERANDS}",
            "cpu" : "${INSTANCE_CPU}",
            "databaseEngineService" : "${DATABASE_ENGN_SVC}",
            "databaseSecureJdbcPort" : "50001",
            "dataGateInstanceName": "${INSTANCE_NAME}",
            "data_gate_database_name": "${DATABASE_NAME}",
            "defaultRouteSubdomain" : "${ROUTE_SUBDOMAIN}",
            "targetDbEncoding" : "${DB_ENCODING}",
            "images_stunnel_limit_cpu" : "${STUNNEL_CPU}m",
            "images_stunnel_request_cpu" : "${STUNNEL_CPU}m",
            "images_server_limit_cpu" : "${SERVER_CPU}m",
            "images_server_limit_memory" : "${SERVER_MEMORY}Gi",
            "images_server_request_cpu" : "${SERVER_CPU}m",
            "images_server_request_memory" : "${SERVER_MEMORY}Gi",
            "images_apply_limit_cpu" : "${APPLY_CPU}m",
            "images_apply_limit_memory" : "${APPLY_MEMORY}Gi",
            "images_apply_request_cpu" : "${APPLY_CPU}m",
            "images_apply_request_memory" : "${APPLY_MEMORY}Gi",
            "images_api_limit_cpu" : "${API_CPU}m",
            "images_api_request_cpu" : "${API_CPU}m",
            "images_ui_limit_cpu" : "${UI_CPU}m",
            "images_ui_request_cpu" : "${UI_CPU}m",
            "mem" : "${INSTANCE_MEMORY}",
            "name" : "IBM Data Gate",
            "route_data_gate_route_host_name" : "${ROUTE_PREFIX}.${ROUTE_SUBDOMAIN}",
            "route_data_gate_route_port" : "${ROUTE_PORT}", 
            "targetDbInstance" : "${DATABASE_ID}",
            "type" : "dg",
            "version" : "${INSTANCE_VERSION}",
            "pod_affinity" : ${NODE_AFFINITY},
            "storage_type" : "storageClass",
            "storage_storageclass_name":"${STG_CLASS_FILE}",
            "storage_storageclass_size":"${INSTANCE_STORAGE}Gi"
        },
    "transientFields":{}
    }
}
EOF

The following environment variables use the values that are already defined in your installation environment variables script:

${PROJECT_CPD_INST_OPERANDS}
${STG_CLASS_FILE}
${IMAGE_ARCH}

Use an existing persistent volume claim

cat << EOF > ./data-gate-instance.json
{
    "addon_type": "dg",
    "display_name": "${INSTANCE_NAME}",
    "namespace": "${PROJECT_CPD_INST_OPERANDS}",
    "addon_version": "${INSTANCE_VERSION}",
    "preExistingOwner": false,
    "create_arguments": {
        "resources" : {
            "cpu" : "${INSTANCE_CPU}",
            "memory" : "${INSTANCE_MEMORY}"
        },
        "metadata": {
            "arch" : "${IMAGE_ARCH}",
            "baseNamespace" : "${PROJECT_CPD_INST_OPERANDS}",
            "cpu" : "${INSTANCE_CPU}",
            "databaseEngineService" : "${DATABASE_ENGN_SVC}",
            "databaseSecureJdbcPort" : "50001",
            "dataGateInstanceName": "${INSTANCE_NAME}",
            "data_gate_database_name": "${DATABASE_NAME}",
            "defaultRouteSubdomain" : "${ROUTE_SUBDOMAIN}",
            "targetDbEncoding" : "${DB_ENCODING}",
            "images_stunnel_limit_cpu" : "${STUNNEL_CPU}m",
            "images_stunnel_request_cpu" : "${STUNNEL_CPU}m",
            "images_server_limit_cpu" : "${SERVER_CPU}m",
            "images_server_limit_memory" : "${SERVER_MEMORY}Gi",
            "images_server_request_cpu" : "${SERVER_CPU}m",
            "images_server_request_memory" : "${SERVER_MEMORY}Gi",
            "images_apply_limit_cpu" : "${APPLY_CPU}m",
            "images_apply_limit_memory" : "${APPLY_MEMORY}Gi",
            "images_apply_request_cpu" : "${APPLY_CPU}m",
            "images_apply_request_memory" : "${APPLY_MEMORY}Gi",
            "images_api_limit_cpu" : "${API_CPU}m",
            "images_api_request_cpu" : "${API_CPU}m",
            "images_ui_limit_cpu" : "${UI_CPU}m",
            "images_ui_request_cpu" : "${UI_CPU}m",
            "mem" : "${INSTANCE_MEMORY}",
            "name" : "IBM Data Gate",
            "route_data_gate_route_host_name" : "${ROUTE_PREFIX}.${ROUTE_SUBDOMAIN}",
            "route_data_gate_route_port" : "${ROUTE_PORT}",
            "targetDbInstance" : "${DATABASE_ID}",
            "type" : "dg",
            "version" : "${INSTANCE_VERSION}",
            "pod_affinity" : ${NODE_AFFINITY},
            "storage_type" : "existingClaim",
            "storage_existingClaim_name": "${INSTANCE_PVC_NAME}"
        },
    "transientFields":{}
    }
}
EOF

The following environment variables use the values that are already defined in your installation environment variables script:

${PROJECT_CPD_INST_OPERANDS}
${STG_CLASS_FILE}
${IMAGE_ARCH}

Set the PAYLOAD_FILE environment variable to the fully qualified name of the JSON payload file on your workstation:
```
export PAYLOAD_FILE=<fully-qualified-JSON-file-name>
```
Create the service instance from the payload file.
The command that you run depends on whether the instance of IBM Software Hub where you want to create the service instance uses a self-signed certificate or a certificate signed by a trusted certificate authority.
The instance uses a certificate signed by a trusted certificate authority
```
curl --request POST \
--url "https://${CPD_ROUTE}/zen-data/v3/service_instances" \
--header "Authorization: ZenApiKey ${API_KEY}" \
--header 'Content-Type: application/json' \
--data @${PAYLOAD_FILE}
```
The instance uses a self-signed certificate (default)
```
curl -k --request POST \
--url "https://${CPD_ROUTE}/zen-data/v3/service_instances" \
--header "Authorization: ZenApiKey ${API_KEY}" \
--header 'Content-Type: application/json' \
--data @${PAYLOAD_FILE}
```
If the request was successful, the command returns one of the following HTTP response codes:
- 200 - The request was successfully completed and the service instance was provisioned.
- 202 - The request was successfully submitted. The service instance is being provisioned.
If the request was not successful, use the HTTP response code to determine the reason.

Validating that the service instance was created

To validate that the service instance was created:

Set the INSTANCE_ID environment variable to the ID that was returned by the POST cURL command:
```
export INSTANCE_ID=<ID-from-response>
```
Get the status of the service instance.
The command that you run depends on whether the instance of IBM Software Hub where you want to create the service instance uses a self-signed certificate or a certificate signed by a trusted certificate authority.
The instance uses a certificate signed by a trusted certificate authority
```
curl --request GET \
  --url "https://${CPD_ROUTE}/zen-data/v3/service_instances/${INSTANCE_ID}" \
  --header "Authorization: ZenApiKey ${API_KEY}" \
  --header 'Content-Type: application/json'
```
The instance uses a self-signed certificate (default)
```
curl -k --request GET \
  --url "https://${CPD_ROUTE}/zen-data/v3/service_instances/${INSTANCE_ID}" \
  --header "Authorization: ZenApiKey ${API_KEY}" \
  --header 'Content-Type: application/json'
```
- If the request was successful, the command returns the following HTTP response code: 200
  Find the provision_status parameter in the JSON response.
  - If the value is PROVISIONED, the service instance was successfully created.
  - If the value is PROVISION_IN_PROGRESS, wait a few minutes and run the command again.
  - If the value is FAILED, review the pod logs for the zen-core-api and zen-watcher pods for possible causes.
- If the request was not successful, use the HTTP response code to determine the reason.

Result

The provisioning process makes changes to the selected target database (Db2 or Db2 Warehouse). See Changes to the target database.

What to do next

Create the pairing. For this purpose, you must complete steps 24 through 30 in Creating a service instance for Data Gate from the web client. To open your instance from the web client and continue, follow the instructions in Updating a source definition for Data Gate.
You must give users access to the service instance. For more information, see Managing user access to Data Gate.