Creating a Streams application resource template

You can customize the resources that your IBM Streams applications use by creating custom application resource templates.

Before you begin

Ensure that the following task has been completed:

Install a Streams service. For more information, see Installing Streams.

About this task

An application resource template can be used to define information about the resource, such as the amount of memory or the number of CPUs. An application resource template corresponds to a Streams resource tag. When a job is submitted and tags are specified, the application resource templates are searched for a matching template:

If tags are specified and the matching application resource template is found, the application resource template for the specified tags is used.
If no tags are specified, the default application resource template (called application) is used.
If tags are specified and no matching application resource template is found, the job submission fails.

Caveats:

When you create a custom application resource template, Streams augments the application resource with attributes that are required by Streams, such as labels, environment variables, container specification, liveness detection, and lifecycle management.
If the resource template contains an attribute, such as an environment variable, that has a name that Streams uses, Streams uses the information in the Streams attribute when it creates the application resource.
If you use a custom application resource template and your application uses the geospatial toolkit, your custom template must define a shared memory volume that can grow as memory is needed. The default shared memory that is provided by Docker is not sufficient for the geospatial toolkit.

You can change an application resource template at any time. The change is applied when the application pods are restarted or when a new application is submitted.

Important: Use caution if you update an existing custom configuration map:

If an application is submitted or restarted and its resource tags specify a resource template that isn't found, the application submission or restart will fail.
You are responsible for managing the custom resource templates and any objects that they require. If an application resource template specifies a custom application image or other objects such as a persistent volume claim mount or a pull secret, those objects must exist before the associated resource is restarted or an application that uses the resource is submitted. If not, the resource restart or the application submission or restart will fail.

The following list shows some of the resource information that you can customize by using an application resource template:

images
image pull secrets
number of CPUs
memory size
persistence volumes
resource affinity

For more information about these and other settings, see the Kubernetes documentation. For example, for more information about resource limits and requests, see Assign CPU Resources to Containers and Pods.

Procedure

Tip: The IBMStreams/streams-resource-templates repository contains sample configuration maps and associated files for several scenarios that you can use to get started.

Create or update a configuration map with the entries for each of your application resource templates. You can configure more than one resource template in the same configuration map. The name of the resource template must match the name of the tag to be used when the job is submitted. If you don't specify a setting, the default settings for the resource are used.
Important:
- The name of the container must be controller:
```
examplepvc.yaml: |
. . .
   spec:      
     containers:
       - name: controller
```
- The configuration map must include the label streams-custom-resource-template: "true":
```
labels:
    streams-custom-resource-template: "true"
```
  This label ensures that when a Streams instance is provisioned or an existing Streams instance is edited, the configuration map name can be selected from the list of available configuration maps.
- If your application uses the geospatial toolkits, you must include the volume and volumeMount specifications:
```
. . .
    volumes:   
      - name: shm-volume
          emptyDir:
            medium: Memory
. . .
    
    containers:
      - name: controller
. . .
       volumeMounts:
         - mountPath: /dev/shm
              name: shm-volume
```
  Where shm-volume is the name that you assign to the shared memory volume.
The code snippets in this step come from the resource templates repository sample called CustomDefaultTagAndPVC. This sample also demonstrates:
- How to set different Kubernetes characteristics than what is provided by the default application resource template (for example, if your application requires more CPU or memory).
- How to configure a persistent volume for your application.
Apply the updated application configuration map by entering the following command:
```
oc apply -f example-resource-template-config
```
The application resource templates aren't applied to currently running application resources, but will be applied if an application resource is restarted.

What to do next

After you customize the Streams instance configuration map, you must provision a Streams instance that uses the configuration map. If the Streams instance is already provisioned, you can edit the instance to specify the new configuration map. If the new configuration map affects resources that are in use by jobs that are in progress, those jobs do not automatically use the updated resource template.

If your application resource template requires any Kubernetes resource objects, make sure these objects are created before you submit your streaming application.

If the YAML syntax in the configuration map is incorrect, job submissions will fail. For information about how to debug this problem, see Job fails when submitted with a custom application resource template (Streams).

Important: Make sure that your application references the custom application resources (that is, the resource tags) when the job is submitted. For more information about how to use Streams resource tags to customize the deployment of a job to the appropriate resource, see Streams resource tags.