Configuring global objects for Watson Pipelines

Defining a pipeline parameter

1. Create a pipeline parameter from the configuration panel for a node or from the toolbar for the Pipeline editor.
2. Enter a name and optional description. Note that the name must be lower snake case with lower-case letters, numbers, and underscores. For example, lower_snake_case_with_numbers_123 is a valid name. The name must begin with a letter. If the name does not comply, you will get a 404 error when trying to run the pipeline.
3. Assign a parameter type. Depending on the parameter type, you might need to provide more details or assign a default value. Note: types for parameter sets are limited to: String, Integer, Float.
4. Click Add to list to save the pipeline parameter.

Note: You can define an individual parameter, or create a parameter set. Parameters you define individually can also be added to a set.

Parameter types

Parameter types are categorized as:

• Basic: including data types to structure input to a pipeline or options for handling the creation of a duplicate space or asset.
• CPD Path: for selecting a project, catalog, space, or asset.
• Instance: for selecting a machine learning instance or a Cloud Object Storage instance
• Other: for specifying details such as creation mode or error policy.

Creating a parameter set as a project asset

You can create a parameter set as a reusable project asset to use in pipelines.

1. Open an existing project or create a project.
2. Click New Asset and select Parameter set from the available asset types.
3. Assign a name for the set, and specify the details for each parameter in the set, including:
   • Name for the parameter
   • Data type
   • Prompt
   • Default value
4. Optionally create value sets for the parameter. The value sets can be the different values for different contexts. For example, you can create a Test value set with values for a test environment, and a production set for production values.
5. When you have created all of the parameters, save the parameter set. It will then be available for use in pipelines created in that project.

Adding a parameter set for use in a pipeline

To add a parameter set from a project:

1. Click the Global objects icon and switch to the Parameter sets tab.
2. Click Add parameter set to add parameter sets from your project that you want to use in your pipeline.
3. You can add or remove parameter sets from the list. The parameter sets you specify for use in your pipeline will be available when you assign parameters as input in the pipeline.

Creating a parameter set from the parameters list in your pipeline

You can create a parameter set from the parameters list for your pipeline

1. Click the Global objects icon and open the Parameters list.
2. Select the parameters you want in the set, then click the Save as parameters icon.
3. Enter a name and optional description for the set.
4. Save to add the parameter set for use in your pipeline.

Using a parameter set in a pipeline

To use a parameter set:

1. From a node property sheet, choose Assign pipeline parameter as an input type.
2. A list displays all available parameters of the type required for that input. This includes individual parameters, and parameters defined as part of a set. Choose the parameter to assign. Note that the parameter set name precedes the name of the parameter. For example, Parameter_set_name.Parameter_name.
3. When you run the pipeline, you can select a value set for the corresponding value (if available), assign a value for the parameter, or accept the default value.

Usage notes for parameter sets

• You can use a parameter set in the expression builder using the format param_sets.<MyParamSet.BRParam>. Note: If a parameter set value contains an environment variable, you must use this syntax in the expression builder: param_sets.MyParamSet.["$ICU_DATA"].
• Use parameters from a parameter set to configure the Run DataStage job node, using the Parameter_set_name.Parameter_name syntax.

Attention: If you delete a parameter, ensure that you remove the references to the parameter from your job design. If you do not remove the references, your job might fail.

Editing a parameter set in a job

If you use a parameter set when you define a job, you can choose a value set to populate variables with the values in that set. Note that if you change and save the values, then edit the job and save changes, the parameter set values will reset to the defaults.

Defining a user variable

You can create user variables for use in your pipeline. Note that user variables, like parameters, are defined on the global level and are not specific to any node. The initial value for a user variable must be set when you define it and cannot be set dynamically as the result of any node output. Once you define a user variable, you can use the Set user variables node to update it with node output.

To create a user variable:

1. Create a variable from the configuration panel for an Update variable node or from the toolbar for the Pipeline editor.
2. Enter a name and optional description. Note that the name must be lower snake case with lower-case letters, numbers, and underscores. For example, lower_snake_case_with_numbers_123 is a valid name. The name must begin with a letter. If the name does not comply, you will get a 404 error when trying to run the pipeline.
3. Choose an input type:
   • Expression - use the expression builder to construct an expression that will run with the variable is called. For details, see Coding elements. You must also specify a data type and an optional default value for the expression.
   • Node output - use the output from a previous node in the pipeline. Specify the node and the output.
   • Pipeline parameter - use the content generated by the pipeline parameter you specify. The parameter can be a part of a parameter set.
4. Click Add to add the variable to the list. It is now available for use in a node.

Configuring the size limit for a user variable

4.6.4 and later

You can change the size of user variables in your Pipelines config map.

1. Open your config map by running the command kubectl edit -n <namespace> configmap watson-pipelines-config where namespace is your Cloud Pak for Data instance namespace.
2. Set the value for user_variables_size_limit with the value in Kubernetes quantity format.
3. Restart your Tekton pods by running the command kubectl -n <namespace> delete pod -l 'app=tekton-extensions-controller'.

Updating a variable

Unlike a pipeline parameter, you can update a variable during a flow.

1. Add an Update variable node to the canvas and open the parameters pane.
2. Select a variable to update.
3. Edit the variable to specify a new value.
4. Repeat to update other variables. Note that you can also add variables from this node.
   The output for this node is the definition of the new or updated variables.

Parent topic: Creating a pipeline