Programming IBM Orchestration Pipelines

You can program in a pipeline by using a notebook, a CPDCTL (IBM Cloud Pak for Data Command Line Interface), or running Bash scripts in a pipeline.

Programming with the CPDCTL tool

CPDCTL is a command line interface (CLI) that you can use to manage the lifecycle of a model. You can manage settings and automate an end-to-end pipeline that trains a model, saves it, creates a deployment space, and deploys the model.

Installing CPDCTL

See CPDCTL. The folder also contains samples that you can use for learning how to build and run pipelines. Refer to samples to get started with some end-to-end example code.

Customizing hardware specifications

You might use CPDCTL to manage custom hardware specifications in a project:

  1. To list hardware specifications:
    cpdctl environment hardware-specification list [--space-id SPACE-ID] [--project-id PROJECT-ID] [--name NAME]
  2. To create a new custom hardware specification:
    cpdctl environment hardware-specification create --name='my hardware specification' \
--nodes='{"cpu": {"units": "2"}, "mem": {"size": "4Gi"}, "num_nodes": 1}' \
--project-id=exampleString

Programming with Bash scripts

Run Bash scripts in a pipeline to compute or process data as part of the flow.

Run Bash scripts can run:

  • Bash scripts (bash)
  • Korn shell scripts (ksh)

Programming with notebooks

You can use a notebook to run an end-to-end pipeline or to run parts of a pipeline, such as model training.

  • For details on creating notebooks and for links to sample notebooks, see Notebooks.
  • For details on running a notebook as a pipeline job, see Run notebook job.

Using the Python client

To install the library, use pip to install the latest package of ibm-orchestration-pipelines in your coding environment. For example, run the following code in your notebook environment or console.

! pip install ibm-orchestration-pipelines

Use the client documentation for syntax and descriptions for commands that access pipeline components. Currently, the old library is still in use. See Orchestration Pipelines Python client for more details.

Using the IBM Orchestration Pipelines APIs

Use the Orchestration Platform API for working with Orchestration Pipelines in a developer setting.

Learn more

Parent topic: Creating a pipeline