Samples for programming a pipeline (Pipelines)

See samples for programming a pipeline with Bash scripts, such as installing third party packages, and more.

Using CPDCTL

Refer to samples to get started with some end-to-end example code, including how to deploy and monitor pipelines.

Programming with Bash

There are various development tools you can use by running Bash scripts in pipelines.

Installing packages with Bash

You can install third party packages with the Run Bash script node. In the following example, you will learn to install a package (PSQL) to connect to an external database.

Note:

The image RH UBI 9 is used in Pipelines 4.8.3 or later, while RH UBI 8 is used in Pipelines 4.8.2 or earlier. You need to extract the psql package from Red-hat Package Manager.

Download postgresql*.rpm and libpq*.rpm from the proper repository for RH UBI:
```
dnf download postgresql
dnf download libpq
```

Extract the rpm file from postgresql:

rpm2cpio postgresql-*.....*_64.rpm | cpio -idmv

Verify that the psql file exists in extracted files:
```
find . -name psql -print
```
Your output should be: ./usr/bin/psql

Extract the rpm file from libpq:

rpm2cpio libpq-13.11-1.el8.x86_64.rpm | cpio -idmv

Verify that the libpq.so.private13-5 file exists in extracted files.
1. Run the following code:
```
find . -name libpq.so.private13-5* -print 
```
  Your output should be:
  ./usr/lib64/libpq.so.private13-5.13
  ./usr/lib64/libpq.so.private13-5
2. Rename ./usr/lib64/libpq.so.private13-5.13 to ./usr/lib64/libpq.so.private13-5 as it is a link to the latter.
Verify that the plibpq.so.5 file exists in extracted files.
1. Run the following code:
```
find . -name libpq.so.5* -print
```
  Your output should be:
  ./usr/lib64/libpq.so.5
  ./usr/lib64/libpq.so.5.13
2. Rename libpq.so.5.13 to libpq.so.5 as it is a link to the latter.
Copy files to the storage connected to the project. For example, if you have a project named TEST-PSQL with storage connection FVT to volume: cpd-instnace::FVT:
1. In the file browser, create a new folder psql and upload files psql and libpq.so.5.
2. Upload test.sql file with the following content:
```
\l 
\q
```
If you do not have storage or storage connection, see:
- Managing storage volumes
- Storage volume connection If you do not have storage or storage connection, please see:

In the Run Bash script node, add the following

environment variables:

-- LD_LIBRARY_PATH = /mnts/fvt/psql
-- USER 
-- DB
-- PORT 
-- HOST_IP
-- PGPASSWORD

script code:

chmod a+x ${LD_LIBRARY_PATH}/psql 
${LD_LIBRARY_PATH}/psql -U ${USER} -h ${HOST_IP} -p ${PORT}  -d ${DB} -a -f ${LD_LIBRARY_PATH}/test.sql

Run the pipeline. Your output should be the following:

\l 
                                List of databases
Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
        |          |          |             |             | postgres=CTc/postgres
template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
        |          |          |             |             | postgres=CTc/postgres
(3 rows)

\q

Using API samples

See some samples for using Pipelines API for cases such as:

List all pipelines in a project.
Finds a specific pipeline by ID.
Deletes a pipeline and its pipeline versions.
Lists all pipeline versions of a given pipeline.
Gets a pipeline version by pipeline version ID.
Deletes a pipeline version by pipeline version ID.
Upload a pipeline file and create a new pipeline
Upload a pipeline file and create a new pipeline version
Commit a pre-existing volatile default version to finished state.
Create a new pipeline from existing pipeline.

Parent topic: Programming IBM Orchestration Pipelines