Samples for programming a pipeline (Pipelines)

See samples for programming a pipeline with Bash scripts, such as installing third party packages, and more.

Installing packages with Bash

You can install third party packages with the Run Bash script node. In the following example, you will learn to install a package (PSQL) to connect to an external database.

Note:

The image RH UBI 9 is used in Pipelines 4.8.3 or later, while RH UBI 8 is used in Pipelines 4.8.2 or earlier. You need to extract the psql package from Red-hat Package Manager.

Download postgresql*.rpm and libpq*.rpm from the proper repository for RH UBI:
```
dnf download postgresql
dnf download libpq
```

Extract the rpm file from postgresql:

rpm2cpio postgresql-*.....*_64.rpm | cpio -idmv

Verify that the psql file exists in extracted files:
```
find . -name psql -print
```
Your output should be: ./usr/bin/psql

Extract the rpm file from libpq:

rpm2cpio libpq-13.11-1.el8.x86_64.rpm | cpio -idmv

Verify that the libpq.so.private13-5 file exists in extracted files.
1. Run the following code:
```
find . -name libpq.so.private13-5* -print 
```
  Your output should be:
  ./usr/lib64/libpq.so.private13-5.13
  ./usr/lib64/libpq.so.private13-5
2. Rename ./usr/lib64/libpq.so.private13-5.13 to ./usr/lib64/libpq.so.private13-5 as it is a link to the latter.
Verify that the plibpq.so.5 file exists in extracted files.
1. Run the following code:
```
find . -name libpq.so.5* -print
```
  Your output should be:
  ./usr/lib64/libpq.so.5
  ./usr/lib64/libpq.so.5.13
2. Rename libpq.so.5.13 to libpq.so.5 as it is a link to the latter.
Copy files to the storage connected to the project. For example, if you have a project named TEST-PSQL with storage connection FVT to volume: cpd-instnace::FVT:
1. In the file browser, create a new folder psql and upload files psql and libpq.so.5.
2. Upload test.sql file with the following content:
```
\l 
\q
```
  {: .codeblock} If you do not have storage or storage connection, please see:

In the Run Bash script node, add the following

environment variables:

-- LD_LIBRARY_PATH = /mnts/fvt/psql
-- USER 
-- DB
-- PORT 
-- HOST_IP
-- PGPASSWORD

script code:

chmod a+x ${LD_LIBRARY_PATH}/psql 
${LD_LIBRARY_PATH}/psql -U ${USER} -h ${HOST_IP} -p ${PORT}  -d ${DB} -a -f ${LD_LIBRARY_PATH}/test.sql

Run the pipeline. Your output should be the following:

\l 
                                List of databases
Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
        |          |          |             |             | postgres=CTc/postgres
template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
        |          |          |             |             | postgres=CTc/postgres
(3 rows)

\q

Using CPDCTL

Refer to samples to get started with some end-to-end example code, including how to deploy and monitor pipelines.

Parent topic: Programming IBM Watson Pipelines