Using custom stages

See the two examples of how to build, deploy and run the custom stage in DataStage®.

"Hello world" operator

"Hello world" operator prints "Hello, World" or "hello, world" with a specified number of times, and copies its input stream to its output stream.

  • Usage: hello [-u uppercase] [-n times] < input > output

Example below prints "Hello, World" once in the log. 

osh "hello -u 2 -n 1 <input.txt>| output.txt"

Example below prints "hello, world" three times in the log.

osh "hello -u -2 -n 3 <input.txt>| output.txt"

To build your operator library hello.so, follow the information in the [README](src/HelloWorld/README.md). If you want to use the operator library directly (libs/hello.so), skip this step.

To run a .zip file:

  1. Copy the zips/customOpHelloWorld.zip file that is located in /opt/ibm/samples to your local environment.
  2. Go to your project on the IBM Cloud Pak® for Data cluster, then upload the .zip file by way of the DataStage designer canvas.
  3. After import completes, it will contain one custom stage and one DataStage parallel flow.

To run the .isx file:

  1. Copy the flows/customOpHelloWorld.isx file to your local environment.
  2. Go to your project on the IBM Cloud Pak for Data cluster, then upload the .isx file by way of the DataStage designer canvas.
  3. After import completes, it will contain one custom stage and one DataStage parallel flow.
  4. Go to the Custom Stage page, then upload the libs/hello.so file or use your built operator library. Click Replace.
  5. Compile and run the flow.

StringFields operator

String Fields operator for example transforms the first character of a string to upper case

  • Usage: string fields [-uppercase] < input > output
For the following input text: 
1 a 8 abc nm
2 b 7 def nb
example below prints:
1 A 8 Abc Nm
2 B 7 Def Nb
osh "stringfields -uppercase < [record(a:int32; b:ustring; c:int32; d:string; e:string[2])] input.txt >| output.txt"

To build your operator library stringfields.so, follow the information in the [README](src/StringFields/README.md). If you want to use the operator library directly (libs/stringfields.so), skip this step.

To run the .zip file:

  1. Copy the zips/customOpStringFields.zip file to your local environment.
  2. Go to your project on the IBM Cloud Pak for Data cluster, then upload the .zip file by way of the DataStage designer canvas.
  3. After import completes, it will contain one custom stage and one DataStage parallel flow.
  4. Compile and run the flow.sh-4.4$.