Processing a Complex Pipeline

During processing, labels must be defined on a stage before being used as a connector. The label on a stage is the definition or declaration of the label. When the label is later used by itself it is known as a connector.

A label is used to create multiple data streams for a stage. Data streams are numbered, starting with 1, and can go as high as 10 depending on the stage. When a stage is processed, the number 1, or primary, input stream is connected to the previous stage, if any, and the number 1, or primary, output stream is connected to the following stage, if any.

An end character placed before or after a stage prevents connection to the adjacent stage on the side the end character is located. For example, if the end character for the following pipeline fragment was defined with the % character, a connection does not occur between CONSOLE stage and the STEM stage. In this example, STEM acts as a first stage:

...
|CONSOLE
%
|STEM VARN.
...

When a connector is encountered later in the pipeline specification, a data stream is defined and then connected from the stage where the label was defined to the connector. The lowest stream number available is assigned to the data stream.

If the labeled stage has an output in a simple pipeline within a complex pipeline, the data stream will be an output from the stage defining the label and an input to the stage following the connector. If the labeled stage is an output to a stage in the pipeline specification, the data stream will be an input to the stage defining the label and an output to the stage preceding the connector.

It is possible for a connector to be neither first nor last, in which case, the connector defines both an input and an output for the labeled stage. It is also possible to use two connectors in a row. This usage connects the output of one labeled stage to the input of another.

In the following example, the secondary output of LOCATE is connected to the secondary input of FANIN:

PIPE (END ¬)
     | < SOMEMEM
     | COLOR YELLOW
     | RD: LOCATE /GREEN/
     | COLOR GREEN
     | BK: FANIN
     | CONSOLE ONLY
     ¬ RD:
     | BK:

The PIPE stages in this example are explained in detail in Pipeline Stages and Syntax. For now, understand that < reads data from a data set member called SOMEMEM, the COLOR stage changes the color of text presented on the CONSOLE, and FANIN collects data from multiple input streams and passes the data to a single output stream.

In this pipeline, all the records in the member SOMEMEM are read and given the color attribute YELLOW. Then, all records containing the word GREEN are colored green. Records containing the word GREEN flow through the pipeline directly to the FANIN stage and then to the console. Records that do not contain the word GREEN flow to the RD: connector from the LOCATE/GREEN/ stage, which defines the RD: label. Because the BK: connector follows the RD: label, the data flows from the BK: connector as input to the stage defining it (BK: FANIN).