Basic Concepts and Functions of CMS Pipelines

The operand of the PIPE command consists of at least one pipeline. A pipeline is a series of stages, separated by a character called a stage separator.

A stage reads data, processes the data, and writes the data to the next stage. The output from one stage is the input to the next stage. Each stage reads its input and writes its output through a device-independent interface without concern for other stages in the pipeline. Because data enters a pipeline and moves through a pipeline's stages, the data is said to flow through a pipeline. The flow of data records into a stage is called an input stream and the flow of data out of a stage is called an output stream.

CMS Pipelines includes an extensive set of stage commands that:
  • Issue CP and CMS commands
  • Read from or write to your terminal
  • Read, create, or append to a CMS file
  • Read a CMS file backwards
  • Copy records
  • Select a subset of records
  • Manipulate multiple records
  • Rearrange the contents of records
  • Sort records
  • Discard or retain duplicate records
  • Truncate records
  • Count lines, blank-delimited words, and bytes
  • Block data
  • Deblock external data formats.

Filters, device drivers, and host command interfaces are three types of stage commands. Filters transform data by performing operations such as sorting, truncating, and selecting records. Device drivers and host command interfaces read data from or write data to a device, CMS, CP, or other host environments (such as, DB2, ISPF, XEDIT). Another term for these types of stage commands is built-in, meaning their functions are provided by CMS Pipelines. A user-written stage command may be built with pipelines subcommands and Assembler macros when there is no built-in function available to perform the required task. Both built-in and user-written stage commands are invoked with the PIPE command.

To further extend the versatility of CMS Pipelines, an unlimited number of interconnected pipelines are supported. One PIPE command can specify multiple pipelines.