Runtime column propagation (DataStage)

You can define part of your schema to specify that extra columns be adopted and propagated through the rest of the job.

IBM DataStage uses Runtime column propagation (RCP) in situations where metadata isn't fully defined. When RCP is set, if your job encounters extra columns that are not defined in the metadata when it runs, it adopts these extra columns and propagates them through the rest of the job. That avoids errors due to missing mappings.

You cannot perform transformations on individual columns that were not explicitly defined in the DataStage flow. To transform a column, you must define it on the stage performing the transformation. Required columns must be defined on the stage that requires them. RCP propagates columns from source to target unchanged.

Runtime column propagation can be set in the Output tab for all connectors and stages.

RCP limitations

You cannot control the schema to the target stages. Job execution determines the final schema to the target stage.

Even when the RCP is enabled, for some stages, you must specify input columns that are used in the stage. In this case, specify only those columns that exist in the stage output link. For example, you must specify input columns for the following stages: Address Verification, Investigate, Lookup, Standardize, constraint condition, output derivation, stage variables, and loop variables. If the flow where those stages are used, has a file connector as a source, you must specify the precedence columns.

For target stages, such as Sequential File, you must specify the Null field value. Otherwise, the results might be rejected by the target stage even though run is successful.

If the Database contains Datatypes that are marked to the Large Object (LOB), you must set the Array Size to 1 on the database connector. Otherwise, run fails.