Purpose of transforming data in a policy
In a transform, you specify how to alter the operational data in the stream so that the data is consumable at its target destination. The two categories of transform are splitter transforms and filter transforms.
Split and unsplit data
The Data Streamer accepts data in two formats, split and unsplit.
- Split data
- Split data is divided into records and is sent as an ordered list of individual text strings. Each individual text string represents an individual record from the data source.
- Unsplit data
- Unsplit data is sent as a single text string and is not divided into records.
Overview of transform categories
The following information provides a brief overview of each transform category and some tips about when to use a transform from that category. Each category includes one or more transforms, which are listed and described in Transform configuration.
- Splitter transforms
- Based on specified criteria, a splitter transform splits data that is received as one message
into multiple messages. For Generic z/OS® Job Output data
stream, Generic ZFS File data stream, and Generic VSAM Cluster data stream, you must add a CRLF or
FixedLength Splitter to stream the data to subscriber.
- Usage tips
-
- For splitting a single message into multiple messages, based on occurrences of carriage return (CR) or line feed (LF) characters, use the CRLF Splitter transform.
- For splitting data records that have a fixed record length into multiple messages, based on configuration values that you provide, use the FixedLength Splitter transform.
- For splitting data records into multiple messages based on the start pattern of a record, using the StartPattern Splitter.
- Filter transforms
- Based on specified criteria, a filter transform discards messages from the data stream.
- Usage tips
-
- For filtering messages in the data stream according to a regular expression (regex) pattern, which you can define, use the Regex Filter transform.
- For filtering messages in the data stream according to a specified schedule, which you can define, use the Time Filter transform.
For filtering SMF or IMS data: For information about how to filter System Management Facilities (SMF) or IBM® IMS data, see Filtering a System Data Engine data stream by using a data stream definition.