Development stages

You can use several development stages to help develop and test flows.

Note: Do not use development stages in production flows.

You can use the following stages when developing or testing flows:

Dev Data Generator source: Generates records with the specified field names, field types, and field attributes. For the field type, you can select a data type, such as String and Integer, or a type of data, such as address information, phone numbers, and book titles.; You can use a Map or List-Map root field type. You can also specify header attributes to include with generated records.; The source can generate events to test event handling functionality. To generate event records, select the Produce Events property.; When generating events, the source uses the configured fields as the body of the event record and adds event record header attributes. You can also specify the event type with the Event Name property. For example, to create a no-more-data event, enter no-more-data for the event name. For more information about the event framework and event record header attributes, see Dataflow triggers overview.; The source can also generate multiple threads for testing for a multithreaded flow. To generate multiple threads, enter a number larger than 1 for the Number of Threads property. For more information about multithreaded flows, see Multithreaded flow overview.; The On Record Error property has no effect in this stage.
Dev Random Record Source source: Generates records with the configured number of Long fields. You can define a delay between batches and a maximum number of records to generate.; The On Record Error property has no effect in this stage.
Dev Raw Data Source source: Generates records based on user-supplied data. You can enter raw data, select the data format of the data, and then configure any format-related configuration options.; For example, you can enter a set of log data, select the log data format, and then define the log format and other log properties for the data.; The source can generate events to test event handling functionality. To generate event records, select the Produce Events property.
Dev Identity processor: Passes all records through to the next stage. Use as a placeholder in the flow. You can define required fields and preconditions, and configure stage error handling.
Dev Random Error processor: Generates error records so you can test flow error handling. You can configure the stage to discard records, define required fields and preconditions, and configure stage error handling.
Dev Record Creator processor: Generates two records for each record that enters the stage. You can define required fields and preconditions, and configure stage error handling.
To Event target: Generates events for testing event handling functionality. To generate events, select the Produce Events property.; The target generates an event record for each incoming record. It uses the incoming record as the body of an event record and adds event record header attributes. Note that any record header attributes in the incoming record might be lost or replaced.; For more information about the event framework and event record header attributes, see Dataflow triggers overview.