Sort
The Sort processor sorts incoming data based on one or more specified fields. The processor can sort data in ascending or descending order.
For example, let's say that you create a batch pipeline to read all available data in the
orders
table in a relational database, transform the data, and then
write the data to a destination system. Before writing the data, you want the pipeline
to sort all records by the order ID. To do this, you add a Sort processor before the
destination, and configure the processor to sort by the order_id
field
in ascending order.
Sort by Multiple Fields
When you sort by multiple fields, the Sort processor sorts data according to the order of the listed fields on the Sort tab.
grade
last_name
first_name
You preview the pipeline with sample data. Preview displays the following input and output data for the Sort processor, showing how the record order has changed:
Notice how grade 2 students are listed in the last three records in the input data, but the processor reorders them as the first three records in the output data. The output data also shows how the processor additionally sorts the grade 2 students alphabetically by last name and then by first name.
Configuring a Sort Processor
Configure a Sort processor to sort incoming data based on specified fields.