Expression Evaluator
The Expression Evaluator performs calculations and writes the results to new or existing fields. You can also use the Expression Evaluator to add or modify record header attributes and field attributes.
To create an expression, you define the name of the field, record header attribute, or field attribute to receive the results of the expression. And then, use the IBM StreamSets expression language to define the expression to use.
You can use runtime parameters in expressions. Define a runtime parameter when you want to specify the value for a pipeline property when you start the pipeline.
You can also use the time:now()
function to include the Data Collector
server time in the record.
For more information about configuring expressions, see Expression Configuration.
Output Fields and Attributes
When you configure an expression, the Expression Evaluator writes the results of the expression to an output field or attribute. You can use an existing field or attribute, or create a new one.
When you use an existing field or attribute, the Expression Evaluator replaces the incoming value with the new value. When you use a new field or attribute, the Expression Evaluator adds it to the record and passes the results of the expression.
Record Header Attribute Expressions
You can use expressions to add or modify header attributes for a record.
For example, you might use Expression Evaluator to set the sdc.operation.type attribute to enable writing to MongoDB.
Some destinations can use record header attributes to perform record-based writes. For more information, see Record Header Attributes for Record-Based Writes. For general information about record header attributes, see Record Header Attributes.
Field Attribute Expressions
You can use expressions to add or modify field attributes for a record. For example, you can create a field attribute based on record data, then pass the record to a Stream Selector that routes data based on the attribute value.
You can also use field attribute functions in a field expression to include field attribute values in the record.
For more information about field attributes, see Field Attributes. For more information about field attribute functions, see Record Functions.
Configuring an Expression Evaluator Processor
Configure an Expression Evaluator to perform calculations on a record-by-record basis.
For more information about the IBM StreamSets expression language, see Expression Language. For more information about configuring expressions, see Expression Configuration.