Operator TupleToAvro

Specialized toolkits - release 4.3.1.0-prod20190605 > com.ibm.streamsx.avro 1.2.2 > com.ibm.streamsx.avro > TupleToAvro

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.ibm.streamsx.avro/op$com.ibm.streamsx.avro$TupleToAvro.svg

This operator converts Streams tuples into binary Avro messages. The input tuples can benested types with lists and tuples, but the attribute types must be mappable to the Avro primitive types. boolean, float32, float64, int32, int64, rstring and ustring are respectively mapped to Boolean, Float, Double, Integer, Long, String.

If parameter embedAvroSchema is false, the operator passes window punctuation marker transparently to the output port. If parameter embedAvroSchema is true, the operator generates window punctuation markers.

If the output message attribute is not found or has no blob type, the operator will fail.

This operator must not be used inside a consistent region.

Summary

Ports
This operator has 1 input port and 1 output port.
Windowing
This operator does not accept any windowing configurations.
Parameters
This operator supports 7 parameters.

Required: avroMessageSchemaFile

Optional: bytesPerMessage, embedAvroSchema, outputAvroMessage, submitOnPunct, timePerMessage, tuplesPerMessage

Metrics
This operator does not report any metrics.

Properties

Implementation
Java

Input Ports

Ports (0)

Port that ingests tuples.

Properties

Output Ports

Assignments
Java operators do not support output assignments.
Ports (0)

Port that produces Avro records.

Properties

Parameters

This operator supports 7 parameters.

Required: avroMessageSchemaFile

Optional: bytesPerMessage, embedAvroSchema, outputAvroMessage, submitOnPunct, timePerMessage, tuplesPerMessage

avroMessageSchemaFile

File that contains the Avro schema to serialize the Avro message(s).

Properties
bytesPerMessage

This parameter controls the minimum size in bytes that the Avro message block should be before it is submitted to the output port. Default is 0l. Only valid if Avro schema is embedded in the output.

Properties
embedAvroSchema

Embed the schema in the generated Avro message. When generating Avro messages that must be persisted to a file system, the schema is expected to be included in the file. If this parameter is set to true, incoming tuples are batched and a large binary object that contains the Avro schema and 1 or more messages is generated. Also, you must specify one of the parameters (submitOnPunct, bytesPerMessage, tuplesPerMessage, timePerMessage) that controls when Avro message block is submitted to the output port.After submitting the Avro message to the output port, a punctuation is generated so that the receiving operator can potentially create a new file.

Properties
outputAvroMessage

The ouput stream attribute which contains the output Avro message(s). This attribute must be of type blob. Default is the sole output attribute when the schema has one attribute otherwise avroMessage.

Properties
submitOnPunct

When set to true, the operator will submit the block of Avro messages what was built and generate a punctuation so that the receiving operator can potentially create a new file. Default is false. Only valid if Avro schema is embedded in the output.

Properties
timePerMessage

This parameter controls the maximum time in seconds before the Avro message block is submitted to the output port. Default is 0l. Only valid if Avro schema is embedded in the output.

Properties
tuplesPerMessage

This parameter controls the minimum number of tuples that the Avro message block should contain before it is submitted to the output port. Default is 0l. Only valid if Avro schema is embedded in the output.

Properties

Libraries

Operator class library
Library Path: ../../impl/lib/com.ibm.streamsx.avro.jar, ../../opt/downloaded/*