Redis
The Redis destination writes data to Redis. For information about supported versions, see Supported Systems and Versions.
When you configure the destination, you specify the mode that the destination uses to process records. You configure properties to connect to Redis, including the URI of the Redis server. You can also use a connection to configure the destination.
In batch mode, the Redis destination can use CRUD operations defined in the sdc.operation.type record header attribute to process data. When CRUD operations are not specified in a record, the destination treats it like an Upsert record. For information about Data Collector change data processing and a list of CDC-enabled origins, see Processing Changed Data.
Mode
The Redis destination writes data to Redis using one of the following modes:
- Batch mode
- In batch mode, the destination writes data to Redis key-value pairs. You configure each key-value pair by selecting the incoming fields to use as the key and the value. You also select the data type of the Redis value. You can configure the destination to write to multiple key-value pairs.
- Publish mode
- In publish mode, the destination publishes data as messages to a Redis channel. You specify the channel to use and the data format of the messages. The publish mode pushes each record as one message to the specified Redis channel.
Data Types for Batch Mode
When you configure the destination for batch mode, you select the incoming fields to use as the Redis key and value. You also select the data type of the Redis value. If needed, the Redis destination converts the Data Collector data type of the incoming value field to the selected Redis data type.
When appropriate, use a Field Type Converter processor earlier in the pipeline to convert data types.
The following table lists the Data Collector data types that can be converted to Redis data types:
Data Collector Data Type | Redis Data Type |
---|---|
String | String |
List | List or Set |
Map | Hash |
Data Formats for Publish Mode
- Avro
- The stage writes records based on the Avro schema. You can use one of the following methods to specify the location of the Avro schema definition:
- Binary
- The stage writes binary data to a single field in the record.
- Delimited
- The destination writes records as delimited data. When you use this data format, the root field must be list or list-map.
- JSON
- The destination writes records as JSON data. You can use one of
the following formats:
- Array - Each file includes a single array. In the array, each element is a JSON representation of each record.
- Multiple objects - Each file includes multiple JSON objects. Each object is a JSON representation of a record.
- Protobuf
- Writes one record in a message. Uses the user-defined message type and the definition of the message type in the descriptor file to generate the message.
- SDC Record
- The destination writes records in the SDC Record data format.
- Text
- The destination writes data from a single text field to the destination system. When you configure the stage, you select the field to use.
Define the CRUD Operation
When using the Redis destination in batch mode, you can use CRUD operations to write to Redis. To use CRUD operations, define the CRUD operation record header attribute for each record earlier in the pipeline. Records without the attribute defined are treated as Upsert: new records are written and existing records are updated.
- sdc.operation.type
- When defined, the Redis destination uses the CRUD operation in the
sdc.operation.type
record header attribute when writing to Redis. The destination supports the following values for thesdc.operation.type
attribute:- 1 for INSERT
- 2 for DELETE
- 3 for UPDATE
- 4 for UPSERT
Configuring a Redis Destination
Configure a Redis destination to write data to Redis.