url-open, Kafka

Syntax for Kafka URLs with the dp:url-open extension element.

Syntax

A producer to write to a topic without a key
dpkafka://cluster?RequestTopic=topic[&TimeOut=seconds]
A producer to write to a topic with a text key
dpkafka://cluster?RequestTopic=topic&Key=textKey[&TimeOut=seconds]
A producer to write to a topic with a hex key
dpkafka://cluster?RequestTopic=topic&Key=hexEncodedKey&KeyType=binary[&TimeOut=seconds]
A consumer to read from a topic
dpkafka://cluster?ReplyTopic=topic[&TimeOut=seconds]
A consumer in a consumer group to read from a topic
dpkafka://cluster?ReplyTopic=topic[&ConsumerGroup=group][&TimeOut=seconds]

Attributes

dpkafka
Identifies the Kafka protocol identifier.
cluster
Specifies the name of an enabled Kafka cluster configuration. The configuration provides the information to access the remote Kafka server.
ReplyTopic=topic
Specifies the name of the topic where the consumer reads messages.
RequestTopic=topic
Specifies the name of the topic where the producer writes messages.
ConsumerGroup=group
Specifies the name of the consumer group.
Key=textKey
Key=hexEncodedKey&KeyType=binary
Specifies the message key for partitioning. Include the KeyType=binary parameter only when the message key is hex-encoded, not text.
TimeOut=seconds
Specifies the operational timer in seconds for the operation.

Guidelines

The dp:url-open element uses a Kafka cluster to read messages from a topic or to write messages to a topic. You can use this element as a single consumer or as a single producer.
  • A producer writes messages to a topic that are to be read by consumers.
  • A consumer reads messages from a topic that were written by producers.

You cannot use this element in both roles at the same time. If you need to use for both roles, you must use two separate calls. You might need to use this paradigm when the completion of an action by one service triggers a message to be written to another topic for use by another service.

Examples

  • The request topic of the test Kafka cluster to write the message.
    dpkafka://test?RequestTopic=request
  • The analytics topic of the test Kafka cluster to write the message with a hex-encoded key.
    dpkafka://test?RequestTopic=analytics&Key=616263096465&KeyType=binary
  • The reply topic of the test Kafka cluster to read the message.
    dpkafka://test?ReplyTopic=reply