Kafka sender transactional details

When you select the Commit of this message depends on parent transaction checkbox, the Kafka sender that is created internally is a transactional sender.

This transactional component differs from the regular sender in the following ways:

  • It is enabled to use Kafka transaction API.
  • As it is transactional, the internal Kafka producer instance is automatically configured with idempotence that is infinite retries, ack = ALL, and guarantees sending only once.
  • When a Kafka producer is created internally, a Kafka transaction is also started.
  • Any further calls to obtain Kafka producer in the same Order Management System transaction with the same configuration return the same instance and are part of the same Kafka transaction.
  • When an error is encountered, the Order Management System transaction is closed as part of the existing error handling mechanism. This action also rolls back the Order Management System transaction and the Kafka producer, which aborts any writes that are done in the current transaction.
  • When the Order Management System transaction is concluded, the Kafka transaction is also closed.
Note: In the Kafka sender in Service Definition Framework (SDF), transactional.id is an internally controlled value that is linked to the internal ID of the Order Management System transaction. Do not define this value as an end user.

If you are using a secure Kafka cluster and you receive the following error:

org.apache.kafka.common.errors.TransactionalIdAuthorizationException: Transactional Id authorization failed

To resolve this error, you can allow the user, Bob in this example, to use any transactional ID by using the wildcard character (*). Run the following example command or refer to the Kafka vendor documentation for setting ACLs in your Kafka vendor:

./kafka-acls.sh --bootstrap-server localhost:9092 --add --allow-principal User:Bob --operation Write --transactional-id "*"

The --transactional-id [transactional-id] parameter specifies the transactional ID to which ACLs are added or removed. A value of * indicates that the ACLs apply to all transactional IDs. For more information, see Authorization and ACLs.