Kafka Monthly Digest: August 2018
This post is wrap up of important things going on in the Kafka community in August 2018.
No new releases this month.
Kafka Improvement Proposals:
Once again, the community has been very active and has submitted 22 KIPs (KIP-348 to KIP-369) since last month. These are the ones that caught my eye:
KIP-349: Priorities for Source Topics It is common to have consumers subscribed to several topics, and this KIP proposes adding a mechanism to specify the priority of each topic. That way, consumers will consume topics with the highest priority first.
KIP-352: Distinguish URPs caused by reassignment Currently, when reassigning partitions, they are counted as under-replicated until the reassignment completes. This is inconvenient for administrators that often have alerts for under-replicated partitions and it can also mask replication issues. This KIP’s goal is to change this behavior and instead track partitions being reassigned under a new metric called “ReassignedCount.”
KIP-357: Add support to list ACLs per principal Using the kafka-acl tool, it is only possible to list all ACLs or to list them by resource. This KIP’s objective is to allow listing them for specific users as this is a really common use case for administrators.
KIP-368: Allow SASL Connections to Periodically Re-Authenticate At the moment, once an authenticated (SASL) session is established between a client and broker, it is never revalidated. This makes it hard for administrators to block a user as any active session can just stay connected. This KIP targets specifically the OAUTH SASL mechanism and aims at providing a method to re-authenticate clients periodically.
KIP-369: Alternative Partitioner to Support “Always Round-Robin” Selection Kafka Producers use a Partitioner to select the partition each message is sent to in case the user has not specified it explicitly. The default Partitioner has different behaviors depending if messages have keys or not. This KIP proposes adding a new Partitioner that always assigns partitions using round robin, even for messages with keys.