Kafka Monthly Digest: April 2019

By: Mickael Maison

What happened in the Kafka community in April 2019?

Releases:

The planning for version 2.3.0 started as Colin McCabe volunteered to run this release on April 13, 2019. The initial release date is June 12, 2019. The release plan is available on the wiki.

On the April 25, 2019, Vahid Hashemian volunteered to run the 2.2.1 bugfix release and its release plan is also available on the wiki.

KIPs:

Last month, the community submitted 17 KIPs (KIP-448 to KIP-464), and these are the ones that caught my eye:

KIP-449: Add connector contexts to Connect worker logs A Connect process consists of many threads that are each assigned a role (e.g., handle REST requests, run a connector task, etc). However, all the threads write to the same log, making it relatively hard to interpret. This KIP aims at making Connect logs more readable by prefixing each logs line with the name of the thread that emitted it.

KIP-455: Create an Administrative API for Replica Reassignment Currently, in order to reassign partitions, a user needs direct access to zookeeper. This comes with a number of drawbacks, such as lack of security, auditability, and obscure errors. As a result, the goal of this KIP is to provide support for this feature via the Kafka API and expose it in the AdminClient API.

KIP-460: Admin Leader Election RPC At the moment, users can trigger preferred leader elections via the Admin API. However, in some cases, it would be useful to be able to trigger an unclean leader election and, currently, this can require changing broker settings. This KIP proposes updating the current API to support unclean elections and expose this feature in the AdminClient.

KIP-464: Defaults for AdminClient#createTopic When using the AdminClient API to create topics, users have to specify the desired replication factor and number of partitions. In some cases, picking these values is not obvious for users, and relying on the broker's defaults (like for auto topic creation) instead would be better. This KIP proposes making the replication factor and partition count fields optional when creating a topic to make Kafka simpler to use and more accessible.

Blogs:

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Be the first to hear about news, product updates, and innovation from IBM Cloud