Open Source

Kafka Monthly Digest: February 2019

Share this post:

What happened in the Kafka community in February 2019?


– 2.2.0: On February 23, 2019, Matthias J. Sax cut the first Release Candidate but the community found a blocker JIRA that required a new RC. RC1 was cut on March 1, and the voting process is currently opened. As a result, this new minor release should be available early March.

– 2.1.1: On February 9, 2019, Colin McCabe released this bugfix version. It contains dozens of fixes including a 4 blocker JIRAs. See the release notes for the full list of changes.


Last month, the community submitted 13 KIPs (KIP-425 to KIP-437), and these are the ones that caught my eye:

KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers
Sending application logs to Kafka is a very common use case, and, in fact, Kafka includes a Log4J appender. However, currently, it does not support producing to brokers that use SASL, so this KIP proposes adding a couple of new properties to enable users to configure the appender to use SASL.

KIP-429: Smooth Auto-Scaling for Kafka Streams
In the past few months, the community started looking at ways to improve the rebalance mechanism used by clients to distribute work. One potential option is to use incremental cooperative rebalances. This KIP aims at bringing such a mechanism to Kafka Streams in order to support easy scaling without downtime.

KIP-430 – Return Authorized Operations in Describe Responses
At the moment, users have no way to determine which operations they can do on each Kafka resource. Basically, the only way to figure it out is to try the operation and see that it succeeds or if the response contains an authorization error. The goal of this KIP is to include authorization details in the Metadata and DescribeGroups responses so allowed operations are retrievable through the API.

KIP-435: Incremental Partition Reassignment
Currently, when a replicated partition is reassigned, all new replicas will have to be in-sync before any of the old replicas are deleted. As a result, this can cause significant load when reassigning a partition with three replicas to three different brokers as, at worst, Kafka will have to maintain six replicas. This KIP aims at optimizing the reassignment logic to decrease the impact of large reassignments and to delete replicas incrementally as new replicas come online.



IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Software Engineer

More Open Source stories
April 30, 2019

Introducing IBM Analytics Engine v1.2 and Announcing the Deprecation of IBM Analytics Engine v1.0

We are excited to inform you about the new version of IBM Analytics Engine v1.2 that will be available starting May 15, 2019. Along with this release, Analytics Engine v1.0 will be retired.

Continue reading

April 23, 2019

Announcing the Deprecation of the Watson Machine Learning JSON Token Authentication Service

We’d like to inform you about the deprecation of the Watson Machine Learning JSON Token Authentication service. This method of authentication will be retired on May 30, 2019.

Continue reading

April 19, 2019

Introducing IBM Cloud Object Storage Firewall: Further Secure Your Data

IBM Cloud Object Storage (COS) is giving you more control over who can access your data. We have introduced a new capability allowing you to configure your buckets with trusted IP address(es) that will dictate access to the data in COS.

Continue reading