In this 21st edition of the Kafka Monthly Digest, I will cover what happened in the Kafka Community in October 2019.

For last month’s digest, see “Kafka Monthly Digest: September 2019.”


  • 2.3.1: This bugfix version was released on October 25, 2019 by David Arthur. It contains 46 fixes and improvements, including six blocker JIRAs. See the release notes for the full details.

There are still one bugfix release and one minor release in progress.

  • 2.2.2: After a couple of late bugs (KAFKA-9053, KAFKA-8649) related to Kafka Streams had been fixed, Randall Hauch prepared 2.2.2 RC2. The vote on this release candidate is currently ongoing. If no further issues are found, it should release in the next few days.
  • 2.4.0: The release process continued, and the code freeze happened on October 16, 2019. A few issues were found and there is currently one blocker left open (KAFKA-8677). The first release candidate should be available in the next few days.


Last month, the community submitted 17 KIPs (KIP-530 to KIP-546, 529 was skipped) and these are the ones that caught my eye:

KIP-531: Drop support for Scala 2.11 in Kafka 2.5: At the moment, Kafka builds for three Scala versions: 2.11, 2.12, and 2.13. The proposal is to stop building for 2.11 as this version has reached end of support. This will simplify the compatibility matrix and reduce development and tests costs. Finally, this will allow for taking advantage of more recent Scala features, especially around interoperability with Java 8 functional interfaces.

KIP-542: Partition Reassignment Throttling: Partition reassignments often cause significant load on a cluster. Since 0.10.1, quotas can be applied to control them, but these quotas also apply to existing replicas. The goal of this KIP is to allow for defining separate quotas for replication and reassignment. This will enable administrators to control reassignments without impacting regular replication.

KIP-543: Expand ConfigCommand’s non-ZK functionality: One of the pre-requirements to remove Zookeeper (KIP-500) is to update all tools to work without Zookeeper. At the moment, the kafka-configs tool still requires Zookeeper to update topic configurations and quotas. This KIP’s goal is to update this tool so it does not require on Zookeeper for any operations.

KIP-544: Make metrics exposed via JMX configurable: Kafka exposes a lot of metrics. In fact, in large clusters, monitoring agents can sometimes timeout while listing metrics. This KIP proposes adding configurations to the broker to blacklist or whitelist metrics. By also allowing to update these settings at runtime, administrators will be able to specify the metrics they want to omit, precisely.

KIP-545: support automated consumer offset sync across clusters in MM 2.0: MirrorMaker 2 added some tooling to translate consumer group offsets between clusters. This KIP aims to make failover between clusters even simpler by automatically populating the __consumer_offsets topics in downstream clusters. This will allow consumers to transparently find their new positions when switching clusters.


Get started

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams


More from Announcements

IBM TechXchange underscores the importance of AI skilling and partner innovation

3 min read - Generative AI and large language models are poised to impact how we all access and use information. But as organizations race to adopt these new technologies for business, it requires a global ecosystem of partners with industry expertise to identify the right enterprise use-cases for AI and the technical skills to implement the technology. During TechXchange, IBM's premier technical learning event in Las Vegas last week, IBM Partner Plus members including our Strategic Partners, resellers, software vendors, distributors and service…

Introducing Inspiring Voices, a podcast exploring the impactful journeys of great leaders

< 1 min read - Learning about other people's careers, life challenges, and successes is a true source of inspiration that can impact our own ambitions as well as life and business choices in great ways. Brought to you by the Executive Search and Integration team at IBM, the Inspiring Voices podcast will showcase great leaders, taking you inside their personal stories about life, career choices and how to make an impact. In this first episode, host David Jones, Executive Search Lead at IBM, brings…

IBM watsonx Assistant and NICE CXone combine capabilities for a new chapter in CCaaS

5 min read - In an age of instant everything, ensuring a positive customer experience has become a top priority for enterprises. When one third of customers (32%) say they will walk away from a brand they love after just one bad experience (source: PWC), organizations are now applying massive investments to this experience, particularly with their live agents and contact centers.  For many enterprises, that investment includes modernizing their call centers by moving to cloud-based Contact Center as a Service (CCaaS) platforms. CCaaS solutions…

See what’s new in SingleStoreDB with IBM 8.0

3 min read - Despite decades of progress in database systems, builders have compromised on at least one of the following: speed, reliability, or ease. They have two options: one, they could get a document database that is fast and easy, but can’t be relied on for mission-critical transactional applications. Or two, they could rely on a cloud data warehouse that is easy to set up, but only allows lagging analytics. Even then, each solution lacks something, forcing builders to deploy other databases for…