In this 18th edition of the Kafka Monthly Digest, I will cover what happened in the Kafka Community in July.

For details about Kafka 2.3.0 that was released in June, see last month’s Community Digest.


No new releases this month.


Last month, the community submitted 14 KIPs (KIP-486 to KIP-499) and these are the ones that caught my eye:

KIP-487: Client-side Automatic Topic Creation on Producer: Historically, Kafka brokers have allowed auto topic creations. Now it is recommended to not rely on this feature and, instead, use the AdminClient API to create topics. This KIP aims to move the auto-creation further away from brokers and provide it in the Producer. If enabled, the Producer would automatically create topics that don’t exist.

KIP-488: Clean up Sum,Count,Total Metrics: Metrics are a critical part of monitoring Kafka but it’s key to be able to reason about them and clearly identify what they track. Unfortunately, at the moment, it’s a bit of a mess, as we have Counts, Totals, Sums, SampledTotals, and CumulativeSums, and the relations between them are complex. The goal of this KIP is to expose four clearly named types to remove ambiguity.

KIP-496: Administrative API to delete consumer offsetsKIP-211 was introduced in Kafka 2.1 to prevent consumer offsets from being deleted while a group is active. In some cases, it could be useful to be able to delete consumer offsets even if the group is still active. For example, if the subscription changed, offsets that are not meaningful anymore can pollute lag monitoring. This KIP proposes adding a new API “OffsetDelete” and expose it via the AdminClient to handle those use cases.

KIP-497: Add inter-broker API to alter ISR: The leader and in-sync replica (ISR) data is stored in Zookeeper in a ZNode per partition. At the moment, the Controller is responsible for updating the leader data, but it’s the leader that is responsible for updating the ISR data. This requires extra care to ensure updates are correctly protected. This KIP aims at fixing this issue by making the controller do both types of updates. A new API will be added (AlterISR) to allow leaders to send their ISR data to the controller.


Get started

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams


More from Announcements

IBM TechXchange underscores the importance of AI skilling and partner innovation

3 min read - Generative AI and large language models are poised to impact how we all access and use information. But as organizations race to adopt these new technologies for business, it requires a global ecosystem of partners with industry expertise to identify the right enterprise use-cases for AI and the technical skills to implement the technology. During TechXchange, IBM's premier technical learning event in Las Vegas last week, IBM Partner Plus members including our Strategic Partners, resellers, software vendors, distributors and service…

Introducing Inspiring Voices, a podcast exploring the impactful journeys of great leaders

< 1 min read - Learning about other people's careers, life challenges, and successes is a true source of inspiration that can impact our own ambitions as well as life and business choices in great ways. Brought to you by the Executive Search and Integration team at IBM, the Inspiring Voices podcast will showcase great leaders, taking you inside their personal stories about life, career choices and how to make an impact. In this first episode, host David Jones, Executive Search Lead at IBM, brings…

IBM watsonx Assistant and NICE CXone combine capabilities for a new chapter in CCaaS

5 min read - In an age of instant everything, ensuring a positive customer experience has become a top priority for enterprises. When one third of customers (32%) say they will walk away from a brand they love after just one bad experience (source: PWC), organizations are now applying massive investments to this experience, particularly with their live agents and contact centers.  For many enterprises, that investment includes modernizing their call centers by moving to cloud-based Contact Center as a Service (CCaaS) platforms. CCaaS solutions…

See what’s new in SingleStoreDB with IBM 8.0

3 min read - Despite decades of progress in database systems, builders have compromised on at least one of the following: speed, reliability, or ease. They have two options: one, they could get a document database that is fast and easy, but can’t be relied on for mission-critical transactional applications. Or two, they could rely on a cloud data warehouse that is easy to set up, but only allows lagging analytics. Even then, each solution lacks something, forcing builders to deploy other databases for…