August 7, 2019 By Mickael Maison 2 min read

In this 18th edition of the Kafka Monthly Digest, I will cover what happened in the Kafka Community in July.

For details about Kafka 2.3.0 that was released in June, see last month’s Community Digest.

Releases

No new releases this month.

KIPs

Last month, the community submitted 14 KIPs (KIP-486 to KIP-499) and these are the ones that caught my eye:

KIP-487: Client-side Automatic Topic Creation on Producer: Historically, Kafka brokers have allowed auto topic creations. Now it is recommended to not rely on this feature and, instead, use the AdminClient API to create topics. This KIP aims to move the auto-creation further away from brokers and provide it in the Producer. If enabled, the Producer would automatically create topics that don’t exist.

KIP-488: Clean up Sum,Count,Total Metrics: Metrics are a critical part of monitoring Kafka but it’s key to be able to reason about them and clearly identify what they track. Unfortunately, at the moment, it’s a bit of a mess, as we have Counts, Totals, Sums, SampledTotals, and CumulativeSums, and the relations between them are complex. The goal of this KIP is to expose four clearly named types to remove ambiguity.

KIP-496: Administrative API to delete consumer offsetsKIP-211 was introduced in Kafka 2.1 to prevent consumer offsets from being deleted while a group is active. In some cases, it could be useful to be able to delete consumer offsets even if the group is still active. For example, if the subscription changed, offsets that are not meaningful anymore can pollute lag monitoring. This KIP proposes adding a new API “OffsetDelete” and expose it via the AdminClient to handle those use cases.

KIP-497: Add inter-broker API to alter ISR: The leader and in-sync replica (ISR) data is stored in Zookeeper in a ZNode per partition. At the moment, the Controller is responsible for updating the leader data, but it’s the leader that is responsible for updating the ISR data. This requires extra care to ensure updates are correctly protected. This KIP aims at fixing this issue by making the controller do both types of updates. A new API will be added (AlterISR) to allow leaders to send their ISR data to the controller.

Blogs

Get started

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

More from Announcements

IBM and SAP unlock business and industry value with new generative AI solutions 

3 min read - IBM Consulting is delivering on our commitment to co-innovate with SAP and collaborate with our clients. As part of our Value Generation Partnership initiative announced earlier this month with SAP, we are releasing the first 10 of 100 planned AI solutions to help clients transform their industries, optimize their business processes and successfully deliver their SAP programs.  Delivering AI business and industry innovation at scale  With the recently announced Value Generation Partnership initiative, IBM and SAP are co-innovating intelligent industry…

IBM SevOne 7.0: Reaching application-centric multicloud network observability  

2 min read - As enterprises increasingly rely on network connectivity to support cloud-based applications and remote workers, network managers require new methods to monitor and safeguard connectivity across diverse environments, including corporate networks, software-defined WANs and multiple public cloud providers.   According to the recent EMA Network Megatrends Report, responding network professionals believe that 53% of network outages and performance issues could be prevented with improved network management tools, yet only 9% find it very easy to hire skilled networking personnel. This is why…

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity 

2 min read - To meet customer demands, applications are expected to be performing at their best at all times. Simultaneously, applications need to be flexible and cost effective, and therefore supported by an underlying infrastructure that is equally reliant, performant and secure as the applications themselves.   Easier said than done. According to EMA's 2024 Network Management Megatrends report only 42% of responding IT professionals would rate their network operations as successful.   In this era of hyper-distributed infrastructure where our users, apps, and data…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters