August 7, 2019 By Mickael Maison 2 min read

In this 18th edition of the Kafka Monthly Digest, I will cover what happened in the Kafka Community in July.

For details about Kafka 2.3.0 that was released in June, see last month’s Community Digest.

Releases

No new releases this month.

KIPs

Last month, the community submitted 14 KIPs (KIP-486 to KIP-499) and these are the ones that caught my eye:

KIP-487: Client-side Automatic Topic Creation on Producer: Historically, Kafka brokers have allowed auto topic creations. Now it is recommended to not rely on this feature and, instead, use the AdminClient API to create topics. This KIP aims to move the auto-creation further away from brokers and provide it in the Producer. If enabled, the Producer would automatically create topics that don’t exist.

KIP-488: Clean up Sum,Count,Total Metrics: Metrics are a critical part of monitoring Kafka but it’s key to be able to reason about them and clearly identify what they track. Unfortunately, at the moment, it’s a bit of a mess, as we have Counts, Totals, Sums, SampledTotals, and CumulativeSums, and the relations between them are complex. The goal of this KIP is to expose four clearly named types to remove ambiguity.

KIP-496: Administrative API to delete consumer offsetsKIP-211 was introduced in Kafka 2.1 to prevent consumer offsets from being deleted while a group is active. In some cases, it could be useful to be able to delete consumer offsets even if the group is still active. For example, if the subscription changed, offsets that are not meaningful anymore can pollute lag monitoring. This KIP proposes adding a new API “OffsetDelete” and expose it via the AdminClient to handle those use cases.

KIP-497: Add inter-broker API to alter ISR: The leader and in-sync replica (ISR) data is stored in Zookeeper in a ZNode per partition. At the moment, the Controller is responsible for updating the leader data, but it’s the leader that is responsible for updating the ISR data. This requires extra care to ensure updates are correctly protected. This KIP aims at fixing this issue by making the controller do both types of updates. A new API will be added (AlterISR) to allow leaders to send their ISR data to the controller.

Blogs

Get started

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

More from Announcements

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity 

2 min read - To meet customer demands, applications are expected to be performing at their best at all times. Simultaneously, applications need to be flexible and cost effective, and therefore supported by an underlying infrastructure that is equally reliant, performant and secure as the applications themselves.   Easier said than done. According to EMA's 2024 Network Management Megatrends report only 42% of responding IT professionals would rate their network operations as successful.   In this era of hyper-distributed infrastructure where our users, apps, and data…

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

3 min read - Security operations is getting more complex and inefficient with too many tools, too much data and simply too much to do. According to a study done by IBM, SOC team members are only able to handle half of the alerts that they should be reviewing in a typical workday. This potentially leads to missing the important alerts that are critical to an organization's security. Thus, choosing the right SIEM solution can be transformative for security teams, helping them manage alerts…

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power 

2 min read - As companies undergo digital transformation, they rely on APIs as the backbone for providing new services and customer experiences. While APIs can simplify application development and deliver integrated solutions, IT shops must have a robust solution to effectively manage and govern them to ensure that response times and costs are kept low for all applications. Many customers use Salesforce’s MuleSoft, named a leader by Gartner® in full lifecycle API management for seven consecutive times, to manage and secure APIs across…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters