Important happenings in the Kafka community in September 2018


Kafka Summit SF 2018

The next Kafka Summit will happen on October 16 and 17 in San Francisco. The full schedule for all the sessions is already available on the Summit website, and there are still tickets available at the time I’m writing this article.

It will be my third time attending a Kafka Summit—come say Hi if you see me there!

Kafka releases:

On September 10, Dong Lin started the release process for 2.1.0. KIP freeze happened on October 24. The next step—code freeze—should happen mid-October, and, hopefully, there will be a release by the end of October or early November.

As always, the full release plan can be found on the Apache wiki.


In contrast to previous months, the community only submitted 8 KIPs in September (KIP-370 to KIP-378). These are the ones that caught my eye:

KIP-370: Remove Orphan Partitions
Reassigning partitions is a fairly common administrative operation on Kafka clusters. When a broker is down, an administrator can migrate partitions off this broker to ensure availability. In this case, upon restarting, the broker will not delete logs for partitions it’s not a replica for anymore. As a result, this can lead to wasted disk space and it is hard for administrators to track which brokers have “orphaned” logs. Therefore, the goal of this KIP is to enable brokers to automatically detect and delete such logs upon restarting.

KIP-372: Naming Repartition Topics for Joins and Grouping
Currently, Kafka Streams generates names of repartition topics. While this ensures names are unique, it also means that if the topology is updated, topic names can change. This prevents users from simply doing rolling restarts and can also make it hard to map topics to operator when debugging Streams applications. The goal of this KIP is to allow users to name operators (and by extension repartition topics) to allow rolling restart even in some case when the topology changed.

KIP-377: TopicCommand to use AdminClient
This KIP proposes updating the “kafka-topics” tool in order to allow it to use the AdminClient API. While the functionality will remain the same, it will enable users to run this tool in environments where zookeeper is not directly available (it is often the case in Cloud environments).


IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams


More from Announcements

IBM TechXchange underscores the importance of AI skilling and partner innovation

3 min read - Generative AI and large language models are poised to impact how we all access and use information. But as organizations race to adopt these new technologies for business, it requires a global ecosystem of partners with industry expertise to identify the right enterprise use-cases for AI and the technical skills to implement the technology. During TechXchange, IBM's premier technical learning event in Las Vegas last week, IBM Partner Plus members including our Strategic Partners, resellers, software vendors, distributors and service…

Introducing Inspiring Voices, a podcast exploring the impactful journeys of great leaders

< 1 min read - Learning about other people's careers, life challenges, and successes is a true source of inspiration that can impact our own ambitions as well as life and business choices in great ways. Brought to you by the Executive Search and Integration team at IBM, the Inspiring Voices podcast will showcase great leaders, taking you inside their personal stories about life, career choices and how to make an impact. In this first episode, host David Jones, Executive Search Lead at IBM, brings…

IBM watsonx Assistant and NICE CXone combine capabilities for a new chapter in CCaaS

5 min read - In an age of instant everything, ensuring a positive customer experience has become a top priority for enterprises. When one third of customers (32%) say they will walk away from a brand they love after just one bad experience (source: PWC), organizations are now applying massive investments to this experience, particularly with their live agents and contact centers.  For many enterprises, that investment includes modernizing their call centers by moving to cloud-based Contact Center as a Service (CCaaS) platforms. CCaaS solutions…

See what’s new in SingleStoreDB with IBM 8.0

3 min read - Despite decades of progress in database systems, builders have compromised on at least one of the following: speed, reliability, or ease. They have two options: one, they could get a document database that is fast and easy, but can’t be relied on for mission-critical transactional applications. Or two, they could rely on a cloud data warehouse that is easy to set up, but only allows lagging analytics. Even then, each solution lacks something, forcing builders to deploy other databases for…