What happened in the Kafka community in December 2018?
In this post, I will cover what happened in the Apache Kafka community in December as well as some of the milestones the project and community reached in 2018!
No new releases this month.
Last month, the community submitted 13 KIPs (KIP-399 to KIP-411). These are the ones that caught my eye:
KIP-402: Improve fairness in SocketServer processors Currently, SocketServer favors handling new connections over existing ones. Occasionally, like during a sudden connection storm, this can affect already active clients as the processing of their requests is delayed. This KIP proposes improving this behaviour by making Processors only handle a limited number of new connections before processing existing connections.
KIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api The goal of this KIP is to allow overriding some of the Connect runtime configurations in connectors configuration. The main use case is overriding client (producers for source connectors and consumers for sink connectors) configurations used by the Connect runtime.
KIP-409: Allow creating under-replicated topics and partitions At the moment, the controller will only create a topic if it is able to create all replicas immediately. This is a pessimistic approach, and, in a few scenarios, it would be valuable to allow deferring the creation of some replicas and still create the topic. For example, in many cases, at least “min.in.sync” replicas are enough. Also, this would allow topic creation when some brokers are unavailable (during a rolling restart) in small clusters.
Milestones that the project and community achieved in 2018
Releases in 2018:
The project followed its time-based release plan. Consequently, it released three major versions: 1.1.0, 2.0.0 and 2.1.0.
In addition, four bugfixes were also released: 0.10.2.2, 0.11.0.3, 1.1.1 and 2.0.1.
KIPs in 2018:
Since the KIP process was introduced in 2015, it has raised in popularity each year. In the past 12 months, the community raised over 160 KIPs, almost one ever other day, which was an all-time high!
Code and contributors
In the past 12 months, over 200 unique contributors made more than 1200 commits.
Committers and PMC:
In 2018, four very active contributors joined the Committers team:
Matthias J. Sax
Likewise, two Committers also joined the Apache Kafka PMC:
IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.