What happened in the Kafka community in October 2018?
Kafka Summit SF 2018
The Kafka Summit SF 2018 went down on October 16 and 17 at the Pier 27, Embarcadero in San Francisco. The videos for the sessions are already available online.
We also know the dates for the next two Summits:
-
New York: April 2, 2019
-
London: May 13-14, 2019
Kafka Releases:
-
2.1.0: The release process of 2.1.0 continued. The vote for RC0 has started, so if the community does not find any major issues, it should release in early November.
-
2.0.1: On the October 16, Manikumar Reddy started the process for this bugfix release. It contains 49 fixes, including three blocker JIRAs and other small improvements. In order to see the full list of fixes, check the release plan on the Wiki. The vote for RC0 has also started.
KIPs:
Since last month, the community submitted 7 KIPs (KIP-379 to KIP-386). These are the ones that caught my eye:
KIP-379: Multiple Consumer Group Management
At the moment, managing consumer groups using the kafka-consumer-groups tool can be tedious because most of the actions (describe, delete, reset-offsets) can only be performed on a single group at a time. Therefore, the goal of this KIP is to allow this tool to handle several groups in a single command. A nice quality-of-life enhancement!
KIP-380: Detect outdated control requests and bounced brokers using broker generation
This KIP aims to fix several gaps in the inter-broker protocol. Many correctness issues happen because brokers can’t distinguish when another broker has bounced. Therefore, the proposed fix is to include the broker epoch in inter-broker request to allow brokers to reject requests intended to be sent to their previous generations.
KIP-382: MirrorMaker 2.0
Replicating data between Kafka clusters is a very common operation. However, MirrorMaker, the built-in tool in Apache Kafka, suffers from a number of issues and it is also lacking features. This KIP’s goal is to replace MirrorMaker with a new multi-cluster, cross-data center replication engine based on the Kafka Connect framework.
Blogs:
IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.