Integration

Kafka Monthly Digest: October 2018

Share this post:

What happened in the Kafka community in October 2018?

Kafka Summit SF 2018

The Kafka Summit SF 2018 went down on October 16 and 17 at the Pier 27, Embarcadero in San Francisco. The videos for the sessions are already available online.

We also know the dates for the next two Summits:

  • New York: April 2, 2019
  • London: May 13-14, 2019

Kafka Releases:

  • 2.1.0: The release process of 2.1.0 continued. The vote for RC0 has started, so if the community does not find any major issues, it should release in early November.
  • 2.0.1: On the October 16, Manikumar Reddy started the process for this bugfix release. It contains 49 fixes, including three blocker JIRAs and other small improvements. In order to see the full list of fixes, check the release plan on the Wiki. The vote for RC0 has also started.

KIPs:

Since last month, the community submitted 7 KIPs (KIP-379 to KIP-386). These are the ones that caught my eye:

KIP-379: Multiple Consumer Group Management
At the moment, managing consumer groups using the kafka-consumer-groups tool can be tedious because most of the actions (describe, delete, reset-offsets) can only be performed on a single group at a time. Therefore, the goal of this KIP is to allow this tool to handle several groups in a single command. A nice quality-of-life enhancement!

KIP-380: Detect outdated control requests and bounced brokers using broker generation
This KIP aims to fix several gaps in the inter-broker protocol. Many correctness issues happen because brokers can’t distinguish when another broker has bounced. Therefore, the proposed fix is to include the broker epoch in inter-broker request to allow brokers to reject requests intended to be sent to their previous generations.

KIP-382: MirrorMaker 2.0
Replicating data between Kafka clusters is a very common operation. However, MirrorMaker, the built-in tool in Apache Kafka, suffers from a number of issues and it is also lacking features. This KIP’s goal is to replace MirrorMaker with a new multi-cluster, cross-data center replication engine based on the Kafka Connect framework.

Blogs:


IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Software Engineer

More Integration stories
December 5, 2018

Kafka Monthly Digest: November 2018

Here's what happened in the Kafka community in November 2018.

Continue reading

October 11, 2018

IBM Aspera CLI for IBM Cloud Private

IBM Aspera is very excited to release our first deliverable to support IBM Cloud Private (ICP). ICP is built on open standards using Docker and Kubernetes. The Aspera CLI can be easily provisioned within ICP environments to provide fast and secure data transport into or out of private cloud environments.

Continue reading

October 4, 2018

Improve Digital Experiences with SES Networks and IBM

With new innovations, including AI and IoT, our global network connections can revolutionize engineering, manage operations, and engage citizens by leveraging our IBM Cloud Direct Link service and intelligent asset management solutions.

Continue reading