Integration

Kafka Monthly Digest: October 2018

Share this post:

What happened in the Kafka community in October 2018?

Kafka Summit SF 2018

The Kafka Summit SF 2018 went down on October 16 and 17 at the Pier 27, Embarcadero in San Francisco. The videos for the sessions are already available online.

We also know the dates for the next two Summits:

  • New York: April 2, 2019
  • London: May 13-14, 2019

Kafka Releases:

  • 2.1.0: The release process of 2.1.0 continued. The vote for RC0 has started, so if the community does not find any major issues, it should release in early November.
  • 2.0.1: On the October 16, Manikumar Reddy started the process for this bugfix release. It contains 49 fixes, including three blocker JIRAs and other small improvements. In order to see the full list of fixes, check the release plan on the Wiki. The vote for RC0 has also started.

KIPs:

Since last month, the community submitted 7 KIPs (KIP-379 to KIP-386). These are the ones that caught my eye:

KIP-379: Multiple Consumer Group Management
At the moment, managing consumer groups using the kafka-consumer-groups tool can be tedious because most of the actions (describe, delete, reset-offsets) can only be performed on a single group at a time. Therefore, the goal of this KIP is to allow this tool to handle several groups in a single command. A nice quality-of-life enhancement!

KIP-380: Detect outdated control requests and bounced brokers using broker generation
This KIP aims to fix several gaps in the inter-broker protocol. Many correctness issues happen because brokers can’t distinguish when another broker has bounced. Therefore, the proposed fix is to include the broker epoch in inter-broker request to allow brokers to reject requests intended to be sent to their previous generations.

KIP-382: MirrorMaker 2.0
Replicating data between Kafka clusters is a very common operation. However, MirrorMaker, the built-in tool in Apache Kafka, suffers from a number of issues and it is also lacking features. This KIP’s goal is to replace MirrorMaker with a new multi-cluster, cross-data center replication engine based on the Kafka Connect framework.

Blogs:


IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Software Engineer

More Integration stories
February 15, 2019

Liberty for Java Buildpack Now Contains Two Liberty Production Runtimes

The Liberty runtime is moving from a quarterly to a four-week release cycle. As a result, the buildpack will release new functions and fixes quicker. Users of the Liberty for Java buildpack will notice a few changes to the Liberty versions packaged with the buildpack.

Continue reading

February 13, 2019

IBM Cloudant Security Update: TLS 1.2+ and Service Endpoints

The IBM Cloudant team wants to share some very important updates that will bring enhanced security to the service. These updates will change the way you engage with Cloudant, so please review to ensure that your applications can take advantage and continue uninterrupted.

Continue reading

February 13, 2019

Introducing IBM Cloudant Partition Queries: Faster, Cheaper, and More Scalable

To meet our customer demands while improving the scalability and performance of Cloudant, we are excited to introduce partition queries—a scalable, faster, and cheaper way to query a Cloudant database.

Continue reading