Integration

Kafka Monthly Digest: October 2018

Share this post:

What happened in the Kafka community in October 2018?

Kafka Summit SF 2018

The Kafka Summit SF 2018 went down on October 16 and 17 at the Pier 27, Embarcadero in San Francisco. The videos for the sessions are already available online.

We also know the dates for the next two Summits:

  • New York: April 2, 2019
  • London: May 13-14, 2019

Kafka Releases:

  • 2.1.0: The release process of 2.1.0 continued. The vote for RC0 has started, so if the community does not find any major issues, it should release in early November.
  • 2.0.1: On the October 16, Manikumar Reddy started the process for this bugfix release. It contains 49 fixes, including three blocker JIRAs and other small improvements. In order to see the full list of fixes, check the release plan on the Wiki. The vote for RC0 has also started.

KIPs:

Since last month, the community submitted 7 KIPs (KIP-379 to KIP-386). These are the ones that caught my eye:

KIP-379: Multiple Consumer Group Management
At the moment, managing consumer groups using the kafka-consumer-groups tool can be tedious because most of the actions (describe, delete, reset-offsets) can only be performed on a single group at a time. Therefore, the goal of this KIP is to allow this tool to handle several groups in a single command. A nice quality-of-life enhancement!

KIP-380: Detect outdated control requests and bounced brokers using broker generation
This KIP aims to fix several gaps in the inter-broker protocol. Many correctness issues happen because brokers can’t distinguish when another broker has bounced. Therefore, the proposed fix is to include the broker epoch in inter-broker request to allow brokers to reject requests intended to be sent to their previous generations.

KIP-382: MirrorMaker 2.0
Replicating data between Kafka clusters is a very common operation. However, MirrorMaker, the built-in tool in Apache Kafka, suffers from a number of issues and it is also lacking features. This KIP’s goal is to replace MirrorMaker with a new multi-cluster, cross-data center replication engine based on the Kafka Connect framework.

Blogs:


IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Software Engineer

More Integration stories
May 2, 2019

Seamless Integration: Istio and External Services

By defining our own MCP server, we allow users to move to the Istio service mesh without any code and deployment model changes. This means we can easily use Istio to control, observe, connect, and secure services running outside Kubernetes clusters.

Continue reading

April 30, 2019

Introducing IBM Analytics Engine v1.2 and Announcing the Deprecation of IBM Analytics Engine v1.0

We are excited to inform you about the new version of IBM Analytics Engine v1.2 that will be available starting May 15, 2019. Along with this release, Analytics Engine v1.0 will be retired.

Continue reading

April 26, 2019

Help Shape the Future of Cloud Foundry

Are you a Cloud Foundry user? If so, here's your opportunity to influence the future of Cloud Foundry with the 2019 user survey.

Continue reading