Kafka Monthly Digest: January 2019

What happened in the Kafka community in January 2019?

Releases:

There were no new releases in January but Matthias J. Sax started the process for version 2.2.0. As always, the release plan is available on the wiki. The current target date for GA is the February 28, 2019.

At the same time, Colin McCabe started the process for bugfix release 2.1.1. Since 2.1.0, the community fixed over 30 issues, including three blocker JIRAs. Check out the wiki if you are interested in the full release plan.

KIPs:

Last month, the community submitted 13 KIPs (KIP-412 to KIP-424). These are the ones that caught my eye:

KIP-412: Extend Admin API to support dynamic application log levels
In order to quickly debug an issue with a Kafka cluster, it’s important to be able to retrieve logs of the affected components. While Kafka already allows you to change log levels at runtime, it’s currently only possible via JMX, and it is relatively hard to do (especially if you’ve never done it before and your cluster is burning!). The goal of this KIP is to provide an easy mechanism via the AdminClient API to dynamically adjust log levels.

KIP-415: Incremental cooperative rebalancing in Kafka Connect
Today, Kafka Connect is reusing the same group membership API as Consumers do. However, because Connector and Tasks can change configuration at runtime, this can lead to a lot of extra rebalances also interrupting other existing connectors. This KIP proposes introducing an Incremental Cooperative protocol to deal with those issues.

KIP-419: Safely notify Kafka Connect SourceTask is stopped
Currently, when Kafka Connect stops a Source Task, the task does not know whether it will restart shortly or if it’s a proper shutdown. Also, in some cases, the Connect Runtime can even call stop() multiple times. This KIP aims at improving the lifecycle of Source Connectors by providing a new method to clearly identify when the Runtime stopped a task for good so it can safely release its resources.

KIP-422: Add support for user/client configuration in the Kafka Admin Client
Kafka allows you to configure four types of entities: brokers, topics, users, and client-ids. Unfortunately, at the moment, the Admin API only supports brokers and topics. In order to configure the other two entity types, users have to run the kafka-configs.sh command line tool and require direct Zookeeper access. The KIP proposes updating the Admin APIs to handle all entity types so the AdminClient supports all entities.

Blogs:

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Mickael Maison

Software Engineer

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity

2 min read - To meet customer demands, applications are expected to be performing at their best at all times. Simultaneously, applications need to be flexible and cost effective, and therefore supported by an underlying infrastructure that is equally reliant, performant and secure as the applications themselves. Easier said than done. According to EMA's 2024 Network Management Megatrends report only 42% of responding IT professionals would rate their network operations as successful. In this era of hyper-distributed infrastructure where our users, apps, and data…

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

3 min read - Security operations is getting more complex and inefficient with too many tools, too much data and simply too much to do. According to a study done by IBM, SOC team members are only able to handle half of the alerts that they should be reviewing in a typical workday. This potentially leads to missing the important alerts that are critical to an organization's security. Thus, choosing the right SIEM solution can be transformative for security teams, helping them manage alerts…

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power

2 min read - As companies undergo digital transformation, they rely on APIs as the backbone for providing new services and customer experiences. While APIs can simplify application development and deliver integrated solutions, IT shops must have a robust solution to effectively manage and govern them to ensure that response times and costs are kept low for all applications. Many customers use Salesforce’s MuleSoft, named a leader by Gartner® in full lifecycle API management for seven consecutive times, to manage and secure APIs across…

What happened in the Kafka community in January 2019?

Releases:

KIPs:

Blogs:

More from Announcements

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power

IBM Newsletters