February 4, 2019 By Mickael Maison 2 min read

What happened in the Kafka community in January 2019?

Releases:

There were no new releases in January but Matthias J. Sax started the process for version 2.2.0. As always, the release plan is available on the wiki. The current target date for GA is the February 28, 2019.

At the same time, Colin McCabe started the process for bugfix release 2.1.1. Since 2.1.0, the community fixed over 30 issues, including three blocker JIRAs. Check out the wiki if you are interested in the full release plan.

KIPs:

Last month, the community submitted 13 KIPs (KIP-412 to KIP-424). These are the ones that caught my eye:

KIP-412: Extend Admin API to support dynamic application log levels
In order to quickly debug an issue with a Kafka cluster, it’s important to be able to retrieve logs of the affected components. While Kafka already allows you to change log levels at runtime, it’s currently only possible via JMX, and it is relatively hard to do (especially if you’ve never done it before and your cluster is burning!). The goal of this KIP is to provide an easy mechanism via the AdminClient API to dynamically adjust log levels.

KIP-415: Incremental cooperative rebalancing in Kafka Connect
Today, Kafka Connect is reusing the same group membership API as Consumers do. However, because Connector and Tasks can change configuration at runtime, this can lead to a lot of extra rebalances also interrupting other existing connectors. This KIP proposes introducing an Incremental Cooperative protocol to deal with those issues.

KIP-419: Safely notify Kafka Connect SourceTask is stopped
Currently, when Kafka Connect stops a Source Task, the task does not know whether it will restart shortly or if it’s a proper shutdown. Also, in some cases, the Connect Runtime can even call stop() multiple times. This KIP aims at improving the lifecycle of Source Connectors by providing a new method to clearly identify when the Runtime stopped a task for good so it can safely release its resources.

KIP-422: Add support for user/client configuration in the Kafka Admin Client
Kafka allows you to configure four types of entities: brokers, topics, users, and client-ids. Unfortunately, at the moment, the Admin API only supports brokers and topics. In order to configure the other two entity types, users have to run the kafka-configs.sh command line tool and require direct Zookeeper access. The KIP proposes updating the Admin APIs to handle all entity types so the AdminClient supports all entities.

Blogs:

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

More from Announcements

Unify and share data across Netezza and watsonx.data for new generative AI applications

3 min read - In today's data and AI-driven world, organizations are generating vast amounts of data from various sources. The ability to extract value from AI initiatives relies heavily on the availability and quality of an enterprise's underlying data. In order to unlock the full potential of data for AI, organizations must be able to effectively navigate their complex IT landscapes across the hybrid cloud.   At this year’s IBM Think conference in Boston, we announced the new capabilities of IBM watsonx.data, an open…

IBM and SAP unlock business and industry value with new generative AI solutions 

3 min read - IBM Consulting is delivering on our commitment to co-innovate with SAP and collaborate with our clients. As part of our Value Generation Partnership initiative announced earlier this month with SAP, we are releasing the first 10 of 100 planned AI solutions to help clients transform their industries, optimize their business processes and successfully deliver their SAP programs.  Delivering AI business and industry innovation at scale  With the recently announced Value Generation Partnership initiative, IBM and SAP are co-innovating intelligent industry…

IBM SevOne 7.0: Reaching application-centric multicloud network observability  

2 min read - As enterprises increasingly rely on network connectivity to support cloud-based applications and remote workers, network managers require new methods to monitor and safeguard connectivity across diverse environments, including corporate networks, software-defined WANs and multiple public cloud providers.   According to the recent EMA Network Megatrends Report, responding network professionals believe that 53% of network outages and performance issues could be prevented with improved network management tools, yet only 9% find it very easy to hire skilled networking personnel. This is why…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters