Kafka Monthly Digest: March 2019

What happened in the Kafka community in March 2019?

Releases:

After three Release Candidates, Matthias J. Sax released Apache Kafka 2.2.0 on the March 26, 2019. This new minor version contains a number of interesting features:

The ability to re-authenticate SASL connections periodically (KIP-368)
Hardened inter-broker protocol (KIP-380)
The ability to separate Controller traffic from data plane (KIP-291)
Producer/AdminClient/Streams API improvements
Improved consumer group management (KIP-289)
Metrics without a value are now emitted as NaN instead of various values (KIP-386)
Updated kafka-topics.sh tool—it now uses the AdminClient API and does not require access to zookeeper anymore (KIP-377)
All command line tools now accept the --help flag (KIP-374)

KIPs:

Last month, the community submitted 10 KIPs (KIP-438 to KIP-447). These are the ones that caught my eye:

KIP-440: Extend Connect Converter to support headers
At the moment, Kafka Connect Converters don’t support message headers. In environments that rely on a lot of headers, this makes it impossible to use Connect to import or export messages. This KIP aims at fixing this discrepancy and allowing Converters to use headers.

KIP-443: Return to default segment.ms and segment.index.bytes in Streams repartition topics
In 2.0.0, the default configurations of Kafka Streams repartition topics changed in order to improve high throughput applications. However, it turns out these settings are a bit too aggressive for low throughput use cases. This KIP proposes removing these settings and instead use the default values.

KIP-444: Augment metrics for Kafka Streams
Operating Streams applications is currently relatively hard due to the lack of several key metrics. This KIP aims at addressing these issues and started doing a full review of all existing Streams metrics. The idea is to remove metrics that turned out not useful and provide new metrics to allow better monitoring and ease debugging.

Blogs:

IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.

Get started with IBM Event Streams

Mickael Maison

Software Engineer

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity

2 min read - To meet customer demands, applications are expected to be performing at their best at all times. Simultaneously, applications need to be flexible and cost effective, and therefore supported by an underlying infrastructure that is equally reliant, performant and secure as the applications themselves. Easier said than done. According to EMA's 2024 Network Management Megatrends report only 42% of responding IT professionals would rate their network operations as successful. In this era of hyper-distributed infrastructure where our users, apps, and data…

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

3 min read - Security operations is getting more complex and inefficient with too many tools, too much data and simply too much to do. According to a study done by IBM, SOC team members are only able to handle half of the alerts that they should be reviewing in a typical workday. This potentially leads to missing the important alerts that are critical to an organization's security. Thus, choosing the right SIEM solution can be transformative for security teams, helping them manage alerts…

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power

2 min read - As companies undergo digital transformation, they rely on APIs as the backbone for providing new services and customer experiences. While APIs can simplify application development and deliver integrated solutions, IT shops must have a robust solution to effectively manage and govern them to ensure that response times and costs are kept low for all applications. Many customers use Salesforce’s MuleSoft, named a leader by Gartner® in full lifecycle API management for seven consecutive times, to manage and secure APIs across…

What happened in the Kafka community in March 2019?

Releases:

KIPs:

Blogs:

More from Announcements

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power

IBM Newsletters