What happened in the Kafka community in May 2019?
- 2.3.0: The release process of Kafka 2.3.0 continued. KIP freeze happened on May 13 and feature freeze on May 17. On May 20, Colin McCabe cut the 2.3 branch. The release date is still planned for mid-June.
- 2.2.1: A vote for RC0 started on May 8, but an issue was found. The vote for RC1 was successful and Vahid Hashemian released 2.2.1 on the June 3.
Last month, the community submitted 12 KIPs (KIP-465 to KIP-476), and these are the ones that caught my eye:
It is relatively common to want to send a record containing a list of objects. At the moment, users have to either convert the List into a supported type (ByteBuffer or ByteArray for example) or provide their own SerDes. This KIP proposes adding a Serializer and a Deserializer for the
List<T> Java type to simplify this use case.
Upon receiving records, Kafka brokers perform some validations to ensure all records are valid. At the moment, if a record looks corrupted, brokers reject the whole batch containing that record, making it hard to identify exactly what happened. The goal of this KIP is to easily identify bad records and improve the handling of such errors.
After KIP-345, which introduced static membership in Consumer Group, this KIP aims to reduce potential rebalances even further. This KIP suggests a new Admin API to pre-register group members in order to only rebalance once all expected members have joined, instead of when each member joins.
RocksDB is the default state store for Streams. It has a number of metrics that can be really useful to find performance bottlenecks and tune it, but at the moment, users have to gather them explicitly. This KIP proposes exposing the most common RocksDB metrics directly with the Streams metrics.
IBM Event Streams for Cloud is Apache Kafka-as-a-service for IBM Cloud.