Overview of Data Gate for Confluent

Edit online

Data Gate for Confluent is the most efficient way to bring streaming data on an IBM Z® mainframe computer to the Confluent Platform.

Data Gate for Confluent is a solution designed to integrate Db2® for z/OS® data into Kafka environments using the Confluent Platform. It acts as a source connector for Kafka Connect, leveraging Data Gate technology to provide data in open formats, specifically the Debezium event format. This enables standardized integration into Kafka-based ecosystems and ensures interoperability for data consumers.

Overall, Data Gate for Confluent provides a robust and efficient data integration solution for organizations looking to leverage their Db2 for z/OS data in Kafka environments.

Benefits Data Gate for Confluent

The benefits of using Data Gate for Confluent include:

  • Efficient data replication through optimized change data processing pipeline and transaction-aware net-effect pruning.
  • High-performance data synchronization with low latency.
  • Minimal operational impact on Db2 for z/OS.
  • Support for various data serialization formats, including Avro and JSON.
  • Seamless integration with Confluent Platform.

Key features

Data Gate for Confluent includes support for Db2 for z/OS as a data source, with capabilities such as table snapshot mirroring via JDBC query and log streaming via a built-in Db2 for z/OS transaction log reader (also known as "integrated synchronization" technology from the Data Gate product family). The solution is designed to work seamlessly with the Confluent Platform, providing features like plugin installation and integration in Kafka Connect, monitoring via Confluent Control Center, and event metadata publishing to the Confluent Schema Registry.

Architecture

The architecture of Data Gate for Confluent involves working with Kafka Connect and the Debezium Connector plugin. It operates with minimal operational impact on Db2 for z/OS by applying required data processing and transformations off the z platform. This results in high-performance data synchronization with low latency.

Prerequisites

To use Data Gate for Confluent, the following prerequisites must be met:

  • Confluent Platform for running the Data Gate source connector and Kafka target.
  • IBM Data Gate for z/OS.
  • IBM Db2 Connect for connecting from Data Gate to the Db2 for z/OS source database.
Important: Confluent Cloud deployments (aaS) and third-party Kafka environments outside of Confluent Platform are not currently supported.

Getting started

To get started with Data Gate for Confluent, follow these steps:

  • Ensure that the prerequisites are met.
  • Obtain the Data Gate for Confluent Installer for deployment in the existing Confluent Platform. The connector can be downloaded using IBM Passport Advantage.
  • Install the connector plugin for Kafka Connect.
  • Configure the connector according to the specific requirements, including setting up authentication and data serialization formats.