Kafka configuration properties file
You can specify additional Kafka configuration details (for example, for SSL connectivity) by using a properties files.
If you want to specify Kafka configuration details you must create a properties file in the etc directory with the following name format:
<SERVER>_kafka_<Data_Source_Name>.props
Where <SERVER> is the name of the server on which the data source is located
and <Data_Source_Name> is the name of the data source.
For example: NCI_kafka_MyDataSource.props
Any properties listed in a loaded Kafka properties file take precedence on those specified in the UI.
Kafka configuration properties
Broker settings
bootstrap.servers
Type: list.
List of host/port pairs to use for establishing the initial connection to the Kafka cluster.
Consumer settings
key.deserializer
Type: class.
Deserializer class for key that implements the
org.apache.kafka.common.serialization.Deserializer interface.
value.deserializer
Type: class.
Deserializer class for value that implements the
org.apache.kafka.common.serialization.Deserializer interface.
auto.offset.reset
Type: string.
Specifies what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server.
enable.auto.commit
Type: boolean.
If set to true, the consumer's offset will be periodically committed in the
background.
max.poll.records
Type: int.
The maximum number of records returned in a single call to poll().
max.partition.fetch.bytes
Type: int.
The maximum amount of data the server should return for a fetch request.
Producer settings
key.deserializer
Type: class.
Deserializer class for key that implements the
org.apache.kafka.common.serialization.Deserializer interface.
value.deserializer
Type: class.
Deserializer class for value that implements the
org.apache.kafka.common.serialization.Deserializer interface.
acks
Type string.
The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the durability of records that are sent.
retries
Type int.
Setting a value greater than zero causes the client to resend any record whose send fails with a potentially transient error.
batch.size
Type: int.
The producer attempts to batch records together into fewer requests whenever multiple records are being sent to the same partition. This configuration controls the default batch size in bytes.
linger.ms
Type: int.
Specifies that a small amount of artificial delay is added before sending out records to reduce the number of requests sent.
buffer.memory
Type: int.
The total bytes of memory the producer can use to buffer records waiting to be sent to the server.
Security settings
security.protocol
Type: int.
Protocol used to communicate with brokers. Valid values are: PLAINTEXT,
SSL, SASL_PLAINTEXT, SASL_SSL.
ssl.enabled.protocols
Type: list.
List of protocols that have been enabled for SSL connections.
ssl.keystore.type
Type: string.
The file format of the key store file.
ssl.keystore.location
Type: int.
The location of the key store file.
ssl.keystore.password
Type: int.
The store password for the key store file.
ssl.key.password
Type: int.
The password of the private key in the key store file.
ssl.truststore.type
Type: string.
The file format of the trust store file.
ssl.truststore.location
Type: int.
The location of the trust store file.
ssl.truststore.password
Type: int.
The password for the trust store file.