Flink parameters
Provide the appropriate values to configure Apache Flink processing, the Apache Flink persistent volume claim (PVC), and the Docker image that is used by the initialization containers in pods for deploying Flink event processors.
Notation
Configuration parameters for custom resources start with the spec.bai_configuration. prefix.
For example:
spec:
bai_configuration:
flink:
task_manager_memory: '2728mb'Measurement units
The task_manager_memory and job_manager_memory parameters are used for configuring Flink parameters. For the list of supported units, see the Enum MemorySize.MemoryUnit page of the documentation for Flink configuration.
The task_manager_memory_request,
task_manager_memory_limit, job_manager_memory_request, and
job_manager_memory_limit parameters are used for configuring requests and
limits for Kubernetes memory and CPU resources. Find the valid units on the following pages.
- For memory requests and limits, see the Meaning of Memory page of the Kubernetes documentation.
- For CPU requests and limits, see the Meaning of CPU page of the Kubernetes documentation.
Flink processing
The following table lists the parameters for configuring Apache Flink processing.| Parameter | Description | Default value |
|---|---|---|
| flink.rocks_db_properties_config_map | The name of a ConfigMap object that is already deployed to Kubernetes and
contains RocksDB properties. For more information, see Specifying configuration maps for RocksDB properties. |
None |
| flink.job_checkpointing_interval | The interval between checkpoints of Apache Flink jobs (in milliseconds) | 5000 |
| flink.log_level | The logging level for the Flink event processor. Use this parameter to increase log verbosity
when Flink jobs process events from custom sources through the event forwarder. Valid values are
INFO, WARN, ERROR, FATAL, DEBUG, and TRACE. Note: This parameter only works for the first time
install flink, you have to update the rootLogger.level from FlinkDeployment CR
directly.
|
INFO |
| flink.job_manager_memory | The total size of the Apache Flink job manager process. Flink parameter: jobmanager.memory.process.size | 1728mb |
| flink.job_manager_cpu_request | The CPU request for pods of Flink job managers (in cpu units). Kubernetes parameter: resources.requests.cpu | 1 |
| flink.job_manager_cpu_limit | The CPU limit for pods of Flink job managers (in cpu units). Kubernetes parameter: resources.limits.cpu | 1 |
| flink.job_manager_memory_request | The memory request for pods of Apache Flink job managers. Kubernetes parameter: resources.requests.memory | 1728Mi |
| flink.job_manager_memory_limit | The memory limit for pods of Flink job managers. Kubernetes parameter: resources.limits.memory | 1728Mi |
| flink.task_manager_memory | The total size of the Flink task manager process. Flink parameter: taskmanager.memory.process.size | 1728mb |
| flink.task_manager_memory_request | The memory request for pods of Apache Flink task managers. Kubernetes parameter: resources.requests.memory | 1728Mi |
| flink.task_manager_memory_limit | The memory limit for pods of Flink task managers. Kubernetes parameter: resources.limits.memory | 1728Mi |
| flink.task_manager_cpu | The number of CPUs that are used by Flink task managers (in CPU units). Flink parameter: kubernetes.taskmanager.cpu | 1 |
| flink.task_manager_cpu_request | The CPU request for pods of Flink task managers (in CPU units). Kubernetes parameter: resources.requests.cpu | 1 |
| flink.task_manager_cpu_limit | You can use this parameter to set the CPU limit for pods of Flink task managers (in CPU units). Kubernetes parameter: resources.limits.cpu | 1 |
| flink.additional_task_managers | Use this parameter to set the number of additional Flink task managers that must be created to host custom processing applications. This parameter is mandatory only if you use custom processing applications. | 0 |
| flink.elasticsearch_max_actions | You can use this parameter to set the number of create, delete and update actions to be performed on document indices in a single request. | 1 |
| flink.elasticsearch_flush_interval | The time interval, in milliseconds, at which to flush the buffered actions, regardless of the number of actions that are specified in the elasticsearch_max_actions parameter. The value is ignored if the elasticsearch_max_actions parameter is set to 1. | 1000 if the elasticsearch_max_actions parameter is greater than 1. |
| flink.verbose_logs | Set this parameter to true to increase log verbosity when Flink jobs process fixed-format events. | false |
Flink persistent volume claim (PVC)
| Parameter | Description | Default value |
|---|---|---|
| flink_pv.capacity | The capacity of the persistent volume (PV) | None |
| flink_pv.storage_class_name | Inherits the default value of the shared_configuration.sc_dynamic_storage_classname parameter. | None |
| flink_pv.existing_claim_name | The name of an existing claim if one is available. By default, a new persistent volume claim is created. | New persistent volume |
Initialization image
| Parameter | Description | Default value |
|---|---|---|
| init_image.image.repository | The name of the Docker image for initialization containers | bai-init |
| init_image.image.tag | The tag of the Docker image | Inherits the default value of the shared_configuration.sc_image_tag parameter. If that value is not set, the image digest that is predefined by the ICP4BA operator image is used instead. |