Creating a Processing Application

You create a Processing Application, define and import its processing configuration, and send test events. You then see the result in the Business Performance Center.

About this task

This example helps you create a Processing Application named test-app. The application reads events from a Kafka topic and writes the result of the processing in an OpenSearch index. The configuration defines a simple OpenSearch mapping to monitor data correctly in the Business Performance Center.

Note: Kafka topic and OpenSearch index names must be prefixed by icp4ba-bai.

Procedure

  1. Create a Processing Application named test-app.
    management-cli processing-app  create --name="test-app"
    The management service checks that the name is valid and unique among the applications already created.
  2. Define a processing configuration by creating a dedicated folder named test-app.
  3. Inside the test-app folder, create three configuration files: processing-conf.json, monitoring-sources.json, and opensearch-mapping.json.
    1. Create processing-conf.json to define Kafka Ingress and OpenSearch Ingress.
      {
        "ingresses": [
          {
            "uid": "I0001",
            "type": "kafka",
            "topic": "icp4ba-bai-test-topic"
          }
        ],
        "egresses": [
          {
            "uid": "E0001",
            "type": "opensearch",
            "index": {
              "name": "icp4ba-bai-test-index",
              "docId": {
                "type": "jslt-inline",
                "expression": ".id"
              },
              "mapping": {
                "filename": "opensearch-mapping.json"
              }
            }
          }
        ]
      }
    2. Create monitoring-sources.json to define the OpenSearch index to be monitored by Business Performance Center.
      [
        {
          "id": "test monitoring source",
          "elasticsearchIndex": "icp4ba-bai-test-index"
        }
      ]
    3. Create opensearch-mapping.json that contains the OpenSearch mapping definition.
      {
        "dynamic": true,
        "properties": {
          "id": {
            "type": "keyword"
          },
          "status": {
            "type": "keyword"
          },
          "timestamp": {
            "type": "date"
          }
        }
      }
  4. Import the processing configuration.
    management-cli processing-conf import --name="test-app" --directory="./test-app"
    The management service validates the configuration to ensure it complies with the specification. When there are errors, it returns the list to help you fix the configuration.
  5. Deploy the Processing Application.
    management-cli processing-app deploy --name="test-app"
    This command creates or updates OpenSearch indices/alias/mapping, Kafka topics, and monitoring sources in the processing configuration. It requests the deployment to a dedicated Flink processing Job that uses the configuration to process the events.
  6. Check the status of the deployed processing job.
    management-cli processing-jobs list

What to do next

You can now test the Processing Application Testing the Processing Application.

The example Processing Application shown here does not make any transformations to the events it processes. To learn how to transform and filter events in your Processing Application, see Customizing the Event Processing External link opens a new window or tab.