Processing Application

In Business Automation Insights, a Processing Application defines the input, the transformations and the outputs for handling events from custom sources. These events can then be visualized in dashboards with the Business Performance Center, or sent to custom Kafka topics for downstream use to drive automation, populate datalakes, and support other types of data storage.

Overview

A Processing Application consumes events and applies transformations that you specify, in order to extract and format the data that you want to monitor to meet your business needs. Processing can be customized to select, transform, and anonymize data.

Inbound events are read from a Kafka topic. As a result of processing, documents are stored in OpenSearch, or events are sent to other Kafka topics, or both.

The Processing Application is also responsible for:

Providing a Monitoring Source definition for the Business Performance Center to query the relevant data stored in OpenSearch. It has details on locating data to build monitoring dashboards and charts.
Defining an OpenSearch mapping definition to customize how documents and fields that it contains are stored and indexed within OpenSearch.

Note: The content of the inbound events is always a JSON document.

Figure 1. Custom event end-to-end flow with a Processing Application

The Monitoring Application is Business Performance Center within Business Automation Insights.

Custom Processing Application

The Processing Application is managed by the Management service. The service provides a command-line interface (CLI) to develop and test custom Processing Applications.

Using the CLI you can create, update and deploy a Processing Application. The CLI can also be used to produce inbound events and used events to check the results of the event processing.

The Processing Application uses Apache Flink framework and runtime. Typically when you use the Apache Flink framework and runtime you have to develop an Apache Flink job from scratch. The Processing Application does this for you.

A Processing Application is defined by a unique name and a set of configuration files. These files define the source and the destination of events, and how the events are processed. The configuration uses the JSLT External link opens a new window or tab scripting language to define the event transformations, JSON field matching, processing conditions, and filtering.

Figure 2. Development and deployment architecture for creating a Processing Application