Streams processing applications

A stream processing application is a collection of operators, all of which process streams of data.

One of the most basic stream processing applications is a collection of three operators: a source operator, one of the process operators, and a sink or output operator.

From left to right, the operators are the FileSource operator, the Filter operator, and the FileSink operator. — Figure 1. A basic stream processing application: a source operator, a process operator, and a sink or output operator.

Source operator: A source operator creates a stream from an external data source that then can be consumed by downstream operators. The example in Figure 1 shows the FileSource operator that reads data from a file.

Processing operator: In addition to source and sink operators, SPL is supported by many built-in operators in categories such as relational operators and XML operators. The example in Figure 1 shows the Filter operator, which passes to the next operator only that data that satisfies the condition that you specify in the parameter of the filter operator. Other standard processing operators include the Aggregate operator, the Throttle operator, and the Split operator.

Sink operator: A sink operator writes data from its input streams to external data systems. The example in Figure 1 shows a FileSink operator that receives the data in the stream and writes it to a file.

Applications run on the Streams instance in an OpenShift or Kubernetes environment and can run across multiple resources.

Application bundle file

When you compile an application on IBM® Streams Version 4.0 or later, it is compiled into an application bundle file. The application bundle file contains all elements, including Streams toolkit artifacts, that are required to execute an application.

The application bundle file has a file extension of .sab.

You can relocate an application by moving the application bundle file. Except for system libraries, the application bundle file includes all toolkit artifacts that are needed to run your application. The streamtool submitjob command accepts the application bundle file name.

When an application bundle file is submitted for execution, the application bundle file is deployed to all resources on which the application can run.

An application bundle file has an identifier that uniquely distinguishes one build of an application from another. When an application bundle file is submitted for execution, the identifier is used to check whether there is another instance of the same application already running, and if so, it shares the unbundled execution location. In this case, the same runtime application directory hierarchy is used for all executions of a given application bundle file.

You can submit your application bundle file as a Streams job from the following interfaces:

Use the Streams console.
1. Open and log in to the Streams console.
2. Click Submit job in the toolbar.
3. Browse to the executable .sab file and click Submit.
Use the streamsx-streamtool submitjob command to submit the application from the command line. For example:
```
streamsx-streamtool submitjob my-streaming-app.sab
```
For more information about the available options, see the streamsx-streamtool documentation.
Use the Streams instance REST and JMX APIs to submit Streams application bundles.

Additionally, you can control how jobs are deployed by specifying submission-time configuration parameters. For more information, see Optimizing job deployment.