Streams (Source and Target) operators

The Streams operator publishes and subscribes messages to topics in the Streams instance that is associated with your streams flow.

In a streams flow, the Streams operator can be of type Source or of type Target:

• Streams as a Source operator subscribes to topics in the Streams instance.

• Streams as a Target operator publishes to topics in the Streams instance.

Prerequisites

• A Streams instance must be associated with the streams flow.

• If you intend to subscribe to a flow in Streams Studio, it must use the same Streams instance as your streams flow in Visual Studio Code Streams Flow extension.

Topic

A topic is a string that can have a hierarchy of levels that are separated by a slash (/).
This string is the routing information for the Streams instance.

You create a topic in the Streams as Target operator by typing in a name. For example, you might type in Myhome/first_floor. The topic Myhome/first_floor is then created and published in the Streams instance.

In the Streams as Source operator, when you select a topic from a list of existing topics, you are subscribing to a published topic. In the previous example, when you select Myhome/first_floor, you subscribe to that topic.

When subscribing to topic, you have to specify the exact schema. If only one schema matches the topic, that one will be used. However, if more schemas match the topic, you have to select the one you want to subscibe to. Only data matching both, topic pattern and schema, will be ingested.

Persistence

Data is not persisted in Streams. If you subscribe, you get new tuples.

Examples

Let’s say that the following topics are defined in the Streams instance:

• Myhome/first_floor/bedroom/temp_detector

• Myhome/first_floor/kitchen/temp_detector

• Myhome/first_floor/kitchen/smoke_detector

The following examples of Streams of type Source show how to subscribe to different topics:

Subscription to a specific topic

To subscribe to the topic for kitchen temperature data, the Streams as Source operator would subscribe to Myhome/first_floor/kitchen/temp_detector.

Subscription to a single level +

Use a wildcard to subscribe to all topics in the same topic level.

When Streams as Source operator subscribes to the topic Myhome/first_floor/+/temp_detector, the operator subscribes to the following topics:

• Myhome/first_floor/kitchen/temp_detector

• Myhome/first_floor/bedroom/temp_detector

Subscription to multi-level #

If you need to subscribe to an arbitrary number of topic levels, use a multi-level wildcard (#). You can subscribe to all underlying hierarchy levels. The # must be the last character in the topic and it must be preceded by a slash (/).

When Streams as Source operator subscribes to the topic Myhome/first_floor/#, the operator subscribes to all topics that begin with Myhome/first_floor:

• Myhome/first_floor/bedroom/temp_detector

• Myhome/first_floor/kitchen/temp_detector

• Myhome/first_floor/kitchen/smoke_detector