Creating a policy to stream custom data streams

From the Configuration Tool, you can create a policy to stream one or more custom data streams to various subscribers.

1. From the Configuration Tool, click the Create a new policy box.
2. In the resulting Policy Profile Edit window, type the required policy name and, optionally, a policy description.
3. Click the Add Data Stream icon .
   The Select data stream window is shown with a list of categorized data streams. You can find the custom data streams under the Custom Data Streams group.
4. Select one or more custom data streams and click Select. Each data stream that you choose is shown as a node in the graph.
5. If you want to alter the operational data in the stream, click the Transform icon on a data stream node.
   For more information about transforms, see Transform configuration.
6. Click the Subscribe icon on a data stream node, the Policy Profile Edit window opens where you can select a previously defined subscriber, or define a new subscriber by completing the following steps.
   1. In the Subscribe to a data stream or Subscribe to a transform window, click the Add Subscriber icon .
   2. In the resulting Add subscriber window, update the associated configuration values, and click OK to save the subscriber.
      You can update the following values in the Add subscriber window:

      Name

      The name of the subscriber.

      Description

      An optional description for the subscriber.

      Protocol

      The streaming protocol that the Data Streamer uses to send data to the subscriber. For example, you can choose Logstash if you want to use the Elastic Stack as a subscriber, or you can choose Splunk via Data Receiver if you want to use Splunk as a subscriber. Make sure you choose the protocol that meets your requirements. For more information about protocols, see Subscriber configuration.

      Host

      The host name or IP address of the subscriber.

      Port

      The port on which the subscriber listens for data from the Data Streamer.

      Auto-Qualify

      A specification of whether to prepend system names and sysplex names to data source names in the data streams that are sent to the subscriber. The data source name is the value of the dataSourceName field in the data stream configuration.

      If you use the same policy file for multiple systems within one sysplex, the data source names must be unique across all systems in that sysplex. If you use the same policy file for multiple sysplexes, the data source names must be unique across all systems in all sysplexes. You can use this field to fully qualify these data source names.

      You can choose any of the following values. The default value is None.

      None

      Indicates that the data source name from the dataSourceName field in the data stream configuration is used.

      System

      Specifies that the system name and the data source name are used in the following format:

      systemName-dataSourceName

      systemName represents the name of the system on which the Z Common Data Provider runs.

      If you use the same policy file for multiple systems within one sysplex, you might want to use the System value.

      Sysplex

      Specifies that the sysplex name, system name, and data source name are used in the following format:

      sysplexName-systemName-dataSourceName

      systemName represents the name of the system on which the Z Common Data Provider runs. sysplexName represents the name of the sysplex in which the Z Common Data Provider runs.

      If you use the same policy file for multiple sysplexes, you might want to use the Sysplex value.

      For more information about the dataSourceName field in the data stream configuration, see the following topics:

      • Configuration reference for data gathered by Log Forwarder
      • Configuration reference for data gathered by System Data Engine

7. In the Subscribe to a data stream or Subscribe to a transform window, select one or more subscribers, and click Update Subscriptions.
   The subscribers that you choose are then shown on the graph.
8. If you are creating the policy for a System Data Engine data stream, click the SYSTEM DATA ENGINE button to set the configuration values for your System Data Engine environment. Otherwise, if you are creating the policy for an application data stream, you can skip this step.
   1. In the Global Properties section of the Policy Profile Edit window, click SYSTEM DATA ENGINE.
   2. In the Configure System Data Engine properties window, update the following configuration values for your environment, and click OK.

      USER Concatenation

      This field appears only if you are using custom System Data Engine data streams. It is required as part of enabling the Configuration Tool to support custom System Data Engine data streams. For more information, Creating a System Data Engine data stream definition.

      The value must be the name of the USERDEFS data set that contains the custom System Data Engine definitions. This data set is also referenced in the concats.json file, which is in the working directory for the Z Common Data Provider Configuration Tool.

      CDP Concatenation

      This value must be the name of the SHBODEFS data set that is installed with Z Common Data Provider in your environment. This data set is also referenced in the concats.json file, which is in the working directory for the Z Common Data Provider Configuration Tool.

9. To save the policy, click Save.