Creating a policy to stream SMF data

From Z Common Data Provider Configuration Tool, you can create a policy to stream SMF data to various subscribers.

1. From the Configuration Tool, click the Create a new policy box.
2. In the resulting Policy Profile Edit window, type the required policy name and, optionally, a policy description.
3. Click the Add Data Stream icon .
   The Select data stream window is shown with a list of categorized data streams. Expand the category "IBM® Z Application Performance Management Connect" to view the data streams that you can define for IBM Z APM Connect.
4. Select one or more SMF data streams and click Select. Each data stream that you choose is shown as a node in the graph.

   For IBM Z Application Performance Management Connect, enable the APM_SMF_101, APM_SMF_102_QW172, and APM_SMF_102_QW196 data streams, which require SMF 101 & 102 records to be collected by SMF.

5. Ensure the Default Kafka Topic field matches the topic name you set in your values.yaml file or transaction-processor.yaml file (deployment file). In the “configure data stream” window, check the Default Kafka Topic field. The default value should be IBM-ZAPM-zOS-APM_SMF_101, IBM-ZAPM-zOS-APM_SMF_102_QW172, and IBM-ZAPM-zOS-APM_SMF_102_QW196. If you change those names in your values.yaml file or transaction-processor.yaml file, you must also update the Default Kafka Topic fields accordingly.
   Tip: To find the topic names, check the "consumer_<smf record #>" lines in the cdp-factory section of your values.yaml file or transaction-processor.yaml file.
6. Click the Subscribe icon on a data stream node, the Policy Profile Edit window opens where you can select a previously defined subscriber, or define a new subscriber by completing the following steps.
   1. In the Subscribe to a data stream or Subscribe to a transform window, click the Add Subscriber icon .
   2. In the resulting Add subscriber window, update the associated configuration values, and click OK to save the subscriber.
      You can update the following values in the Add subscriber window:

      Name

      The name of the subscriber.

      Description

      An optional description for the subscriber.

      Protocol

      The streaming protocol that the Data Streamer uses to send data to the subscriber.

      Note: To send data to Z APM Distributed Gateway, choose Generic Kafka as the streaming protocol.

      Kafka Producer Config

      This optional configuration value is valid only when you choose Kafka protocol. It specifies the file path for the Kafka producer properties file.

      Note: To enable Transport Layer Security (TLS) communications with the Kafka server, you need to update the field to specify the full path name of the configuration file gatherer.consumer.properties under the CDP_HOME directory.

      Kafka Consumer Config

      This optional configuration value is valid only when you choose Kafka protocol. It specifies the file path for the Kafka consumer properties file.

      Note: To enable Transport Layer Security (TLS) communications with the Kafka server, you need to update the field to specify the full path name of the configuration file gatherer.consumer.properties under the CDP_HOME directory.

      Host

      The host name or IP address of the subscriber.

      Port

      The port on which the subscriber listens for data from the Data Streamer.

      Partitioner Class

      This configuration value is valid only when you choose Kafka protocol. It specifies a class to use to determine which partition to be sent to when producing the records. For more information about Partitioner Class, see Subscriber configuration.

      Remember: To stream data to Z APM Distributed Gateway, choose UniformStickyPartitioner.

      Format

      It specifies the format of data that is sent to the Kafka server.

      Attention: This value must be set to Key-Value for the Z APM Connect Distributed Gateway to process it.

      Auto-Qualify

      A specification of whether to prepend system names and sysplex names to data source names in the data streams that are sent to the subscriber. The data source name is the value of the dataSourceName field in the data stream configuration.

      If you use the same policy file for multiple systems within one sysplex, the data source names must be unique across all systems in that sysplex. If you use the same policy file for multiple sysplexes, the data source names must be unique across all systems in all sysplexes. You can use this field to fully qualify these data source names.

      You can choose any of the following values. The default value is None.

      None

      Indicates that the data source name from the dataSourceName field in the data stream configuration is used.

      System

      Specifies that the system name and the data source name are used in the following format:

      systemName-dataSourceName

      systemName represents the name of the system on which the Z Common Data Provider runs.

      If you use the same policy file for multiple systems within one sysplex, you might want to use the System value.

      Sysplex

      Specifies that the sysplex name, system name, and data source name are used in the following format:

      sysplexName-systemName-dataSourceName

      systemName represents the name of the system on which the Z Common Data Provider runs. sysplexName represents the name of the sysplex in which the Z Common Data Provider runs.

      If you use the same policy file for multiple sysplexes, you might want to use the Sysplex value.

      For more information about the dataSourceName field in the data stream configuration, see the following topic

      • Data stream configuration for data gathered by System Data Engine

7. In the Subscribe to a data stream or Subscribe to a transform window, select one or more subscribers, and click Update Subscriptions.
   The subscribers that you choose are then shown on the graph.
8. Click the SYSTEM DATA ENGINE button to set the configuration values for your System Data Engine environment.
   1. In the Global Properties section of the Policy Profile Edit window, click SYSTEM DATA ENGINE.
   2. In the Configure System Data Engine properties window, update the following configuration values for your environment, and click OK.

      CDP Concatenation

      This value must be the name of the SHBODEFS data set that is installed with Z Common Data Provider in your environment. This data set is also referenced in the concats.json file, which is in the working directory for the Z Common Data Provider Configuration Tool.

      ZAPM Concatenation

      The value must be the name of the SAGMSAM data set that is installed with IBM Z APM Connect in your environment.

      Note: This value is relevant only if you are using IBM Z APM Connect. It is required as part of enabling the Configuration Tool to support SMF data that is destined for IBM Z APM Connect.

      This data set is also referenced in the concats.json file, which is in the working directory for the Z Common Data Provider Configuration Tool.

9. To save the policy, click Save.