Configuring the Data Streamer

The Z Common Data Provider Data Streamer streams operational data to configured subscribers in the appropriate format. It receives the data from the System Data Engine, splits it into individual messages when necessary, and sends the data to the subscriber.

Before you begin

The Data Streamer can stream data to both on-platform and off-platform subscribers. To reduce general CPU usage and costs, you can run the Data Streamer on System z® Integrated Information Processors (zIIPs).

About this task

To configure the Data Streamer, you must create the Data Streamer started task by copying the sample procedure HBODSPRO in the hlq.SHBOSAMP library, and updating the copy.

If you want to run the Data Streamer as a job rather than a procedure, use the sample job HBODS001 in the hlq.SHBOSAMP library rather than procedure HBODSPRO.

The user ID that is associated with the Data Streamer started task must have the appropriate authority to access the Z Common Data Provider program files, which include the installation files and the policy file. It must also have read/execute permissions to the Java™ libraries in the UNIX System Services file system.

Procedure

To create the started task, complete the following steps:

  1. Copy the procedure HBODSPRO in the hlq.SHBOSAMP library to a user procedure library.
    Tip: You can rename this procedure according to your installation conventions. When the name HBODSPRO is used in the Z Common Data Provider documentation, including in the messages, it means the Data Streamer started task.
  2. In your copy of the procedure HBODSPRO, customize the following parameter values for your environment:
    /usr/lpp/IBM/zcdp/v5r1m0/DS/LIB
    Replace this value with the directory where the Data Streamer is installed in your environment. This directory contains the startup.sh script for the Data Streamer.
    /etc/cdpConfig/myPolicy.policy
    Replace this value with the policy file path and name for your environment.
    nnnnn
    Replace this value with the port number on which the Data Streamer listens for data from the data gatherers. The default port on which the Data Streamer listens for data is 51401.
    Important: All data gatherers must send data to the Data Streamer through this port. If you update this port in the Data Streamer configuration, you must also update it in the configuration for all data gatherers. For more information, see Data Streamer port definition.
    start=w and start=c
    This parameter is optional. The letter w stands for warm, and the letter c stands for cold. When stopping, the Data Streamer saves data that has not been sent to the subscriber in a file buffer so that the data can be sent when the Data Streamer is started again. If you want the Data Streamer to load the data in the file buffer on startup and then send it to the subscriber, use start=w. If you want the Data Streamer to ignore the buffered data at startup, use start=c. If this parameter is absent, the default value start=w takes effect.
    statint=numberh/d
    This parameter is optional. This parameter specifies whether to enable the metrics capture function or not. If it doe not exist, or the value of number is 0, then the metrics capture function is disabled. If the value of number is a nonzero value, then the metrics capture function will be enabled and the statistic interval is number h/d. The h/d is the interval time unit, the h means hour, and the d means day.
    host=ip_address
    This parameter is optional. If specified, the Data Streamer will bind to the specified IP address and can accept connections on that IP address only. The value of the host can be a host name, an IPv4 address, or an IPv6 address. The specified IP address, or the resolved IP address of the specified host name, must be a valid IP address that is active on any of the TCP/IP stacks on the LPAR, or a dynamic virtual IP address (DVIPA) which can be activated by any of the TCP/IP stacks on the LPAR. If this parameter is omitted, or is 0.0.0.0, the Data Stream does a generic bind() to accept connection requests from any IP address associated with any TCP/IP stacks in the LPAR.
    Note: The Data Streamer can only accept connections from the same LPAR where the Data Streamer runs. You must run the System Data Engine in the same LPAR as the Data Streamer.
  3. In your copy of the procedure HBODSPRO, set the following environment variables for your environment:
    JAVA_HOME
    Specify the Java installation directory.
    CDP_HOME
    Specify the location of the Data Streamer working directory. The Data Streamer working directory contains files that are created and used during the operation of the Data Streamer, including the Data Streamer truststore and file buffers.
    Guidelines for the working directory
    Use the following guidelines to help you decide which directory to use as the working directory:
    • The directory must be readable and writable by the user ID that runs the Data Streamer.
    • To avoid possible conflicts, do not use a directory that is defined as the Configuration Tool working directory.
    Important: Do not update, delete, or move the files in the CDP_HOME directory. The value of CDP_HOME must not be a symlink.
    TZ
    Specify the time zone offset for the Data Streamer. For more information, see the information about the format of the TZ environment variable in the z/OS® product documentation.
    Important: If the value of the TZ environment variable is incorrect, the time interval that is set in the Time Filter transform is directly affected. For more information about the Time Filter transform, see webui_transform_time.html.
    RESOLVER_CONFIG
    If a TCP/IP resolver must be explicitly provided, uncomment the RESOLVER_CONFIG environment variable, and specify the correct TCP/IP resolver. The Data Streamer must have access to a TCP/IP resolver. For more information, see resolver_search_order_verify.html#resolver_search_order_verify.
    _BPXK_SETIBMOPT_TRANSPORT
    If you want the Data Streamer to have affinity to a certain TCP/IP stack, uncomment the _BPXK_SETIBMOPT_TRANSPORT environment variable, and specify that TCP/IP stack.
    Important: If the LPAR has multiple TCP/IP stacks, you must specify which stack you want the Data Streamer to use and specify the same TCP/IP stack for the System Data Engine (as instructed in Creating the System Data Engine started task for streaming SMF data). Otherwise, the System Data Engine might be unable to connect to the Data Streamer.
    DEFAULT_HEAP
    The heap value that is used by the Data Streamer Java application by default. DEFAULT_HEAP=4g means that the heap size is 4 GB by default. If you want to change the default heap size, uncomment the parameter DEFAULT_HEAP and set a new size. The default value is 4 GB.
    MAXIMUM_HEAP
    The maximum heap value that is available to the Data Streamer Java application. MAXIMUM_HEAP=4g means that the maximum heap size is 4 GB. If you want to change the maximum heap size, uncomment the parameter MAXIMUM_HEAP and set a new size. This value must be no less than the value of DEFAULT_HEAP. The default value is 4 GB. For more information about the maximum heap size, see The Data Streamer maximum heap size.
    Important:
    • In most cases, a heap size of 4 GB is sufficient.
    • If you are running out of memory, you might increase the heap size.
    • If your heap is never filled, you can reduce the heap size to free up storage.
    • For best performance, set the same value for both DEFAULT_HEAP and MAXIMUM_HEAP.
  4. Update your security software, such as the Resource Access Control Facility (RACF®), to permit the Data Streamer started task to run in your environment.
  5. Set HBODSPRO as SYSSTC or other high priority WLM service classes.