Z Common Data Provider overview
IBM® Z Common Data Provider provides the infrastructure for accessing IT operational data from z/OS® systems and streaming it to the analytics platform in a consumable format. It is a single data provider for sources of both structured and unstructured data, and it can provide a near real-time data feed of z/OS operational data, like System Management Facilities (SMF) data and z/OS log data to your analytics platform.
IBM Z Common Data Provider automatically monitors SMF data and z/OS log data, it can collect SMF data and z/OS log data, and forwards it to the configured destination.
In each logical partition (LPAR) from which you want to analyze SMF data or z/OS log data, a unique instance of IBM Z Common Data Provider must be installed and configured to specify the type of data to be gathered and the destination for that data, which is called a subscriber.
IBM Z Common Data Provider includes a web-based configuration tool that is provided as an application for IBM WebSphere® Application Server for z/OS Liberty, or as a plug-in for IBM z/OS Management Facility (z/OSMF).
Flow of operational data to your analytics platform
As illustrated in Figure 1, operational data (such as SMF data or log data) is gathered by data gatherers, such as the System Data Engine or the Log Forwarder, and can be streamed to multiple subscribers.
The data gatherers send the data to the Data Streamer, which transforms the data before it sends the data to the subscribers.
The flow of data is controlled by a policy that you define in the IBM Z Common Data Provider Configuration Tool.
- policy
- In IBM Z Common Data Provider, a set of rules that define what operational data to collect and where to send that data. A policy is created in the Configuration Tool. See Managing policies for more information.
- subscriber
- In the IBM Z Common Data Provider configuration, the software that you define to receive operational data, like IBM Operations Analytics - Log Analysis, Elastic Stack, Splunk, Logstash, and Apache Kafka. For more information about subscribers, see Subscribers to a data stream or transform and Subscriber configuration.
- Batch job
- The System Data Engine can run as a batch job to create output data for the IBM Z® Decision Support, or for the Data Streamer as a part of
the Data Gatherers. See the following tasks about running batch jobs.
- Creating the System Data Engine batch job for writing SMF data to data sets
- Creating the System Data Engine batch job for sending SMF data to the Data Streamer
- Creating the System Data Engine batch job for loading DCOLLECT data
- Creating the System Data Engine batch job for sending DCOLLECT data to the Data Streamer
