Metric-based machine learning overview
The metric-based machine learning system in IBM Z® Anomaly Analytics ensemble detects anomalies in the metric data from z/OS® System Management Facilities (SMF) record types and in the log data from IBM® IMS log record types.
Data flow
The following steps describe the data flow among the components of the metric-based machine
learning system. The step numbers correspond to the numbers that are shown in Figure 1.
- Historical or streaming SMF and IMS data is collected by the Z Common Data Provider and forwarded to Apache Kafka.
- The metric-based machine learning system subscribes to the Apache Kafka topic and processes the SMF and IMS data.
- If a metric data model is available, the summarized data is compared to the model to identify abnormal behavior.
- The score results are written to the Apache Kafka message broker and are stored in a NoSQL long-term data store.
- The rules engine polls Apache Kafka for anomalous events that exceed a configured threshold.
Highly anomalous events are forwarded to the Apache Kafka
Ensemble-Event
topic. - Ensemble subscribes to the
Ensemble-Event
topic, and it groups these anomalous events, together with anomalous events from the log-based machine learning system, into event groups by resource (system or subsystem). Ensemble calculates the severity and confidence score of each event group. The results are stored in a NoSQL long-term data store, and they are forwarded to the Apache KafkaEnsemble-EventGroupNotification
topic. - When a user accesses the ensemble GUI, the score results are retrieved from the NoSQL data store and rendered in the GUI.
