Deploying the IBM Z Operational Log and Data Analytics application on the Elastic Stack platform

To use the dashboards and searches for Z operational insights in the Elastic Stack, deploy the IBM Z Operational Log and Data Analytics application on the Elastic Stack platform. You can visit the video library (https://zaiops.github.io/zlda/) to watch the demo videos about the end-to-end deployment process.

Before you begin

Verify that the system requirements are met, as described in Planning for deployment on the Elastic Stack platform, and that all prerequisite software is configured and is running.

Tips:

Elasticsearch can be installed by a root user, but it must be run by a non-root user.
Python must be in the system path.

About this task

You can configure the Elastic Stack environment in different ways depending on the volume of data, number of users and searches, system availability, and disaster recovery. The different components of the Elastic Stack (Elasticsearch, Logstash and Kibana) can be installed together on a single server or distributed on multiple servers. IBM Z Operational Log and Data Analytics includes applications that are installed on Logstash, Elasticsearch and Kibana.

IBM Z Operational Log and Data Analytics ships the following packages for the Elastic Stack platform:

Z Operational Log and Data Analytics ingestion kit for curated data

This package contains the Logstash configuration files for curated data that is provided for use with the Z Operational Log and Data Analytics data streams. This ingestion kit contains the Logstash configuration files that support the Z Operational Log and Data Analytics Elastic Stack application. For more information, see Deploying the Logstash configuration files in the ingestion kit for curated data.

Z Operational Log and Data Analytics ingestion kit for raw data

This package contains the Logstash configuration files for raw data that is provided for use with Z Common Data Provider data streams. For more information, see Deploying the Logstash configuration files in the ingestion kit for raw data.

Z Operational Log and Data Analytics Elastic Stack application

This package contains the dashboards and saved searches that display curated data that is provided with the Z Operational Log and Data Analytics data streams. For more information, see Deploying the dashboards and saved searches in Kibana or see Deploying the Z Operational Log and Data Analytics Elastic Stack application into the Elastic Stack Docker containers if you run and manage the Elastic Stack on Docker.

Note: The Z Operational Log and Data Analytics Elastic Stack application contains scripts that provide tooling for the application, as well as for the Logstash configuration files for curated and raw data:

Import and delete the dashboards and saved searches with the following commands:
```
#python setup.py import
#python setup.py delete
```
For more information, see Deploying the dashboards and saved searches in Kibana.
Generate Elasticsearch index templates from a policy file for raw data with the following command:
```
#python setup.py process_policy
```
For more information, see Generating Elasticsearch index templates from a policy file for raw data.
Copy the H_ and N_ Logstash configuration files for raw data with the following command:
```
#python confPipeline.py copy_raw_conf_files
```
For more information, see Copying the H_ and N_ Logstash configuration files for raw data by using scripted tooling.
Update the Elasticsearch connection definition in Logstash configuration files for raw and curated data with the following command:
```
#python confPipeline.py update_conf_files
```
For more information, see Updating the Elasticsearch connection definition in Logstash configuration files by using scripted tooling.

Important: The user ID that Logstash uses to connect to Elasticsearch must have authority to create and write indexes with the following prefixes:

cdp-*
zoa-*
zlda-*