Creating an Elasticsearch data store on Linux on IBM Z and LinuxONE

Install the Elasticsearch operator and set up the data store.

Before you begin

Make sure that you prepared your online and offline host to pull images from the external repository. Also, ensure that you added the Helm repo.

For more information, see Preparing to install data store operators.

Elasticsearch operator versions and image tags

The following images are needed for the pinned Helm chart or operator versions.

Table 1. Operator versions and image tags for deployment
Platform Operator versions Helm chart version Image with tag
Linux® on IBM Z® and LinuxONE 3.0.0 3.0.0 artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch:3.0.0_v0.20.0

artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:8.17.2_v0.18.0

Installing the Elasticsearch operator online

Complete these steps to install the Elasticsearch data store.

  1. Create the instana-elastic namespace.

    kubectl create namespace instana-elastic

  2. Create the elasticsearch service account.

    kubectl create serviceaccount elasticsearch -n instana-elastic

  3. Create image pull secrets for the instana-elastic namespace. Update the <download_key> value with your own download key.

    kubectl create secret docker-registry instana-registry --namespace instana-elastic \
--docker-username=_ \
--docker-password=<download_key> \
--docker-server=artifact-public.instana.io

  4. Create custom_values.yaml and specify the toleration and affinity. Skip this step if the file is already created.

    tolerations:
- key: node.instana.io/monitor
  operator: Equal
  effect: NoSchedule
  value: "true"
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: node-role.kubernetes.io/monitor
              operator: In
              values:
              - "true"

  5. Install the Elasticsearch operator.

    helm install elastic-operator instana/eck-operator -n instana-elastic --version=3.0.0 --set image.repository=artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch --set image.tag=3.0.0_v0.20.0 --set imagePullSecrets[0].name=instana-registry -f custom_values.yaml

  6. Create a YAML file, for example elastic.yaml, with the Elasticsearch configuration. 

    apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: instana
spec:
  image: artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:8.17.2_v0.18.0
  version: 8.17.2
  nodeSets:
    - name: default
      count: 3
      config:
        node.roles:
          - master
          - ingest
          - data
        node.store.allow_mmap: false
        xpack.ml.enabled: false
      podTemplate:
        spec:
          serviceAccountName: elasticsearch
          imagePullSecrets:
            - name: instana-registry
          affinity:
             nodeAffinity:
               requiredDuringSchedulingIgnoredDuringExecution:
                 nodeSelectorTerms:
                   - matchExpressions:
                       - key: node-role.kubernetes.io/monitor
                         operator: In
                         values:
                         - "true"
          tolerations:
            - key: node.instana.io/monitor
              operator: Equal
              effect: NoSchedule
              value: "true"
          # Add the following securityContext snippet for Kubernetes offerings other than OCP.
          # securityContext:
          #   fsGroup: 1000
          #   runAsGroup: 1000
          #   runAsUser: 1000
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  http:
    tls:
      selfSignedCertificate:
        disabled: true

  7. Complete the steps in Deploying and verifying Elasticsearch (online and offline).

Installing Elasticsearch offline

If you didn't yet pull the Elasticsearch images from the external registry when you prepared for installation, you can pull them now. Run the following commands on your bastion host. Then, copy the images to your Instana host that is in your air-gapped environment.

docker pull artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch:3.0.0_v0.20.0
docker pull artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:8.17.2_v0.18.0

Complete the following steps on your Instana host.

  1. Retag the images to your internal image registry.

     docker tag artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch:3.0.0_v0.20.0 <internal-image-registry>/elasticsearch/eck-operator:3.0.0_v0.20.0
 docker tag artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:8.17.2_v0.18.0 <internal-image-registry>/elasticsearch/elasticsearch:8.17.2_v0.18.0

  2. Push the images to your internal image registry on your bastion host.

     docker push <internal-image-registry>/operator/elasticsearch:3.0.0_v0.20.0
 docker push <internal-image-registry>/datastore/elasticsearch:8.17.2_v0.18.0

  3. Create the instana-elastic namespace for the Elasticsearch instances.

    kubectl create namespace instana-elastic

  4. Create the elasticsearch service account.

    kubectl create serviceaccount elasticsearch -n instana-elastic

  5. Optional: Create an image pull secret if your internal image registry needs authentication.

    kubectl create secret docker-registry <secret_name> --namespace instana-elastic \
--docker-username=<registry_username> \
--docker-password=<registry_password> \
--docker-server=<internal-image-registry>:<internal-image-registry-port> \
--docker-email=<registry_email>

  6. Create custom_values.yaml and specify the toleration and affinity. Skip this step if the file is already created.

    tolerations:
- key: node.instana.io/monitor
  operator: Equal
  effect: NoSchedule
  value: "true"
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: node-role.kubernetes.io/monitor
              operator: In
              values:
              - "true"

  7. Install the Elasticsearch operator. If you created an image pull secret in the previous step, add --set imagePullSecrets[0].name="<internal-image-registry-pull-secret>" to the following command.

    helm install elastic-operator eck-operator-3.0.0.tgz -n instana-elastic --version=3.0.0 --set image.repository=<internal-image-registry>/operator/elasticsearch --set image.tag=3.0.0_v0.20.0 -f custom_values.yaml

  8. Create a YAML file, for example elasticsearch.yaml, with the Elasticsearch configuration. 

    apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: instana
spec:
  image: <internal-image-registry>/elasticsearch/elasticsearch:8.17.2_v0.18.0
  version: 8.17.2
  nodeSets:
    - name: default
      count: 3
      config:
        node.roles:
          - master
          - ingest
          - data
        node.store.allow_mmap: false
        xpack.ml.enabled: false
      podTemplate:
        spec:
          serviceAccountName: elasticsearch
          imagePullSecrets:
            - name: <internal-image-registry-pull-secret>
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: node-role.kubernetes.io/monitor
                      operator: In
                      values:
                      - "true"
          tolerations:
            - key: node.instana.io/monitor
              operator: Equal
              effect: NoSchedule
              value: "true"
          # Add the following securityContext snippet for Kubernetes offerings other than OCP.
          # securityContext:
          #   fsGroup: 1000
          #   runAsGroup: 1000
          #   runAsUser: 1000
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  http:
    tls:
      selfSignedCertificate:
        disabled: true

  9. Complete the steps in Deploying and verifying Elasticsearch (online and offline).

Deploying and verifying Elasticsearch (online and offline)

To deploy the Elasticsearch instance and create the data store, complete the following steps:

  1. Deploy Elasticsearch. By default, a user by the name elastic is created with a random-generated password.

    kubectl apply -f elastic.yaml -n instana-elastic
kubectl wait elasticsearch/instana --for=condition=ReconciliationComplete --timeout=300s -n instana-elastic

  2. Retrieve the password.

    kubectl get secret instana-es-elastic-user -n instana-elastic -o go-template='{{.data.elastic | base64decode}}' && echo

  3. Store the retrieved password in the config.yaml as shown in the following piece of code. Replace <RETRIEVED_FROM_SECRET> with the password that you got in the previous step.

    datastoreConfigs:
  ...
  elasticsearchConfig:
    adminUser: elastic
    adminPassword: <RETRIEVED_FROM_SECRET>
    user: elastic
    password: <RETRIEVED_FROM_SECRET>
  ...

  4. Verify the Elasticsearch operator deployment.

     kubectl get all -n instana-elastic

    If the Elasticsearch operator is deployed successfully, the command output shows the operator status as Running as shown in the following example:

     NAME                       READY   STATUS    RESTARTS      AGE
 pod/elastic-operator-0     1/1     Running   1 (10m ago)   10m
 pod/instana-es-default-0   1/1     Running   0             6m44s
 pod/instana-es-default-1   1/1     Running   0             6m44s
 pod/instana-es-default-2   1/1     Running   0             6m44s

 NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
 service/elastic-operator-webhook   ClusterIP   192.168.1.41    <none>        443/TCP    10m
 service/instana-es-default         ClusterIP   None            <none>        9200/TCP   6m46s
 service/instana-es-http            ClusterIP   192.168.1.111   <none>        9200/TCP   6m48s
 service/instana-es-internal-http   ClusterIP   192.168.1.196   <none>        9200/TCP   6m48s
 service/instana-es-transport       ClusterIP   None            <none>        9300/TCP   6m48s

 NAME                                  READY   AGE
 statefulset.apps/elastic-operator     1/1     10m
 statefulset.apps/instana-es-default   3/3     6m48s

  5. Make sure that the pods are scheduled on the desired nodes.

    kubectl get pods -n instana-elastic -o wide

    A sample output is shown in the following example.

    NAME                   READY   STATUS    RESTARTS   AGE   IP              NODE                                  NOMINATED NODE   READINESS GATES
elastic-operator-0     1/1     Running   1          22d   10.254.12.202   worker0.instana-odf5.cp.fyre.ibm.com   <none>           <none>
instana-es-default-0   1/1     Running   0          34h   10.254.16.137   worker1.instana-odf5.cp.fyre.ibm.com   <none>           <none>
instana-es-default-1   1/1     Running   0          34h   10.254.24.101   worker2.instana-odf5.cp.fyre.ibm.com   <none>           <none>
instana-es-default-2   1/1     Running   0          34h   10.254.20.186   worker3.instana-odf5.cp.fyre.ibm.com   <none>           <none>