Creating an Elasticsearch data store on Linux x86_64

Before you begin

Make sure that you prepared your online and offline host to pull images from the external repository. Also, ensure that you added the Helm repo.

For more information, see Preparing to install data store operators.

Elasticsearch operator versions and image tags

The following images are needed for the pinned Helm chart or operator versions.

Installing Elasticsearch online

Complete these steps to install the Elasticsearch data store.

1. Create the instana-elastic namespace.

   kubectl create namespace instana-elastic
   

2. Create image pull secrets for the instana-elastic namespace. Update the <download_key> value with your own download key.

   kubectl create secret docker-registry instana-registry --namespace instana-elastic \
   --docker-username=_ \
   --docker-password=<download_key> \
   --docker-server=artifact-public.instana.io
   

3. Install the Elasticsearch operator.

   helm install elastic-operator instana/eck-operator -n instana-elastic --version=2.9.0 --set image.repository=artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch --set image.tag=2.9.0_v0.11.0 --set imagePullSecrets[0].name="instana-registry"
   

4. Create a YAML file, for example elastic.yaml, with the Elasticsearch configuration.

   apiVersion: elasticsearch.k8s.elastic.co/v1
   kind: Elasticsearch
   metadata:
     name: instana
   spec:
     version: {{site.data.keyword.es_amd}}
     image: artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:7.17.24_v0.10.0
     nodeSets:
       - name: default
         count: 3
         config:
           node.master: true
           node.data: true
           node.ingest: true
           node.store.allow_mmap: false
         podTemplate:
           spec:
             imagePullSecrets:
               - name: instana-registry
             # Add the following securityContext snippet for Kubernetes offerings other than OCP.
             # securityContext:
             #   fsGroup: 1000
             #   runAsGroup: 1000
             #   runAsUser: 1000
         volumeClaimTemplates:
           - metadata:
               name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
             spec:
               accessModes:
                 - ReadWriteOnce
               resources:
                 requests:
                   storage: 20Gi
     http:
       tls:
         selfSignedCertificate:
           disabled: true
   

5. Complete the steps in Deploying and verifying Elasticsearch (online and offline).

Installing Elasticsearch offline

If you didn't yet pull the Elasticsearch images from the external registry when you prepared for installation, you can pull them now. Run the following commands on your bastion host. Then, copy the images to your Instana host that is in your air-gapped environment.

docker pull artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch:2.9.0_v0.11.0
docker pull artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:7.17.24_v0.10.0

Complete the following steps on your Instana host.

1. Retag the images to your internal image registry.

   docker tag artifact-public.instana.io/self-hosted-images/3rd-party/operator/elasticsearch:2.9.0_v0.11.0 <internal-image-registry>/operator/elasticsearch:2.9.0_v0.11.0
   docker tag artifact-public.instana.io/self-hosted-images/3rd-party/datastore/elasticsearch:7.17.24_v0.10.0 <internal-image-registry>/datastore/elasticsearch:7.17.24_v0.10.0
   

2. Push the images to your internal image registry.

   docker push <internal-image-registry>/operator/elasticsearch:2.9.0_v0.11.0
   docker push <internal-image-registry>/datastore/elasticsearch:7.17.24_v0.10.0
   

3. Create the instana-elastic namespace for the Elasticsearch instances.

   kubectl create namespace instana-elastic
   

4. Optional: Create an image pull secret if your internal image registry needs authentication.

   kubectl create secret docker-registry <secret_name> --namespace instana-elastic \
   --docker-username=<registry_username> \
   --docker-password=<registry_password> \
   --docker-server=<internal-image-registry>:<internal-image-registry-port> \
   --docker-email=<registry_email>
   

5. Install the Elasticsearch operator. If you created an image pull secret in the previous step, add --set imagePullSecrets[0].name="<internal-image-registry-pull-secret>" to the following command.

   helm install elastic-operator eck-operator-2.9.0.tgz -n instana-elastic --version=2.9.0 --set image.repository=<internal-image-registry>/operator/elasticsearch --set image.tag=2.9.0_v0.11.0
   

6. Create a YAML file, for example elasticsearch.yaml, with the Elasticsearch configuration.

   apiVersion: elasticsearch.k8s.elastic.co/v1
   kind: Elasticsearch
   metadata:
     name: instana
   spec:
     version: {{site.data.keyword.es_amd}}
     image: <internal-image-registry>/datastore/elasticsearch:7.17.24_v0.10.0
     nodeSets:
       - name: default
         count: 3
         config:
           node.master: true
           node.data: true
           node.ingest: true
           node.store.allow_mmap: false
         podTemplate:
           spec:
           # Optional: if you created an image pull secret for your internal registry, uncomment the following lines and update the image pull secret information.
           # imagePullSecrets:
           #   - name: <internal-image-registry-pull-secret>
             # Add the following securityContext snippet for Kubernetes offerings other than OCP.
             # securityContext:
             #   fsGroup: 1000
             #   runAsGroup: 1000
             #   runAsUser: 1000
         volumeClaimTemplates:
           - metadata:
             name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
             spec:
               accessModes:
                 - ReadWriteOnce
               resources:
                 requests:
                   storage: 20Gi
     http:
       tls:
         selfSignedCertificate:
           disabled: true
   

7. Complete the steps in Deploying and verifying Elasticsearch (online and offline).

Deploying and verifying Elasticsearch (online and offline)

To deploy the Elasticsearch instance and create the data store, complete the following steps:

1. Deploy Elasticsearch. By default, a user by the name elastic is created with a random-generated password.

   kubectl apply -f elastic.yaml -n instana-elastic
   kubectl wait elasticsearch/instana --for=condition=ReconciliationComplete --timeout=300s -n instana-elastic
   

2. Retrieve the password.

   kubectl get secret instana-es-elastic-user -n instana-elastic -o go-template='{{.data.elastic | base64decode}}' && echo
   

3. Store the retrieved password in the config.yaml as shown in the following piece of code. Replace <RETRIEVED_FROM_SECRET> with the password that you got in the previous step.

   datastoreConfigs:
   ...
   elasticsearchConfig:
     adminUser: elastic
     adminPassword: <RETRIEVED_FROM_SECRET>
     user: elastic
     password: <RETRIEVED_FROM_SECRET>
   

4. Verify the Elasticsearch operator deployment.

   kubectl get all -n instana-elastic
   

   If the Elasticsearch operator is deployed successfully, the command output shows the operator status as Running as shown in the following example:

   NAME                       READY   STATUS    RESTARTS      AGE
   pod/elastic-operator-0     1/1     Running   1 (10m ago)   10m
   pod/instana-es-default-0   1/1     Running   0             6m44s
   pod/instana-es-default-1   1/1     Running   0             6m44s
   pod/instana-es-default-2   1/1     Running   0             6m44s
   
   NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
   service/elastic-operator-webhook   ClusterIP   192.168.1.41    <none>        443/TCP    10m
   service/instana-es-default         ClusterIP   None            <none>        9200/TCP   6m46s
   service/instana-es-http            ClusterIP   192.168.1.111   <none>        9200/TCP   6m48s
   service/instana-es-internal-http   ClusterIP   192.168.1.196   <none>        9200/TCP   6m48s
   service/instana-es-transport       ClusterIP   None            <none>        9300/TCP   6m48s
   
   NAME                                  READY   AGE
   statefulset.apps/elastic-operator     1/1     10m
   statefulset.apps/instana-es-default   3/3     6m48s