Upgrading Custom Edition

When a new version of the Instana Enterprise operator is released, you can upgrade the operator by using the following steps:

1. Install the target version of the Instana kubectl plug-in. The Instana kubectl plug-in and the Instana Enterprise Operator are versioned together, so install the plug-in version that matches the version of the Operator you are installing. For more information, see Installing the Instana kubectl plug-in.

2. Upgrade the operator by using either of the following methods, where the new operator is applied with a default Instana backend version:

   • Applying Manifests directly
   • Rendering manifests first and applying manifests next

   Notes:

   • Make sure that you apply or generate new YAML manifests. Do not update the image version in the existing YAML manifests directly. If you update the image version in the existing YAML manifests, then you might miss the CustomResourceDefinition (CRD) updates, or other changes that can lead to unpredictable errors.

   • To upgrade to a specific version, you might require to do some extra actions. Refer to the Upgrade notes section. When you skip a release, make sure that you consider the upgrade notes (including upgrade notes for the skipped version and the target version).

1. To upgrade to a higher Instana backend version, you can use the following subcommands of the versions command. All commands have an optional --download-key flag. If you do not specify the flag, the download key of the existing installation is used.

   • The identify subcommand provides a list of currently available Instana backend versions that are compatible with the installed Custom Edition.
   • The list-images subcommand prints a list of images of the Instana Kubernetes operator and all Instana components. You can use the --instana-version flag to specify the operator version. If you do not use the flag, all available operator versions are listed, and you can then select a version.
   • The update command upgrades an existing installation to a new version. You can specify the upgrade version by using the --instana-version flag. Otherwise, all supported upgrade versions are displayed. You can then select a version. Alternatively, you can configure the backend version that you want to upgrade to in the core spec and apply the spec, see the following sample code.

     ...
     spec:
       imageConfig:
         tag: 3.xxx.xxx-0
     ...
      

2. Verify the Instana backend upgrade by running the following commands:

   kubectl get core -n instana-core
    

   kubectl get units -n instana-units
    

Upgrade the ClickHouse data store to version 25.8.6.11. Use the image version 25.8.6.11-1-lts-ibm.

Upgrade the ClickHouseKeeper data store to version 25.8.6.11. Use the image version 25.8.6.11-1-lts-ibm.

No special steps are needed to upgrade from release 1.5.0.

The long-deprecated downloadKey in Core Secret is removed. It is now only read from the Unit Secret. If you still have an installation that has the downloadKey in Core Secret, you must update your installation and move it to the Unit Secret.

For configuring the Unit Secret, see Unit secret.

No special steps are needed to upgrade from release 1.3.0.

Before you upgrade to 1.3.0, complete the following tasks:

• The image configuration for the Instana Enterprise Operator and Webhook deployments is separated. Update your kubectl plug-in custom values files by referring to the new Instana Enterprise operator configuration options.

• Optional: If you plan to enable the new Gateway Controller to manage ingress in your Custom Edition environment, complete the following configuration changes:

  • Enable the Gateway Controller, see Gateway Configuration for more details.
  • Update the selector labels in your Gateway loadbalancer .spec.selector["app.kubernetes.io/component"]: gateway with the entries shown in the following example:

  apiVersion: v1
  kind: Service
  metadata:
    namespace: instana-core
    name: loadbalancer-gateway
  spec:
    type: LoadBalancer
    ...
    selector:
      ...
      app.kubernetes.io/component: gateway-v2
      ...
   

  This update is to make sure that the ingress traffic is forwarded to the new Gateway-v2 component pods.

• Upgrade the ClickHouse data store to version 24.8.x. Use the image version 24.8.12.28-5-lts-ibm.

• Upgrade the Elasticsearch data store to version 8.x. To upgrade the data store, update your Elasticsearch configuration as shown in the following example to prevent interruptions:

Replace:

config:
      node.master: true
      node.data: true
      node.ingest: true
 

with:

    config:
      node.roles:
        - master
        - data
        - ingest
 

Additionally, use the image version 8.15.3_v0.15.0 and update the spec.version in your Elasticsearch manifest to match the upgraded image version to avoid errors when upgrading.

Before you upgrade to 1.2.0, complete the following tasks:

• In the ClickHouseInstallation custom resource, disable the query analyzer by adding default/allow_experimental_analyzer: "0" in the profiles section.
• Upgrade the ClickHouse data store to version 24.3. Use image version 24.3.12.75-lts-ibm.

If you are upgrading from releases older than 1.0.0, check the changes to the versions schema and the changes in managing the kubectl plug-in.

No special steps are needed to upgrade from release 1.0.0.

If you are upgrading from releases older than 1.0.0, check the changes to the versions schema and the changes in managing the kubectl plug-in.

The general upgrade notes and the version schema are changed. But, for the Instana backend updates for Build 1.0.0, no special steps are needed.

If you are upgrading from release 273 or earlier, follow the steps in Upgrading to release 275. No special steps are needed to upgrade from release 275 onwards.

Starting from Instana release 275, a new table is introduced to the ClickHouse logs database that uses the ClickHouse TTL mechanism to automatically move data to cold storage over time and then expire it as needed.

The ClickHouse data store configuration needs to be updated for release-275 migrations to be applied.

Proceed with the following steps:

1. Set your Instana backend into maintenance mode by updating the Core spec:

   kubectl patch core <core_name> -n <core_namespace> --type='merge' -p='{"spec": {"operationMode": "maintenance"}}'
    

2. Add the following configurations to your ClickHouseInstallation resource:

   • Define a new cold_disk in <disks>, create a new policy called logs_policy_v4 and reference the cold_disk in the new policy:

     config.d/storage.xml: |
        <clickhouse>
          <storage_configuration>
            <disks>
              <default/>
              <cold_disk>
                <path>/var/lib/clickhouse-cold/</path>
              </cold_disk>
            </disks>
            <policies>
              <logs_policy>
                <volumes>
                  <data>
                    <disk>default</disk>
                  </data>
                  <cold>
                    <disk>cold_disk</disk>
                  </cold>
                </volumes>
              </logs_policy>
              <logs_policy_v4>
                <volumes>
                  <tier1>
                    <disk>default</disk>
                  </tier1>
                  <tier2>
                    <disk>cold_disk</disk>
                  </tier2>
                </volumes>
              </logs_policy_v4>
            </policies>
          </storage_configuration>
       </clickhouse>
      

   • Define a new volumeClaimTemplate that ClickHouse can use to create an additional PVC for the cold_disk:

     volumeClaimTemplates:
       ...
       ...
       - name: instana-clickhouse-data-cold-volume
         spec:
           accessModes:
              - ReadWriteOnce
           resources:
              requests:
              storage: 100Gi
           # storageClassName: <lower_or_cheaper_tier_storageclassname>
      

   • Finally, mount the new volume in the ClickHouse Pod by defining volumeMounts in the podTemplates:

     containers:
       - name: instana-clickhouse
         ...
         ...
         volumeMounts:
           - mountPath: /var/lib/clickhouse-cold/
             name: instana-clickhouse-data-cold-volume
      

     For the full ClickHouseInstallation resource YAML, see:

     • Installing ClickHouse online (AMD).
     • Installing ClickHouse online (Power).
     • Installing ClickHouse online (Z).

   • Before you proceed with the upgrade to release-275, make sure ClickHouse shows a status of Completed after you apply the new changes. Use the following command:

     % kubectl get clickhouseinstallation -n instana-clickhouse
     
     NAME      CLUSTERS   HOSTS   STATUS      AGE
     instana   1          2       Completed   4d1h
      

3. Set your Instana backend back to normal mode by updating the Core spec:

   kubectl patch core <core_name> -n <core_namespace> --type='merge' -p='{"spec": {"operationMode": "normal"}}'
    

4. Upgrade to Instana 275 by following the Upgrade procedure.

The upgrade does not require any special steps.