Upgrading Streams

A project administrator can upgrade the Streams service on IBM® Cloud Pak for Data.

Important

If you upgrade the Streams service instance from a version of Streams that is earlier than 5.5.0 to Streams 5.5.0 or later, you must first delete all Streams jobs. Upgrading a Streams service instance will fail if Streams jobs exist in the instance.

  • If you have not already upgraded the Cloud Pak for Data control plane, you can delete the Streams jobs from the Cloud Pak for Data web user interface:
    1. From the navigation menu, click My instances, and then click the Jobs tab.
    2. From the job options menu for each job, click Delete.
  • If you have already upgraded the Cloud Pak for Data control plane and the Streams console is enabled, you can use the console to cancel running jobs. For more information or for other methods to cancel the Streams jobs, see Upgrade fails (Streams).

Before you begin

Required role: To complete this task, you must be an administrator of the project (namespace) where Streams is installed.

Before you upgrade Streams, ensure that:

Common core services Streams requires the Cloud Pak for Data common core services. The common core services are installed once in a given Red Hat OpenShift project. If the common core services are not installed in the project where you plan to install Streams or if the common core services are not at the correct version, the common core services will be automatically installed or upgraded when you upgrade Streams. If the common core services need to be installed or upgraded, it might take longer to upgrade Streams. For more information on the common core services, see:

If you are upgrading multiple services on your cluster, you must run the upgrades one at a time and wait until the upgrade completes before upgrading another service. You cannot run the upgrades in parallel.

Tip: For a list of all available options, enter the following command:
./cpd-cli upgrade --help

Procedure

  1. Complete the appropriate steps to upgrade Streams on your environment:
  2. Verifying that the upgrade completed successfully
  3. Upgrading existing service instances
  4. Complete the tasks listed in What to do next

Upgrading on clusters connected to the internet

From your installation node:

  1. Change to the directory where you placed the Cloud Pak for Data command-line interface and the repo.yaml file.
  2. Log in to your Red Hat OpenShift cluster as a project administrator:
    oc login OpenShift_URL:port
  3. Run the following command to see a preview of what will change when you upgrade the service.
    Important: If you are using the internal Red Hat OpenShift registry and you are using the default self-signed certificate, specify the --insecure-skip-tls-verify flag to prevent x509 errors.
    ./cpd-cli upgrade \
    --repo ./repo.yaml \
    --assembly streams \
    --arch Cluster_architecture \  
    --namespace Project \
    --storageclass Storage_class_name \
    --transfer-image-to Registry_location \
    --cluster-pull-prefix Registry_from_cluster \
    --ask-pull-registry-credentials \
    --ask-push-registry-credentials \
    --latest-dependency \
    --dry-run
    Important: By default, this command gets the latest assembly. If you want to upgrade to a specific version of Streams, add the following line to your command after the --assembly flag:
    --version Assembly_version \

    The --latest-dependency flag gets the latest version of the dependent assemblies. If you remove the --latest-dependency flag, the installer will either leave the dependent assemblies at the current version or get the minimum version of the dependent assemblies.

    If you are upgrading with Portworx storage, add the following line to your upgrade command after the --storageclass flag:
    --override-config portworx \
    If you are upgrading with OpenShift Container Storage, add the following line to your upgrade command after the --storageclass flag:
    --override-config ocs \

    Replace the following values:

    Variable Replace with
    Assembly_version
    The version of Streams that you want to install. The assembly versions are listed in System requirements for services.
    Cluster_architecture Specify the architecture of your cluster hardware:
    • For x86-64 hardware, remove this flag or specify x86_64
    Project Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.
    Storage_class_name Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.

    Refresh 2 or later If you are using the 3.5.2 version of the cpd-cli, remove the --storageclass flag from your command. The cpd-cli upgrade command uses the storage class that was specified during installation.

    Registry_location Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.
    Registry_from_cluster Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.
  4. Rerun the previous command without the --dry-run flag to upgrade the service.

Upgrading on air-gapped clusters

From your installation node:

  1. Change to the directory where you placed the Cloud Pak for Data command-line interface.
  2. Log in to your Red Hat OpenShift cluster as a project administrator:
    oc login OpenShift_URL:port
  3. Run the following command to see a preview of what will change when you upgrade the service.
    Important: If you are using the internal Red Hat OpenShift registry:
    • Do not specify the --ask-pull-registry-credentials parameter.
    • If you are using the default self-signed certificate, specify the --insecure-skip-tls-verify flag to prevent x509 errors.
    ./cpd-cli upgrade \
    --assembly streams \
    --arch Cluster_architecture \
    --namespace Project \
    --storageclass Storage_class_name \
    --cluster-pull-prefix Registry_from_cluster \
    --ask-pull-registry-credentials \
    --load-from Image_directory_location \
    --latest-dependency \
    --dry-run
    Note: If the assembly was downloaded using the delta-images command, remove the --latest-dependency flag from the command. If you don't remove the --latest-dependency flag you will get an error indicating that the flag cannot be used.
    If you are upgrading with Portworx storage, add the following line to your upgrade command after the --storageclass flag:
    --override-config portworx \
    If you are upgrading with OpenShift Container Storage, add the following line to your upgrade command after the --storageclass flag:
    --override-config ocs \

    Replace the following values:

    Variable Replace with
    Cluster_architecture Specify the architecture of your cluster hardware:
    • For x86-64 hardware, remove this flag or specify x86_64
    Project Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.
    Storage_class_name Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.

    Refresh 2 or later If you are using the 3.5.2 version of the cpd-cli, remove the --storageclass flag from your command. The cpd-cli upgrade command uses the storage class that was specified during installation.

    Registry_from_cluster Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.
    Image_directory_location Use the value provided by your cluster administrator. You should have obtained this information when you completed Preparing to install and upgrade services.
  4. Rerun the previous command without the --dry-run flag to upgrade the service.

Verifying that the upgrade completed successfully

From your installation node:

  1. Run the following command:
    ./cpd-cli status \
    --assembly streams \
    --namespace Project

    Replace Project with the value you used in the preceding commands.

    • If the upgrade completed successfully, the status of the assembly and the modules in the assembly is Ready.
    • If the upgrade failed, contact IBM Support for assistance.

Upgrading existing service instances

After you upgrade Streams, the service instances that are associated with the installation must also be upgraded. This task must be completed by a Cloud Pak for Data administrator or a service instance administrator.

To upgrade service instances, you must have a cpd-cli profile on your local machine. Ensure that your profile points to the instance of Cloud Pak for Data where the service instances exist. Your profile enables Cloud Pak for Data to ensure that you have the appropriate permissions to upgrade the service instances. For details, see Creating a cpd-cli profile.

Tip: For a list of all available options, enter the following command:
./cpd-cli upgrade --help

After the Streams service instances are upgraded, all application pods and builder pods that are waiting to process builds are restarted. Builds that are in progress are not restarted.

From your installation node:

  1. If you use a custom application image or custom application resource templates, upgrade them by performing the following steps:
    1. If you use a custom application image, rebuild it based on the upgraded default application image, which is streams-application-el7.
      Important: You must assign a new image tag to the rebuilt application image. Ask your administrator about the image registry and image tag to use.
      For detailed instructions, see Customizing the Streams application image.
    2. Update the custom application resource template configuration map with the new image tag. The new image tag can be either the image tag that you assigned to the upgraded custom application image, or the image tag that is assigned to the upgraded default application image. For detailed instructions, see Creating a Streams application resource template.
  2. Run the following command to see the list of service instances:
    ./cpd-cli service-instance list \
    --profile Profile_name

    Replace Profile_name with the name of your local profile.

    The command returns a list of all of the service instances that you have access to.

  3. Run the following command to upgrade all of the service instances:
    ./cpd-cli service-instance upgrade \
    --service-type streams \
    --profile Profile_name \
    --watch \
    --all

    Replace Profile_name with the name of your local profile.

  4. Verify that the service instances were updated and are ready to use:
    1. Run the following command to see a list of the service instances:
      ./cpd-cli service-instance list \
      --service-type streams \
      --profile Profile_name

      Replace Profile_name with the name of your local profile.

      The command returns a list of all of the streams service instances that you have access to.

    2. Run the following command for each service instance to verify that the instance is ready to use:
      ./cpd-cli service-instance status Instance_name\
      --profile Profile_name

      Replace the following values:

      Variable Replace with
      Instance_name The name of the service instance for which you want to see the status.
      Profile_name The name of your local profile.

      Confirm that the service status is Running.

What to do next

The service is ready to use. For more information, see Analyzing streaming data.