What's new and changed in the platform

Refresh 9 of Cloud Pak for Data Version 4.0

A new version of the Cloud Pak for Data was released in May 2022.

Platform operator version: Not refreshing. (Use Version 2.0.8, which was shipped with the previous refresh.)

IBM Cloud Pak foundational services version: 3.18.0 or later

Zen operand version: 4.4.4

Bug fixes

• Issue: If you use LDAP groups to create user groups in Cloud Pak for Data, you cannot give the group access to a service instance until at least one user in the group has logged in. Additionally, the user interface does not indicate that the group was not added to the service instance.

  Resolution: You can now add user groups with no users to a service instance.

• Issue: On environments that have numerous services, you see a 502 or 504 timeout error when you try to access the Support > Diagnostics page. The request to load the page times out before the web client can verify the list of services that are installed (and that have associated workloads) on the platform.

  Resolution: The page loads, regardless of the number of services installed on the platform.

Security fixes

This refresh includes fixes for the following security issues:

• CVE-2022-0778

Refresh 8 of Cloud Pak for Data Version 4.0

A new version of the Cloud Pak for Data was released in April 2022.

Platform operator version: 2.0.8

IBM Cloud Pak foundational services version: 3.17.0 or later

Zen operand version: 4.4.3

Updates

Refresh 8 of Version 4.0 of the Cloud Pak for Data platform includes the following features and updates:

New best practice for installing Cloud Pak for Data

The Cloud Pak for Data documentation includes a new best practices topic on setting up a script to source environment variables with the appropriate information for your environment. The script enables you to run most of the commands in the Installing and Upgrading sections of the documentation without making any changes.

The script can also help you:

• Improve the consistency of installations across multiple environments
• Simplify the process of managing multiple deployments on the same cluster

Restriction: The code blocks in the individual service installation and upgrade topics do not currently use the environment variables, but you can replace variables in the code blocks with the appropriate environment variable before you run the commands.

For details, see Best practices: Setting up install variables.

Updated guidance on pinning the control plane at a specific version

The documentation for installing and upgrading the Cloud Pak for Data control plane includes updated guidance for pinning the control plane at a specific version. This guidance helps prevent unexpected upgrades and is especially important if either of the following situations applies to your installation:

• You have multiple IBM Cloud Paks on your cluster
• You have multiple instances of IBM Cloud Pak for Data on your cluster

In these scenarios, you might need to support different versions of the ZenService control plane for each installation.

Bug fixes

Refresh 8 of Version 4.0 platform includes various fixes.

Refresh 7 of Cloud Pak for Data Version 4.0

A new version of the Cloud Pak for Data was released in March 2022.

Platform operator version: 2.0.7

IBM Cloud Pak foundational services version: 3.16.3 or later

Zen operand version: 4.4.2

Bug fixes

• Issue: Changing the route to the Cloud Pak for Data platform prevents the Zen operator from updating the control plane.

  Resolution: Upgrade IBM Cloud Pak foundational services to Version 3.16.3 or later fixes.

• Issue: If the Db2® Data Management Console service is installed, the Events and alerts page shows a critical alert when the Check replica status (check-replica-status) monitor runs.

  The monitor incorrectly indicates that there are an insufficient number of redis replicas associated with the Db2 Data Management Console service.

  Resolution: The Check replica status (check-replica-status) monitor has been modified. The monitor now takes any StatefulSet policies into account before issuing alerts.

Refresh 6 of Cloud Pak for Data Version 4.0

A new version of the Cloud Pak for Data was released in February 2022.

Platform operator version: Not refreshing. (Use Version 2.0.6, which was shipped with the previous refresh.)

IBM Cloud Pak foundational services version: 3.15.0 or later

Zen operand version: 4.4.0

Bug fixes

• Issue: On the Branding page, you cannot enter text in non-Latin characters, such as text that uses Chinese or Japanese characters.

  Resolution: You can now enter text on the Branding page in languages that use non-Latin characters. For example, you can now enter Chinese and Japanese text.

• Issue: When you create a data request, the Add to project option is disabled, so you cannot specify which project the request is associated with.

  Resolution: You can now select the project that the request is associated with.

Refresh 5 of Cloud Pak for Data Version 4.0

A new version of the Cloud Pak for Data was released in January 2022.

Platform operator version: 2.0.6.

IBM Cloud Pak foundational services version: 3.14.1 or later

Zen operand version: 4.3.2

Security fixes

Refresh 5 of Version 4.0 of the Cloud Pak for Data platform includes various fixes.

Refresh 4 of Cloud Pak for Data Version 4.0

A new version of Cloud Pak for Data was released in December 2021.

Platform operator version: Not refreshing. (Use Version 2.0.5, which was shipped with the previous refresh.)

IBM Cloud Pak foundational services version: 3.14.1 or later

Zen operand version: 4.3.2

New features

Refresh 4 of Version 4.0 of the Cloud Pak for Data platform includes the following features and updates to the control plane:

New method for creating the route to the platform

Cloud Pak for Data now offers two options to create a custom route to the platform. You can use the options separately or in tandem, depending on your needs.

For details, see Changing the route to the platform.

New option to set the install plan on automatically installed operators

Some Cloud Pak for Data services have dependencies that are automatically installed when you install the service. For example, when you install Watson™ Studio, Watson Studio automatically installs the following dependencies if they are not already available:

• Common core services
• Data Refinery

Previously, the operators for these dependencies were created with the automatic install plan (installPlanApproval: Automatic). Starting in Refresh 4 of Version 4.0, you can specify which install plan these operators use.

For details, see Specifying the install plan for operators that are automatically installed by Operand Deployment Lifecycle Manager.

Bug fixes

Refresh 4 of Version 4.0 of the Cloud Pak for Data platform includes various fixes.

Refresh 3 of Cloud Pak for Data Version 4.0

A new version of Cloud Pak for Data was released in November 2021.

Platform operator version: 2.0.5

IBM Cloud Pak foundational services version: 3.13.0 or later

Zen operand version: 4.3.1

This release includes the following changes:

New features

Refresh 3 of Version 4.0 of the Cloud Pak for Data platform includes the following features and updates:

Updated guidance for installing and upgrading

If you are pulling images directly from the IBM Entitled Registry, it is strongly recommended that you create catalog sources that pull specific versions of images from the registry, rather than using the IBM Operator Catalog.

The IBM Operator Catalog automatically updates the images in your environment, so it is recommended only for short-term installations, such as proof-of-concept deployments. In production and test environments, where predictability and stability are important, you can create catalogs sources that pull a specific version of the software images from the IBM Entitled Registry.

For details, see:

• Installations: Creating catalog sources
• Upgrades from Version 3.5, see Creating catalog sources
• Upgrades from Version 4.0.x, see Updating catalog sources

Managing secrets and vaults

Cloud Pak for Data services use sensitive information, such as database access credentials, API keys, and authentication tokens. You can use secrets and vaults to store this sensitive data.

By default, Cloud Pak for Data includes an internal vault that is accessible through the Credentials and Secrets API. However, a Red Hat® OpenShift® Container Platform project administrator can enable the vaults interface, which enables users to:

• Add secrets to the internal vault from the web client
• Use secrets from an external vault, such as CyberArk or HashiCorp

You can use these secrets when you create connections. However, not all services can use connections that use secrets. The following services support connections that use secrets from vaults:

• Data Virtualization
• Decision Optimization
• Execution Engine for Apache Hadoop
• OpenPages®
• Watson Knowledge Catalog
• Watson Studio Runtimes

For information on enabling Cloud Pak for Data to use vaults, see Configuring vault usage.

For information on adding secrets to the internal vault or connecting to external vaults, see Managing secrets and vaults.

Support for Power® 10 hardware

You can now install Cloud Pak for Data on Red Hat OpenShift Container Platform Version 4.8 clusters running on Power 10 hardware. However, the platform does not take advantage of Power 10 optimizations.

Bug fixes

Refresh 3 of Version 4.0 of the Cloud Pak for Data platform includes various fixes.

Refresh 2 of Cloud Pak for Data Version 4.0

A new version of Cloud Pak for Data was released in October 2021.

Platform operator version: 2.0.4

IBM Cloud Pak foundational services version: 3.12 or later

Zen operand version: 4.2.0

This release includes the following changes:

New features

Refresh 2 of Version 4.0 of the Cloud Pak for Data platform includes the following features and updates:

Support for Power and s390x hardware

You can now install Cloud Pak for Data on Red Hat OpenShift Container Platform Version 4.8 clusters running on:

• s390x (z14 or later)
• Power

Not all services support these environments. For a list of services that can be installed on Power or s390x hardware, see Hardware requirements.

Support for Red Hat OpenShift Container Platform Version 4.8

You can now install Cloud Pak for Data on Red Hat OpenShift Container Platform Version 4.8.

New backup and restore utility

This release also includes support for a new backup and restore utility, which is described in What's new and changed in the Cloud Pak for Data command-line interface.

Bug fixes

Refresh 2 of Version 4.0 of the Cloud Pak for Data platform includes various fixes.

Refresh 1 of Cloud Pak for Data Version 4.0

A new version of Cloud Pak for Data was released in August 2021.

Platform operator version: 2.0.3

IBM Cloud Pak foundational services version: 3.10.0 or later

Zen operand version: 4.1.0

This release includes the following changes:

New features

Refresh 1 of Version 4.0 of the Cloud Pak for Data platform includes the following features and updates:

Support for upgrade

You can now upgrade your Cloud Pak for Data installation:

• If you are running Cloud Pak for Data Version 3.0.1, you must upgrade your installation to Version 3.5. before you can upgrade to Version 4.0.x. For details on upgrading to Version 3.5, see Upgrading IBM Cloud Pak for Data control plane in the Version 3.5 documentation.
• If you are running Cloud Pak for Data Version 3.5.3, you can upgrade to Cloud Pak for Data Version Not applicable or later refreshes.
  Important: If you are running an earlier refresh of Cloud Pak for Data Version 3.5, you must upgrade to Version 3.5.3 or later before you upgrade to 4.0.x.
• If you are running Cloud Pak for Data Version 4.0.x, you can upgrade to the latest refresh.

After you upgrade Cloud Pak for Data, upgrade the services on your cluster.

Volumes file browser

The integrated file browser for storage volumes is no longer a tech preview. The file browser is now a fully supported feature in Cloud Pak for Data.

Bug fixes

Refresh 1 of Version 4.0 of the Cloud Pak for Data platform includes various fixes.

Initial release of Cloud Pak for Data Version 4.0

A new version of the control plane was released as part of Cloud Pak for Data Version 4.0.

Platform operator version: 2.0.0

IBM Cloud Pak foundational services version: 3.8.1 or later

Zen operand version: 4.0.1

This release includes the following changes:

Red Hat OpenShift Container Platform support

You can deploy Cloud Pak for Data Version 4.0 on the following versions of Red Hat OpenShift Container Platform:

• Version 4.6

Operator-based installation

The Cloud Pak for Data control plane and services are installed using operators, which simplify the process of upgrading, scaling, and rolling back software on Red Hat OpenShift Container Platform.

For an overview of operators, see the Red Hat OpenShift: Operators Framework video from Red Hat. (This video is also available on YouTube: https://www.youtube.com/watch?v=LymzLHRbQdk.)

For details on installing Cloud Pak for Data and Cloud Pak for Data services, see Installing Cloud Pak for Data.

Cloud Pak for Data Version 4.0 offers two installation methods for installing the control plane:

Express® installation

This method enables you to install the Cloud Pak for Data control plane with fewer manual steps. However, it offers less control over namespace scoping. Additionally, you must be a Red Hat OpenShift cluster administrator to use this installation method.

Specialized installation

This method provides more control over your environment. However, it requires additional manual steps. A Red Hat OpenShift project administrator can complete some of the steps after a cluster administrator completes the initial setup.

Both of the options are explained in more detail in:

• Operator installation architecture
• Installing Cloud Pak for Data

Fewer custom security context constraints

Starting in Cloud Pak for Data Version 4.0, many services can use the default restricted security context constraint (SCC).

However, some services still require custom SCCs. For details about which services require custom SCCs, see Creating custom security context constraints for services.

Adoption of IBM Cloud Pak foundational services

In previous releases of Cloud Pak for Data, you could optionally install the IBM Cloud Pak foundational services on your cluster to integrate IAM Service and the License Service.

In Cloud Pak for Data Version 4.0, the IBM Cloud Pak foundational services are a prerequisite and Cloud Pak for Data integrates with additional features from IBM Cloud Pak foundational services:

Operand Deployment Lifecycle Manager (ODLM)

ODLM enables Cloud Pak for Data services to orchestrate and manage their dependencies.

For details about ODLM, see the IBM/operand-deployment-lifecycle-manager repository on GitHub.

Namespace Scope Operator

The Namespace Scope Operator enables the Cloud Pak for Data operators and the IBM Cloud Pak foundational services operators to be deployed in the same namespace and to manage the namespaces where Cloud Pak for Data services are deployed.

The Namespace Scope Operator can be used in tandem with the ownNamespace operator group to improve security in shared clusters. Combining these means that you do not need to grant cluster-wide authority to the Cloud Pak for Data operators and the IBM Cloud Pak foundational services operators.

For details about the Namespace Scope Operator, see:

• The IBM/ibm-namespace-scope-operator repository on GitHub
• Authorizing foundational services to perform operations on workloads in a namespace in the IBM Cloud Pak foundational services documentation.

Certificate manager

Cloud Pak for Data uses the Certificate manager to generate

• TLS root CA Certificates
• TLS certificates and keys for secure internal communications between microservices.

For details, see IBM Certificate manager in the IBM Cloud Pak foundational services documentation.

Identity and Access Management Service (IAM Service)

The IAM Service enables Cloud Pak for Data to use multiple identity providers for authentication. Additionally, the service enables single sign-on across multiple IBM Cloud Pak installations.

By default, Cloud Pak for Data is not integrated with the IAM Service. Review Security on Cloud Pak for Data before you enable Cloud Pak for Data to use the IAM Service.

New features

Enhanced platform monitoring interface

In Cloud Pak for Data Version 4.0, the Platform management page has been updated and renamed to Monitoring. In addition to the status and quota information for the platform, services, service instances, environments, and pods, the page now includes:

Event and alert information

By default, Cloud Pak for Data records information for the following events:

• A persistent volume claim (PVC) is unbound. (In other words, there is no volume that meets the specifications of the PVC.)
• A StatefulSet or deployment has unavailable replicas.
• A service reaches or exceeds the vCPU or memory alert threshold
• A service reaches or exceeds the vCPU or memory quota

However, you can create custom monitors. A monitor is that checks the state of an entity periodically and generate events.

Additionally, Cloud Pak for Data issues alerts for events that are in warning or critical state for a specific period of time.

The events and alerts are accessible from the Events card on the Monitoring page. For details, see Managing the platform.

Historical use data

By default, Cloud Pak for Data stores the vCPU and memory use data for the last 30 days. You can see the platform resource use data from the last 12 hours from the Monitoring page.

From the Monitoring page, you can access the Status and use page, where you can see up to 72 hours worth of data. On the Status and use page, you can see the resource use by service, services instances, environments, and pods.

For details, see Managing the platform.

User management enhancements

Cloud Pak for Data Version 4.0 includes several enhancements to user management:

More details about permissions

The Roles page now provides more information about the actions that a user can take when they have a specific permission.

New permissions for creating and managing projects and deployment spaces

Depending on the services that you have installed, the Roles page includes the following permissions:

Project permissions

• Create projects
• Manage projects
• Monitor project workloads

Deployment space permissions

• Create deployment spaces
• Manage deployment spaces
• Monitor deployment activity

Updated catalog permissions

The catalog permissions have been updated to separate the creation of catalogs from the management of catalogs. Users with the Create catalog permission can only create catalogs. Users with the Manage catalogs permission can:

• Create catalogs
• View list of all catalogs
• Join any catalog as an Admin
• Reconfigure the default catalog

Updated administration permissions

The administrative permissions have been updated to provide more granular control. Any permissions that are associated with the default admin user are now associated with one of the following permissions:

Platform administration permissions

• Administer platform
• Manage configurations
• Manage platform health
• View platform health

User administration permissions

• Manage platform roles
• Manage user groups
• Manage users

These changes ensure that you do not need to retain the default admin user to complete specific tasks.

For details, see Predefined roles and permissions.

In addition, an administer with the appropriate permissions to manage users can use the View assigned permissions button to see a complete list of the permissions that a user has based on their assigned roles. For details, see Managing users.

Create platform connections that use shared credentials

When you create a platform connection, you can optionally use shared credentials. With shared credentials, all users use the same credentials to access the connection. Previously all platform connections required personal credentials. With personal credentials, each user must specify their own credentials to access the connection. For details, see Connecting to data sources.

Important: This feature is enabled by default. If you do not want to allow users to specify shared credentials, you must update your settings. For details, see Changing shared credentials settings.

Enhanced connections interface

When you create a new connection, the interface includes enhancements that make it easier to create the connection:

• The Provider filter enables you to identify IBM data sources, third-party data sources, and user-defined data source types.
• The Compatible services filter enables you to easily find the connection types that you can use with a specific service.

Alternatively, if you know the name of the connection type that you are looking for, you can enter it in the Find field.

Deprecated features

Support for Red Hat OpenShift Container Platform Version 3.11

You cannot install Cloud Pak for Data Version 4.0 on Red Hat OpenShift Container Platform 3.11.

If you have an existing cluster on Red Hat OpenShift Container Platform 3.11, you must upgrade to Red Hat OpenShift Container Platform 4.6.

Support for Red Hat OpenShift Container Platform 4.5

You cannot install Cloud Pak for Data Version 4.0 on Red Hat OpenShift Container Platform 4.5.

If you have an existing cluster on Red Hat OpenShift Container Platform 4.5, you must upgrade to Red Hat OpenShift Container Platform 4.6.

Services that are not releasing on Cloud Pak for Data Version 4.0

The following services cannot be deployed on Cloud Pak for Data Version 4.0:

• Db2 for z/OS® Connector
• Edge Analytics
• Financial Crimes Insight®
• Master Data Connect
• Streams
• Streams Flows
• Watson Knowledge Studio
• Watson Language Translator

If you need to use any of these services, you must use Cloud Pak for Data Version 3.5.