Transformation Advisor Prerequisites

Prerequisites

IBM Operator Catalog

IMPORTANT NOTE ON CATALOGS: Transformation Advisor 2.3.x was delivered in the Red Hat Certified catalog and has now been removed from there. Transformation Advisor 2.4+ is only be available in the IBM Operator Catalog. Upgrading from 2.3 to 2.4 is not automatic. See Upgrading for more on how to upgrade from 2.3 to 2.4. When you have the IBM Operator Catalog installed and you search for "Transformation Advisor" the OperatorHub UI, you will see two operators appear - one from the IBM Operator Catalog, and one from the Red Hat Certified Catalog. Please select the Provider Type checkbox called IBM Operator Catalog in the OperatorHub UI to ensure that you are installing the latest version of Transformation Advisor from the IBM Operator Catalog.

Transformation Advisor is available in the IBM Operator Catalog. See IBM Operator Catalog documentation for more details.

When installing Transformation Advisor using the CASE actions, there is an option --installIbmCatalog which will automatically install the IBM operator catalog if it is not already installed on your cluster.

When installing Transformation Advisor from the UI, you will need to follow the instructions IBM Operator Catalog documentation to install the catalog.

Running on a FIPS enabled environment

If you are installing Transformation Advisor on an environment where FIPS is enabled you will need to set the FIPS configuration when installing the Transformation Advisor instance. For a UI installation, there is a FIPS Mode Enable toggle on the create instance UI form. For a CASE installation, you need to pass the --fips true argument to the install command.

Cluster Admin Permissions

To install Transformation Advisor, you need to perform some tasks as a user with cluster admin permissions.

Please refer to the Installing documentation for more details.

Access Transformation Advisor images from the entitled registry

Transformation Advisor can be installed with a time limited evaluation license (fully featured), or as an entitled product. Please read the license for more details:

Entitlement to Transformation Advisor is obtained by purchasing one of a number of other IBM products. To find your entitlement key you must first ensure that you have entitlement to any one of the IBM products listed here. Then log in to MyIBM Container Software Library with the IBMid and password that are associated with the entitled software. Your Entitlement Key will be displayed there.

Secret needs to be created in the same namespace where the product instance is installed or globally.

For a namespaced install create an image pull secret called ibm-entitlement-key using your entitlement key as the password, cp as the username, and cp.icr.io as the docker server for use with this deployment. See https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-by-providing-credentials-on-the-command-line for how to create a pull secret.

Alternatively you can update the cluster's global pull secret using your entitlement key as the password, cp as the username, and cp.icr.io as the docker server for use with this deployment. See https://docs.openshift.com/container-platform/4.9/openshift_images/managing_images/using-image-pull-secrets.html#images-update-global-pull-secret_using-image-pull-secrets for how to update the global pull secret.

Notice: Migration to IBM Cloud Container Registry

As of Transformation Advisor 2.4.3, some of the Transformation Advisor images (the operator, bundle and catalog) were migrated from Docker Hub (docker.io/ibmcom) to the IBM Cloud Container Registry (icr.io/cpopen).

This change affects all of your IBM Operator Catalog provided software, not just Transformation Advisor. Please refer to the following documentation for full details of this change: Migrating from Docker to the IBM Container Registry

For Transformation Advisor, the following are the changes that are required:

If you operate in an internet-connected Red Hat® OpenShift® Container Platform cluster:

If you operate in an air-gapped Red Hat OpenShift Container Platform cluster:

Persistent Storage

NOTE: The following section is a guide to how you can configure persistence for Transformation Advisor. It is not a thorough guide to storage on a Kubernetes in general. Please refer to the Kubernetes documentation if you are not familiar with concepts mentioned in this section. Refer to Red Hat OpenShift documentation and your storage administrator to understand the persistence options available on your specific version of OpenShift.

Transformation Advisor requires persistence to be configured in order to persist the database data. When Transformation Advisor starts for the first time, it will create a PersistentVolumeClaim (unless you have set the option to use an existing claim). The persistence options provided by Transformation Advisor, allow you to control the attributes of the PersistentVolumeClaim that Transformation Advisor creates. The PersistentVolumeClaim that Transformation Advisor creates, must bind to a PersistentVolume on the cluster. The properties set for the PersistentVolumeClaim must be compatible with the PersistentVolumes available on your cluster.

From Transformation Advisor 3.0.x onwards, two PersistentVolumeClaims are required.

The persistence may be configured from the UI form view or the YAML view when installing the Transformation Advisor instance.

It is NOT recommended to use local volumes as backing for persistence for Transformation Advisor.

Transformation Advisor Persistence Options

The following properties can be configured to control the persistence that Transformation Advisor uses. See Example Persistence Configurations to see how to use these configuration options.

Enable / Disable

Configuration setting: persistence.enabled: <true|false>

Although it is NOT recommended for a production install, you can also turn the persistence off. Transformation Advisor will have full functionality with persistence disabled except importantly, if persistence is off, you will lose all of your Transformation Advisor data when the database container restarts.

Access modes

The access mode determines the PersistentVolume that the Transformation Advisor PersistentVolumeClaim can bind to. The value for access mode must be supported by the type of storage that is available on your cluster. For example, if you are installing Transformation Advisor on public cloud, the accessMode is ReadWriteOnce (RWO). Consult your storage administrator for more details.

Configuration setting: persistence.accessMode: <"ReadWriteOnce">

Transformation Advisor supports ReadWriteOnce (RWO) access mode. ReadWriteMany (RWX) and ReadOnlyMany (ROX) access modes are not supported by Transformation Advisor.

Storage class

Configuration setting: persistence.storageClassName: <"storage class name">

A StorageClass provides a way for administrators to describe the "classes" of storage they offer. If the PersistentVolumes configured for your cluster have a storage class, then you can set the storageClassName property in Transformation Advisor, so that PersistentVolumeClaim that Transformation Advisor creates will bind to those PersistentVolumes.

If you are configuring persistence in the UI form view, the Storage Class widget will be pre-populated with the StorageClasses available on your cluster. Alternatively, you can navigate to storage classes in the navigation in OpenShift UI to see what storage class are available on your cluster.

Alternatively, you can use the following command to see the storage classes available.

oc get storageclass

Storage type and permissions

If you are installing on an environment that uses block storage - (specified using a storageClass), Transformation Advisor can bind to the storage and set the correct permissions for that storage. If you are installing on an environment that uses file system storage such as NFS, you must ensure that Transformation Advisor has read/write permissions to the storage on the host. For details, see Setting permissions for NFS.

If there are not sufficient permissions for the storage that Transformation Advisor uses, the CouchDB pod fails to start and a permission denied message is added to the couch db pod logs:

[error] 2019-08-23T21:08:33.974674Z nonode@nohost <0.256.0>
-------- Could not open file ./data/_users.couch: permission denied

Setting permissions for NFS

Note: From Transformation Advisor 3.0.x onwards, two PersistentVolumeClaims are required, and you need to repeat the following procedures for both PersistentVolmueClaims accordingly. When shared storage is mounted into a container, it is mounted with the same ownership and permissions that are found on the exported NFS directory. The container that uses the storage is not run with that owner and might not have permission to read or write to the storage. To enable read/write permission to the storage, consider the following options:

Option A: Control permissions with the supplementalGroups setting

Configuration setting: persistence.supplementalGroups: <[id1,id2...]>

Supplemental groups are regular Linux groups and are typically used for controlling access to shared storage, such as NFS. When a process runs in Linux, it has a UID, a GID, and one or more supplemental groups. The supplemental groups can be set for a container’s main process. The supplemental group gives the container process group ownership on the storage and consequently the group level permissions to allow the container process to read/write to the storage.

Consider the following example where the underlying host storage is located at /netstore/transadv/. The nfsnobody group might have full permissions on the shared directory when NFS is shared with the root_squash option:

> ls -ld /netstore/transadv/
drwxrwx---. 7 nfsnobody nfsnobody 182 Aug 31 22:30 /netstore/transadv/

To control permissions with the supplementalGroups setting, and give the container the ability read/write to the storage, add the nfsnobody GID to the supplemental groups as follows:

  1. Get the nfsnobody GID using the id command.

    > id nfsnobody
uid=65534(nfsnobody) gid=65534(nfsnobody) groups=65534(nfsnobody)

  2. Add the GID (65534 in this case) using the supplementalGroups setting, e.g. configuration.

    persistence:
...
<couchdb | neo4j>
supplementalGroups: [65534]
Option B: Create a PersistentVolume specifically for use with Transformation Advisor

Configuration setting: persistence.existingClaim: <"claim name">

Creating a PersistentVolume specifically for Transformation Advisor allows you to set the access permissions for storage usedby that PersistentVolume.

This option describes how to:

  1. Create a NFS PersistentVolume specifically for Transformation Advisor
  2. Create a PersistentVolumeClaim to bind to that PersistentVolume
  3. Use that claim (with appropriate permissions) for Transformation Advisor

We will manually create a PersistentVolumeClaim and ensure that the storage backing its PersistentVolume has the correct (open) permissions. You can ensure a PersistentVolumeClaim binds to a specific PersistentVolume with selector and label attributes.

  1. Define an NFS PersistentVolume. The following sample is an NFS PersistentVolume definition. Substitute the path and server values for your own environment.

    apiVersion: v1
kind: PersistentVolume
metadata:
name: tanfspv
labels:
pvc_for_app: "ta"
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
nfs:
path: /nfs/ta_storage
server: xxx.xxx.xxx.xxx
persistentVolumeReclaimPolicy: Recycle

    Make sure that the directory at the path has open permissions so that the container can read/write there:

    chmod -R 777 /netstore/transadvopen/

  2. Define a PersistentVolumeClaim. The following sample is a PersistentVolumeClaim definition that binds to the PersistentVolume described in the previous sample. Ensure that you create the PersistentVolumeClaim in the Transformation Advisor namespace.

    apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: tapvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
  storage: 20Gi
selector:
matchLabels:
  pvc_for_app: "ta"

If you want Transformation Advisor to use a specific PersistentVolumeClaim, add the name of the claim using the existingClaim property.

Option C: Update permissions for the PersistenceVolume after Transformation Advisor has tried and failed to start

If you are using NFS storage, and you do not complete option A or B, when Transformation Advisor attempts to start - you will see that the Couch DB pod will not start. If you look at the logs for the Couch DB pod you will see a permissions issue:

[error] 2019-08-23T21:08:33.974674Z nonode@nohost <0.256.0>
-------- Could not open file ./data/_users.couch: permission denied

At this point, you can find the PersistentVolumeClaim that Transformation Advisor has created:

oc get pvc

That will reveal the PersistentVolume that it has bound to (under the VOLUME header).

You can describe that PersistentVolume to discover the path for the storage:

oc describe pv tanfspv | grep Path:

You can then open the permissions on that path chmod -R 777 <path>

The Couch DB should then restart sucessfully within a few minutes. Or, you can delete the Couch DB pod and it will be automatically restarted.

Storage Classes Red Hat OpenShift Container Platform on IBM Cloud

IBM Cloud provides a number of storage classes for OpenShift clusters. Some of the storage classes use block storage and can be identified by the "-block-" in their name. Other storage classes use file storage and can be identified by the "-file-" in their name. The storage class with the name "default" uses file storage also. Please note that the storage with the name "default" may be different to the storage class that is marked as the default storage class for the cluster.

Transformation Advisor supports the use of any of the block based storage classes. Transformation advisor does not support the use of the file based storage (including the storage class called "default"), except for the storage classes that have a "-gid" suffix, i.e.

These storage classes allow you to specify a supplemental group when installing so that the Transformation Advisor non-root containers can write to the storage. When using these *-gid storage classes, please specify the supplementalGroups of 65531 at install time.

Please refer to the following document for more information on storage classes on Red Hat OpenShift on IBM Cloud: Storage class reference.

Example Persistence Configurations

Please refer to the Installing document on how to apply these example configurations.

EXAMPLE 1: Using storageClass

Use the storage class to create a PersistentVolumeClaim that will bind to a PersistentVolumeClaim of that class. In this case, we are using "ibmc-block-silver" on public cloud.

For a UI install, the custom resource YAML would look like this:

...
  persistence:
    enabled: true
    <couchdb | neo4j>
      accessMode: "ReadWriteOnce"
      size: 20Gi
      useDynamicProvisioning: true
      existingClaim: ""
      storageClassName: "ibmc-block-silver"
      supplementalGroups: []
...

For a CASE install the arguments would look like this:

--args "... --storageClass ibmc-block-silver --accessMode ReadWriteOnce ..."

EXAMPLE 2: Using supplementalGroups

In the following example, we are using the supplementalGoups setting to ensure the database pods have sufficient permissions on NFS storage we are using. The group ownership of the shared file system has a GID of 65534

For a UI install, the custom resource YAML would look like this:

...
  persistence:
    enabled: true
    <couchdb | neo4j>
      accessMode: "ReadWriteOnce"
      size: 20Gi
      useDynamicProvisioning: true
      existingClaim: ""
      storageClassName: ""
      supplementalGroups: [65534]
...

For a CASE install the arguments would look like this:

--args "... --supplementalGroups [65534] ..."

EXAMPLE 3: Using existingClaim

In the following example, a persistence volume claim called my-ta-pvc has been already set up by the user. The persistence volume claim has open permissions to allow the database container to write to it.

For a UI install, the custom resource YAML would look like this:

...
  persistence:
    enabled: true
    <couchdb | neo4j>
      accessMode: "ReadWriteOnce"
      size: 20Gi
      useDynamicProvisioning: true
      existingClaim: "my-ta-pvc"
      storageClassName: ""
      supplementalGroups: []
...

For a CASE install the arguments would look like this:

--args "... --persistenceClaimCouchDB my-ta-pvc --persistenceClaimNeo4j my-ta-pvc2..."

Deleting a Persistence Volume Claim:

NOTE: Do not delete PersistentVolumes or PersistentVolumeClaims unless you are entirely satisfied that they are not being used by containers. Deleting PersistentVolumes or PersistentVolumeClaims may interfere with running applications and/or delete data that cannot be recovered.

In a normal uninstall of Transformation Advisor, the PersistentVolumeClaim that Transformation Advisor has created will be deleted, and the PersistentVolume that the claim binds to will be released for use.

If you need to manually delete PersistentVolumes or PersistentVolumeClaims please consult the relevant Kubernetes documentation

Image Registry Access

The images required for Transformation Advisor are hosted in the following locations:

- icr.io
- cp.icr.io

Your cluster will need access to these registries. If you operate block or allow lists for registries in your cluster, then you may need to update the image.config.openshift.io/cluster resource accordingly to allow access to icr.io and cp.icr.io

Accessing the Installed Transformation Advisor Product

Transformation Advisor leverages the OpenShift APIs to perform authentication, and as such, will authenticate against the configured identity provider on the cluster. Any user wishing to access Transformation Advisor, must have valid credentials. Anonymous access is not supported.

Resources Required

This section describes the resources required to install and run Transformation Advisor. Choose a configuration profile that aligns with your usage, and allocate resources to Transformation Advisor accordingly. Please note that Transformation Advisor does not support a High Availability configuration.

Configuration Profile Definitions

Configuration Profile Name Usage
Starter minimum size for functioning product, ie demo use case
Production recommended size for production environment

Starter Profile Configuration

Subsystem CPU Memory (GB) Disk Space (GB)
CouchDB 0.5 1 20
GraphDB 0.5 1 5
Server 0.5 1 N/A
UI 0.5 1 N/A
Operator 0.4 0.5 N/A

Production Profile Configuration

Subsystem CPU Memory (GB) Disk Space (GB)
CouchDB 1.5 3 20
GraphDB 1.5 3 5
Server 1.5 3 N/A
UI 1.5 3 N/A
Operator 0.5 0.5 N/A

These values can be modified in the custom resource yaml prior to install. See Configuring for more details.