Isolated migration
Isolated migration will first isolate the shared 3.x version of foundational services on the cluster if needed, then fresh install a new instance of foundational services version 4.x. Any data from the original foundational services will be copied to the new instance.
This method can be used if the foundational services you are migrating is shared for the entire cluster or dedicated to a IBM Cloud Pak.
Note: You can choose to migrate by either using a script or using manual steps.
Prerequisite
- See supported starting versions in Migrating foundational services v3.x to 4.x.
- Download scripts and go to the
installer_scripts
directory. For more information, see Downloading scripts for additional configuration from specific version CASE bundle. - Create new CatalogSource or update existing CatalogSource to the new 4.x version.
- For online migration, reference the required CatalogSource for the corresponding version.
- For offline migration, reference the upgrade steps to mirror images, and apply the new CatalogSource.
Migrate using a script
Step 1: Isolate and migrate
-
Isolate the original instance of foundational services by running the
isolate.sh
script.Run
./isolate.sh --help
for usage instructions. For example, CloudPak instances incpns1
andcpns2
sharing the same foundational services instance inibm-common-services
, createzenservice
in bothcpns1
,cpns2
, andzen-tenant
namespace. When you want to isolatecpns1
and upgrade later, run the following script:./isolate.sh --original-cs-ns ibm-common-services --control-ns cs-control-ns --excluded-ns cpns1,zen-tenant
-
Migrate cluster singletons by running the
./cp3pt0-deployment/setup_singleton.sh
script.Run
./cp3pt0-deployment/setup_singleton.sh --help
for usage instructions. For example, foundational services is installed in theibm-common-services
namespace and user-createdibm-cert-manager-catalog
catalogsource, andibm-licensing-catalog
in theopenshift-marketplace
namespace. Run the following script:./cp3pt0-deployment/setup_singleton.sh --operator-namespace ibm-common-services --enable-licensing --cert-manager-source ibm-cert-manager-catalog --licensing-source ibm-licensing-catalog --license-accept -v 1
If you have deployed License Service Reporter, use the following command to migrate the IBM License Service Reporter:
./cp3pt0-deployment/setup_singleton.sh --operator-namespace ibm-common-services --enable-licensing --enable-license-service-reporter --cert-manager-source ibm-cert-manager-catalog --licensing-source ibm-licensing-catalog --license-accept -v 1
-
Define the foundational services version 4.x topology by running the
./cp3pt0-deployment/common/authorize-namespace.sh
script.Note: You must create the operator namespace and the services namespace for the new topology before running
authorize-namespace.sh
.
Step 2: Preload data
This step will copy data in the isolated original foundational services version 3.x scope and preload it into the foundational services version 4.x tenant's services namespace, and will be ready to be used by the new foundational services version 4.x services whenever they are installed. Such data includes, but not limited to:
- IM admin login credentials
- Mongo data containing IM users
Run the preload_data.sh
script. Make sure to pass in the original common services namespace from the original tenant and the services namespace from the new tenant as parameters.
For example: ./preload_data.sh --original-cs-ns <original cs ns> --services-ns <new services ns>
. For more details, use the -h
option in the script.
See the following notes:
- Security note: Due to the limitations of the community provided mongodump and mongorestore backup and restore tools for MongoDB, during the duration of the preload_data.sh script's runtime, there are non-SSL connections made to the MongoDB instance targeted by the script on Power and Z clusters. During this time, the running MongoDB instance is changed from requireSSL to preferSSL to facilitate the migration of MongoDB data. Once the script has successfully completed, the MongoDB instance is reverted back to requireSSL and all further connections are required to be SSL.
- If new data is introduced after the data has already been copied and before the IBM Cloud Pak has been upgraded, the upgraded IBM Cloud Pak will not have this newly introduced data. If you do something to introduce data while this upgrade process is in progress, the newly introduced data may not be reflected after IBM Cloud Pak upgrade.
Performing the isolated upgrade during a maintenance window is recommended to reduce the risk of this new data loss.
Step 3: Fresh install foundational services version 4.x
- Decide whether you want a simple topology or advanced topology for the foundational services version 4.x tenant.
- If you want a simple topology, install or upgrade
common-service-operator
v4 into the one and only namespace for your foundational services version 4.x tenant. -
If you want an advanced topology, running
./cp3pt0-deployment/setup_tenant.sh
script../cp3pt0-deployment/setup_tenant.sh --operator-namespace <foundational-services-v4-operators-ns> --services-namespace <foundational-services-v4-operands-ns-if-needed> --tethered-namespace <to-be-upgrade-Cloud-Pak-ns> --license-accept
-
Verify that
common-service-operator
isRunning
. - Verify that
common-service-maps
has a new tenant with namespaces for foundational services version 4.x and the to-be-upgraded IBM Cloud Pak.
Step 4: Upgrade IBM Cloud Pak
At this point, the cluster state is as follows:
- Isolated IBM Cloud Pak and foundational services workloads running
- New foundational services version 4.x tenant installed with just
common-service-operator
, ODLM, and NSS, and any of the to-be-upgraded IBM Cloud Pak services -
No IM or other foundational services installed yet
-
Change channel of IBM Cloud Pak operator(s).
- Verify new or updated OperandRequests exist, for example, requesting
ibm-im-operator
,ibm-platformui-operator
, and so on.- IBM Cloud Pak services are responsible for ensuring OperandRequests are updated
- If there is
ZenService
CR in IBM Cloud Pak namespace(s), remove thecsNamespace
field if it exists. - Verify that IBM Cloud Pak works.
If the upgraded IBM Cloud Pak uses SAML or OIDC for single sign-on or if you have configured this, it must be re-registered and the new IM instance will need to be re-configured since the cp-console
route and other foundational services
routes would have changed due to this instance of foundational services being a fresh installation. For more information, see Migrating identity management.
Migrate using manual steps
Step 1: Isolate original instance of foundational services
Step 1 involves isolating the original foundational services so that it will not reconcile, manage, or interfere with the foundational services version 4.x workloads on the cluster. If there are already dedicated foundational services instances, this step is unnecessary.
Important: Only perform this step if the cluster is using a shared instance of foundational services. The cluster is using a shared instance of foundational services if the common-service-maps
configmap in kube-public
namespace does not have controlNamespace
specified.
-
Edit the CSV of common-service-operator and ODLM and set
replicas: 0
to scale them down:oc scale deployment -n <namespace> ibm-common-service-operator --replicas=0 oc scale deployment -n <namespace> operand-deployment-lifecycle-manager --replicas=0
-
Create the configmap,
common-service-maps
, in kube-public namespace if it does not exist already.- Add namespaces of all running IBM Cloud Paks on the cluster to
common-service-maps
as one foundational services version 3.x tenant. - Exclude the IBM Cloud Pak namespaces which will be upgraded to foundational services version 4.x.
- Add namespace of where the shared instance of foundational services is running to the same foundational services version 3.x tenant.
Example
common-service-maps
to isolate original instance of foundational services:kind: ConfigMap apiVersion: v1 metadata: name: common-service-maps namespace: kube-public data: common-service-maps.yaml: | controlNamespace: cs-control namespaceMapping: - requested-from-namespace: - cp1-ns1 - cp2-ns1 - cp2-ns2 - ibm-common-services map-to-common-service-namespace: ibm-common-services
- Add namespaces of all running IBM Cloud Paks on the cluster to
-
Edit the ODLM subscription and change the
ISOLATED_MODE: 'false'
toISOLATED_MODE: 'true'
Note: yq version 4 or later is required.
oc get subscription.operators.coreos.com operand-deployment-lifecycle-manager-app -n <namespace> -o yaml > sub.yaml yq e '.spec.config.env |= (map(select(.name == "ISOLATED_MODE").value |= "true") + [{"name": "ISOLATED_MODE", "value": "true"}] | unique_by(.name))' sub.yaml -i oc apply -f sub.yaml
-
Delete OperandRegistry and OperandConfig, both called
common-service
, in the foundational services namespace:oc delete operandregistry -n <namespace> --ignore-not-found common-service oc delete operandconfig -n <namespace> --ignore-not-found common-service
-
Delete all NSS CRs from the namespace where ODLM is running
Tip: If the
oc
command is stuck here, remove the finalizer in the NSS CRs.oc delete namespacescopes.operator.ibm.com nss-managedby-odlm odlm-scope-managedby-odlm nss-odlm-scope common-service -n <namespace>
The tenant in the common-service-maps
refers to one array element under the namespaceMapping
field as shown in the preceding example.
Step 2: Migrate cluster singletons
Step 2 is necessary because a dedicated instance of foundational services version 3.x requires that singletons be installed in controlNamespace
, hence they must be migrated (uninstalled and reinstalled) from ibm-common-services
to controlNamespace
.
-
Create the namespace specified in
controlNamespace
if it does not exist already.oc create namespace cs-control
-
Create configmap
ibm-cpp-config
incontrolNamespace
and adddeployCSCertManagerOperand: "false"
to the data. -
Delete CertManager CR.
oc delete certmanagers.operator.ibm.com default
-
Delete the
ibm-cert-manager-operator
subscription and CSV.oc delete subscription.operators.coreos.com ibm-cert-manager-operator -n <namespace> export CSV=$(oc get csv -n <namespace> | grep ibm-cert-manager | awk '{print $1}') oc delete ClusterServiceVersion $CSV -n <namespace>
-
To migrate License Service Reporter, complete the following steps:
-
Decouple the License Service Reporter Persistent Volume (PV) and Persistant Volume Claim (PVC).
-
Check the status of the License Service Reporter PVC.
oc get pvc license-service-reporter-pvc --ignore-not-found -n ibm-common-services --no-headers | awk '{print $2}'
As a reply, you get the PVC status, for example
Bound
. -
If the PVC is in
Bound
status, get the name of the License Service Reporter PV.VOL=$(oc get pvc license-service-reporter-pvc --ignore-not-found -n $ibm-common-services -o=jsonpath='{.spec.volumeName}')
-
Change the PV's
persistentVolumeReclaimPolicy
toRetain
and label the License Service Reporter PV asLSR PV
to enable further License Service Reporter upgrade.oc patch pv $VOL -p '{"spec": { "persistentVolumeReclaimPolicy" : "Retain" }}' oc label pv $VOL license-service-reporter-pv=true --overwrite
-
-
Delete the
IBMLicenseServiceReporter
instance.oc delete IBMLicenseServiceReporter instance -n ibm-common-services
-
Migrate License Service Reporter to a new namespace.
-
Create a new namespace for License Service Reporter.
oc create namespace ${LSR_NAMESPACE}
-
Get the
StorageClass
from the existing License Service Reporter PV.Note: You do not need to retrieve the
StorageClass
on IBM ROKS.LSR_PV_NAME=$(oc get pv -l license-service-reporter-pv=true -o=jsonpath='{.items[0].metadata.name}') LSR_STORAGE_CLASS=$(oc get pv -l license-service-reporter-pv=true -o=jsonpath='{.items[0].spec.storageClassName}')
-
Create a new PVC with the existing PV in the new License Service Reporter namespace.
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: license-service-reporter-pvc namespace: ${LSR_NAMESPACE} spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: "${LSR_STORAGE_CLASS}" volumeMode: Filesystem volumeName: ${LSR_PV_NAME}
-
-
Reinstall License Service Reporter operator in the new namespace. For more information, see Installing License Service Reporter from online repository.
-
-
Delete licensing CR.
oc delete ibmlicensing instance
-
Delete ibm-license-operator Subscription and CSV.
oc delete subscription.operators.coreos.com ibm-licensing-operator -n <namespace> export CSV=$(oc get csv -n <namespace> | grep ibm-licensing | awk '{print $1}') oc delete ClusterServiceVersion $CSV -n <namespace>
-
Copy licensing data to
controlNamespace
.The following is a list of possible licensing configmaps to copy:
ibm-licensing-config ibm-licensing-annotations ibm-licensing-products ibm-licensing-products-vpc-hour ibm-licensing-cloudpaks ibm-licensing-products-groups ibm-licensing-cloudpaks-groups ibm-licensing-cloudpaks-metrics ibm-licensing-products-metrics ibm-licensing-products-metrics-groups ibm-licensing-cloudpaks-metrics-groups ibm-licensing-services
-
Delete the two Crossplane Subscriptions and CR, if they exist.
oc get configuration.pkg.ibm.crossplane.io -A --no-headers | awk '{print $1}' | xargs ${OC} delete --ignore-not-found configuration.pkg.ibm.crossplane.io oc get lock.pkg.ibm.crossplane.io -A --no-headers | awk '{print $1}' | xargs ${OC} delete --ignore-not-found lock.pkg.ibm.crossplane.io oc get ProviderConfig -A --no-headers | awk '{print $1}' | xargs ${OC} delete --ignore-not-found ProviderConfig oc delete subscription.operators.coreos.com ibm-crossplane-provider-kubernetes-operator-app -n <control-namespace> oc delete subscription.operators.coreos.com ibm-crossplane-provider-ibm-cloud-operator-app -n <control-namespace> oc delete subscription.operators.coreos.com ibm-crossplane-operator-app -n <control-namespace>
-
Delete the Crossplane CSV.
export CSV=$(oc get csv -n <namespace> | grep ibm-crossplane-provider | awk '{print $1}') oc delete ClusterServiceVersion $CSV -n <namespace> export CSV=$(oc get csv -n <namespace> | grep ibm-crossplane-operator | awk '{print $1}') oc delete ClusterServiceVersion $CSV -n <namespace>
-
Verify that
ibm-cert-manager-operator
,ibm-licensing-operator
, andcrossplane
operators are all installed incontrolNamespace
.- Note: Licensing and crossplane only apply if they existed before deletion.
-
Install global
cert-manager
(ibm-cert-manager-operator v4 or Red Hat cert-manager). See Installing IBM Cert Manager by Console or see Red Hat's instructions. -
Update the channel of
ibm-licensing-operator
Subscription tov4.2
or reinstallibm-licensing-operator
v4.2 in target namespace. -
Scale back up the
common-service-operator
and ODLM.oc scale deployment -n <namespace> ibm-common-service-operator --replicas=1 oc scale deployment -n <namespace> operand-deployment-lifecycle-manager -- replicas=1
Example ibm-cpp-config
:
kind: ConfigMap
apiVersion: v1
metadata:
name: ibm-cpp-config
namespace: ibm-common-services
data:
deployCSCertManagerOperands: "false"
Step 3: Defining foundational services version 4.x topology
Important: Do not modify the common-service-maps
in this step.
- Create the namespace for foundational services version 4.x
-
Optional Do not do this step if the IBM Cloud Pak installs everything into one namespace
a. Create all the namespaces necessary for the foundational services version 4.x workloads and IBM Cloud Pak's workloads
b. Pre-emptively authorize
namespace-scope-operator
(not installed yet) to manage all Cloud Pak namespaces from the foundational services namespace by creating the followingRole
andRoleBinding
in each of the namespaces created in step 3.2
For example:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: nss-managed-role-from-<namespace_where_namespace-scope-operator_is_installed>
rules:
- apiGroups:
- "*"
resources:
- "*"
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- deletecollection
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nss-managed-role-from-<namespace_where_namespace-scope-operator_is_installed>
subjects:
- kind: ServiceAccount
name: ibm-namespace-scope-operator
namespace: <namespace_where_namespace-scope-operator_is_installed>
roleRef:
kind: Role
name: nss-managed-role-from-<namespace_where_namespace-scope-operator_is_installed>
apiGroup: rbac.authorization.k8s.io
Step 4: Preload data
This step will copy data in the isolated original foundational services version 3.x scope and preload it into the foundational services version 4.x tenant's services namespace, and will be ready to be used by the new foundational services version 4.x services whenever they are installed. Such data includes, but not limited to:
- IM admin login credentials
-
Mongo data containing IM users
-
Note: when copying resources from one namespace to another, use extra caution to remove instance specific information, such as the namespace value, uid, resourceVersion, ownerReferences, etc.
-
Copy the following secrets from the original foundational services namespace and create new secrets with the same name and data in the new namespace created in step 3.1:
- platform-auth-idp-credentials
- platform-auth-ldaps-ca-cert
-
Copy the following configmaps from the original foundational services namespace and create new configmaps with the same name and data in the new namespace created in step 3.1:
- ibm-cpp-config
- common-web-ui-config
- platform-auth-idp
Note: If new data is introduced after the data has already been copied and before the IBM Cloud Pak has been upgraded, the upgraded IBM Cloud Pak will not have this newly introduced data. If you do something to introduce data while this upgrade process is in progress, the newly introduced data may not be reflected after IBM Cloud Pak upgrade.
Performing the isolated upgrade during a maintenance window is recommended to reduce the risk of this new data loss.
If you encounter issues after running the preload_data.sh
script, see preload_data.sh
fails due to cert manager.
Step 5: Fresh install foundational services version 4.x
- Decide whether you want a simple topology or advanced topology for the foundational services version 4.x tenant.
- If you want a simple topology, install or upgrade
common-service-operator
v4 into the one and only namespace for your foundational services version 4.x tenant. -
If you want an advanced topology, running the
./cp3pt0-deployment/setup_tenant.sh
script.a. Install the
ibm-namespace-scope-operator
into the namespace created in step 3.1. This will now be known as theoperatorNamespace
for foundational services.b. Install the
common-service-operator
into the same namespace.c. Create a NamespaceScope CR
apiVersion: operator.ibm.com/v1 kind: NamespaceScope metadata: name: common-service spec: csvInjector: enable: true namespaceMembers: <comma-delimited list of namespaces from step 3, e.g. cloudpak1, cpfs1> restartLabels: intent: projected
d. Install the
common-service-operator
into theoperatorNamespace
.e. Verify that a CommonService CR named
common-service
exists in this namespace. Configure the default CommonService CRapiVersion: operator.ibm.com/v3 kind: CommonService metadata: name: common-service spec: operatorNamespace: operatorNamespace # this should already be set by default, but if not, set it servicesNamespace: <namespace where operands will be installed into> # this is optional size: <set as either small, medium, or large> # this is optional
-
Verify that
common-service-operator
isRunning
. - Verify that
common-service-maps
has a new tenant with namespaces for foundational services version 4.x and the to-be-upgraded IBM Cloud Pak.
Step 6: Upgrade IBM Cloud Pak
At this point, the cluster state is as follows:
- Isolated IBM Cloud Pak and foundational services workloads running
- New foundational services version 4.x tenant installed with just
common-service-operator
, ODLM, and NSS, and any of the to-be-upgraded IBM Cloud Pak services -
No IM or other foundational services installed yet
-
Change channel of IBM Cloud Pak operator(s).
- Verify new or updated OperandRequests exist, for example, requesting
ibm-im-operator
,ibm-platformui-operator
, and so on.- IBM Cloud Pak services are responsible for ensuring OperandRequests are updated
- If there is
ZenService
CR in IBM Cloud Pak namespace(s), remove thecsNamespace
field if it exists. - Verify that IBM Cloud Pak works.
If the upgraded IBM Cloud Pak uses SAML or OIDC for single sign-on or if you have configured this, it must be re-registered and the new IM instance will need to be re-configured since the cp-console
route and other foundational services
routes would have changed due to this instance of Bedfoundational services being a fresh installation. For more information, see Migrating identity management.