Upgrading your console and components

Step one: Update the webhook image

Log in to your cluster and run the following command to update the webhook image in the ibpinfra namespace or project:

kubectl set image deploy/ibp-webhook -n ibpinfra ibp-webhook="cp.icr.io/cp/ibp-crdwebhook:2.5.4-20230613-amd64"

Step two: Update the CRDs

This step is NOT REQUIRED when upgrading from 2.5.1, 2.5.2 or 2.5.3.

1. Extract the webhook TLS certificate from the ibpinfra namespace by running the following command:

   TLS_CERT=$(kubectl get secret/webhook-tls-cert -n ibpinfra -o jsonpath={'.data.cert\.pem'})
   

2. When you deploy IBM Blockchain Platform 2.5.4, you need to apply the following four CRDs for the CA, peer, orderer, and console. If you are upgrading to 2.5.4, before you can update the operator, you need to update the CRDs to include a new v1beta1 section as well as the webhook TLS certificate that you just stored in the TLS_CERT environment variable. In either case, run the following four commands to apply or update each CRD.

Run this command to update the CA CRD:

cat <<EOF | kubectl apply  -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: ibpcas.ibp.com
  labels:
    app.kubernetes.io/instance: ibpca
    app.kubernetes.io/managed-by: ibp-operator
    app.kubernetes.io/name: ibp
    helm.sh/chart: ibm-ibp
    release: operator
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: "${TLS_CERT}"
        service:
          name: ibp-webhook
          namespace: ibpinfra
          path: /crdconvert
      conversionReviewVersions:
      - v1beta1
      - v1alpha2
      - v1alpha1
  group: ibp.com
  names:
    kind: IBPCA
    listKind: IBPCAList
    plural: ibpcas
    singular: ibpca
  scope: Namespaced
  versions:
  - name: v1beta1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: true
    subresources:
      status: {}
  - name: v1alpha2
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
  - name: v210
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: false
    storage: false
    subresources:
      status: {}
  - name: v212
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: false
    storage: false
    subresources:
      status: {}
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
status:
  acceptedNames:
    kind: IBPCA
    listKind: IBPCAList
    plural: ibpcas
    singular: ibpca
  conditions: []
  storedVersions:
  - v1beta1
EOF

Depending on whether you are creating or updating the CRD, when successful, you should see:

customresourcedefinition.apiextensions.k8s.io/ibpcas.ibp.com created

or

customresourcedefinition.apiextensions.k8s.io/ibpcas.ibp.com configured

Run this command to update the peer CRD:

cat <<EOF | kubectl apply  -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: ibppeers.ibp.com
  labels:
    release: "operator"
    helm.sh/chart: "ibm-ibp"
    app.kubernetes.io/name: "ibp"
    app.kubernetes.io/instance: "ibppeer"
    app.kubernetes.io/managed-by: "ibp-operator"
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: "${TLS_CERT}"
        service:
          name: ibp-webhook
          namespace: ibpinfra
          path: /crdconvert
      conversionReviewVersions:
      - v1beta1
      - v1alpha2
      - v1alpha1
  group: ibp.com
  names:
    kind: IBPPeer
    listKind: IBPPeerList
    plural: ibppeers
    singular: ibppeer
  scope: Namespaced
  versions:
  - name: v1beta1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: true
    subresources:
      status: {}
  - name: v1alpha2
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
status:
  acceptedNames:
    kind: IBPPeer
    listKind: IBPPeerList
    plural: ibppeers
    singular: ibppeer
  conditions: []
  storedVersions:
  - v1beta1
EOF

When successful, you should see:

customresourcedefinition.apiextensions.k8s.io/ibppeers.ibp.com created

or

customresourcedefinition.apiextensions.k8s.io/ibppeers.ibp.com configured

Run this command to update the console CRD:

cat <<EOF | kubectl apply  -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: ibpconsoles.ibp.com
  labels:
    release: "operator"
    helm.sh/chart: "ibm-ibp"
    app.kubernetes.io/name: "ibp"
    app.kubernetes.io/instance: "ibpconsole"
    app.kubernetes.io/managed-by: "ibp-operator"
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: "${TLS_CERT}"
        service:
          name: ibp-webhook
          namespace: ibpinfra
          path: /crdconvert
      conversionReviewVersions:
      - v1beta1
      - v1alpha2
      - v1alpha1
  group: ibp.com
  names:
    kind: IBPConsole
    listKind: IBPConsoleList
    plural: ibpconsoles
    singular: ibpconsole
  scope: Namespaced
  versions:
  - name: v1beta1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: true
    subresources:
      status: {}
  - name: v1alpha2
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
status:
  acceptedNames:
    kind: IBPConsole
    listKind: IBPConsoleList
    plural: ibpconsoles
    singular: ibpconsole
  conditions: []
  storedVersions:
  - v1beta1
EOF

When successful, you should see:

customresourcedefinition.apiextensions.k8s.io/ibpconsoles.ibp.com created

or

customresourcedefinition.apiextensions.k8s.io/ibpconsoles.ibp.com configured

Run this command to update the orderer CRD:

cat <<EOF | kubectl apply  -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: ibporderers.ibp.com
  labels:
    release: "operator"
    helm.sh/chart: "ibm-ibp"
    app.kubernetes.io/name: "ibp"
    app.kubernetes.io/instance: "ibporderer"
    app.kubernetes.io/managed-by: "ibp-operator"
spec:
  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: "${TLS_CERT}"
        service:
          name: ibp-webhook
          namespace: ibpinfra
          path: /crdconvert
      conversionReviewVersions:
      - v1beta1
      - v1alpha2
      - v1alpha1
  group: ibp.com
  names:
    kind: IBPOrderer
    listKind: IBPOrdererList
    plural: ibporderers
    singular: ibporderer
  scope: Namespaced
  versions:
  - name: v1beta1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: true
    subresources:
      status: {}
  - name: v1alpha2
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        x-kubernetes-preserve-unknown-fields: true
    served: true
    storage: false
    subresources:
      status: {}
status:
  acceptedNames:
    kind: IBPOrderer
    listKind: IBPOrdererList
    plural: ibporderers
    singular: ibporderer
  conditions: []
  storedVersions:
  - v1beta1
EOF

When successful, you should see:

customresourcedefinition.apiextensions.k8s.io/ibporderers.ibp.com created

or

customresourcedefinition.apiextensions.k8s.io/ibporderers.ibp.com configured

Step three: Update the ClusterRole

This step IS REQUIRED when upgrading from 2.5.1, 2.5.2 or 2.5.3.

ATTENTION!! When upgrading to Kubernetes v1.25, you must overwrite the label after creating the components namespace, as follows:

kubectl label --overwrite ns <NAMESPACE>  pod-security.kubernetes.io/enforce=baseline

Or for any Hyperledger Fabric v1.4 peers, overwrite the label as follows:

kubectl label --overwrite ns <NAMESPACE> pod-security.kubernetes.io/enforce=privileged

You need to update the ClusterRole that is applied to your components namespace. Copy the following text to a file on your local system and save the file as ibp-clusterrole.yaml. Edit the file and replace <NAMESPACE> with the name of your IBM Blockchain Platform deployment components namespace.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: <NAMESPACE>
  labels:
    release: "operator"
    helm.sh/chart: "ibm-ibp"
    app.kubernetes.io/name: "ibp"
    app.kubernetes.io/instance: "ibp"
    app.kubernetes.io/managed-by: "ibp-operator"
rules:
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - persistentvolumeclaims
      - persistentvolumes
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
    verbs:
      - get
  - apiGroups:
      - route.openshift.io
    resources:
      - routes
      - routes/custom-host
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
  - apiGroups:
      - ""
    resources:
      - pods
      - pods/log
      - persistentvolumeclaims
      - persistentvolumes
      - services
      - endpoints
      - events
      - configmaps
      - secrets
      - nodes
      - serviceaccounts
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
  - apiGroups:
      - "batch"
    resources:
      - jobs
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
  - apiGroups:
      - "authorization.openshift.io"
      - "rbac.authorization.k8s.io"
    resources:
      - roles
      - rolebindings
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
      - bind
      - escalate
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - apps
    resources:
      - deployments
      - daemonsets
      - replicasets
      - statefulsets
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
  - apiGroups:
      - monitoring.coreos.com
    resources:
      - servicemonitors
    verbs:
      - get
      - create
  - apiGroups:
      - apps
    resourceNames:
      - ibp-operator
    resources:
      - deployments/finalizers
    verbs:
      - update
  - apiGroups:
      - ibp.com
    resources:
      - ibpcas.ibp.com
      - ibppeers.ibp.com
      - ibporderers.ibp.com
      - ibpconsoles.ibp.com
      - ibpcas
      - ibppeers
      - ibporderers
      - ibpconsoles
      - ibpcas/finalizers
      - ibppeers/finalizers
      - ibporderers/finalizers
      - ibpconsoles/finalizers
      - ibpcas/status
      - ibppeers/status
      - ibporderers/status
      - ibpconsoles/status
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection
  - apiGroups:
      - extensions
      - networking.k8s.io
      - config.openshift.io
    resources:
      - ingresses
      - networkpolicies
    verbs:
      - get
      - list
      - create
      - update
      - patch
      - watch
      - delete
      - deletecollection

After you edit and save the file, run the following command:

kubectl apply -f ibp-clusterrole.yaml

Step four: Upgrade the IBM Blockchain operator

Run the following command to upgrade the operator, specifying the Kubernetes <NAMESPACE> for the platform:

kubectl set image deploy/ibp-operator -n <NAMESPACE> ibp-operator="cp.icr.io/cp/ibp-operator:2.5.4-20230613-amd64"

After applying the new image to the operator deployment, the operator will restart and pull the latest image. This upgrade takes about one minute. You can then run the following command to confirm the update to the operator specification:

kubectl get deployment ibp-operator -o yaml

While the upgrade is taking place, the console is upgraded also, so you will not be able to deploy or manage your blockchain components.

You can check that the upgrade is complete by running the following command:

kubectl get deployment

If the upgrade is successful, you will see the following table, with four numerical 1 characters displayed for your operator and console:

NAME           READY     UP-TO-DATE   AVAILABLE   AGE
ibp-operator   1/1       1            1           1m
ibpconsole     1/1       1            1           4m

After the operator restarts, update the console configuration to deploy the latest Fabric component patches by running the following commands:

CONSOLE_SPEC=$(kubectl get ibpconsoles.ibp.com --no-headers -n $NAMESPACE | awk '{print $1}')
kubectl patch ibpconsoles.ibp.com ${CONSOLE_SPEC} --type=json -p="[{'op': 'remove', 'path': '/spec/versions'}]" -n ${NAMESPACE}
kubectl patch ibpconsoles.ibp.com ${CONSOLE_SPEC} --type=json -p="[{'op': 'replace', 'path':'/spec/usetags', 'value':true}]" -n $NAMESPACE  
kubectl delete cm ${CONSOLE_SPEC}-deployer -n $NAMESPACE
kubectl rollout restart deploy ${CONSOLE_SPEC} -n $NAMESPACE

Step five: Upgrade your nodes

You can now follow the steps to upgrade your blockchain nodes. Be aware that after the nodes are upgraded, there is no way to roll back the upgrade from 2.5.4 to 2.5.x. After you upgrade to 2.5.4, you can take advantage of the new Fabric v2.x lifecycle deployment process for your smart contracts. But to avoid your peers crashing, you need to ensure that you upgrade your peers before you upgrade your channels. Learn more about considerations when Upgrading to a new version of Fabric.

After upgrading your nodes and updating your channels, make sure you update your organizations to add an organization-level endorsement policy.

Step six: Update MSPs in consortium to add organization-level endorsement policy

To use the 2.x smart contract lifecycle, an organization must have an endorsement policy defined. If any organization in the consortium (the list of organizations maintained by the ordering service that are allowed to create channels) do not have an endorsement policy defined, a warning message will appear on the Details page of the ordering service with a list of organization MSPs that must be updated.

The best practice to add this endorsement policy to the MSP is to delete the MSP from the system channel and then re-add the MSP. The console detects the fact that the MSP does not contain the endorsement policy and automatically adds it. Note that this action can only be completed by an ordering service administrator. You do not need to delete and re-add the MSPs in the configuration of any application channels that have already been created. For these MSPs, the endorsement policy is added as part of the process of deploying the smart contract.