Application DR

Steps to setup an Application DR, enroll an application, failover applications, and disable applications.

Step 1-setup Application DR

You can setup an Application DR anytime after you set up the Metro-DR and before you enable DR for an application.
  1. Set up Metro-DR. For the procedure to setup, see ../metrodr/sf_disaster_recovery.html#sf_disaster_recovery__set. After Site 1, Site 2, and Tiebreaker are installed, proceed with the next steps.
  2. Do the following steps on both Site 1 and Site 2 of Metro-DR:
    1. If OADP (version that is used by Backup & Restore) operator does not exist, then create a namespace and install it.
      Note: The installed OADP must align with the version that is used by Fusion Backup & Restore so that the following issues do not exist:
      • OADP inconsistencies
      • CRD conflicts between Fusion Backup & Restore and Metro-DR
      1. Go to OperatorHub > Operator Installation.
      2. Find OADP, the Red Hat® distribution of Velero.
      3. In the Install Operator window, select the default namespace openshift-adp. You can also install it in another namespace.
      4. Click Install.
    2. Create DataProtectionApplication instance to start the velero operator.
      
apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  labels:
    app.kubernetes.io/component: velero
  name: velero
  namespace: openshift-adp
spec:
  backupImages: false
  configuration:
    restic:
      enable: false
    velero:
      defaultPlugins:
        - openshift
        - aws
      noDefaultBackupLocation: true
      podConfig:
        resourceAllocations:
          limits:
            cpu: '1'
            ephemeral_storage: 25Mi
            memory: 1Gi
          requests:
            cpu: 100m
            ephemeral_storage: 25Mi
            memory: 256Mi
  podDnsConfig: {}
  3. On the site1, do the following steps:
    1. Obtain the minio secret content.
      site1_keyid=$(oc extract -n ibm-spectrum-fusion-ns secret/isf-metrodr-minio-site1 --keys=AWS_ACCESS_KEY_ID --to=-);site1_key=$(oc extract -n ibm-spectrum-fusion-ns secret/isf-metrodr-minio-site1 --keys=AWS_SECRET_ACCESS_KEY --to=-);echo "[default]" > cloud-credentials-site1.yaml;echo "aws_access_key_id = ${site1_keyid}" >> cloud-credentials-site1.yaml;echo "aws_secret_access_key = ${site1_key}" >> cloud-credentials-site1.yaml
      It creates a cloud-credentials-site1.yaml.
    2. Create secret cloud-crentials-site1 from the file generated in the previous step. 
      oc create secret generic cloud-credentials-site1 --namespace openshift-adp --from-file cloud=./cloud-credentials-site1.yaml
    3. Obtain the minio secret for site2.
      site2_keyid=$(oc extract -n ibm-spectrum-fusion-ns secret/isf-metrodr-minio-site2 --keys=AWS_ACCESS_KEY_ID --to=-);site2_key=$(oc extract -n ibm-spectrum-fusion-ns secret/isf-metrodr-minio-site2 --keys=AWS_SECRET_ACCESS_KEY --to=-);echo "[default]" > cloud-credentials-site2.yaml;echo "aws_access_key_id = ${site2_keyid}" >> cloud-credentials-site2.yaml;echo "aws_secret_access_key = ${site2_key}" >> cloud-credentials-site2.yaml
    4. Create cloud-credentials-site2 from the file generated in the previous step. 
      oc create secret generic cloud-credentials-site2 --namespace openshift-adp --from-file cloud=./cloud-credentials-site2.yaml
    5. Check whether the cloud credential secrets are created for both the sites. Example output: 
      [root@roadiem9 cpd-cli-linux-EE-13.1.2-89]# oc get secret -n openshift-adp |grep -E "NAME|cloud"
NAME                                               TYPE                                  DATA   AGE
cloud-credentials-site1                            Opaque                                1      57s
cloud-credentials-site2                            Opaque                                1      31s
  4. Repeat the previous step and substeps on site2.
  5. On site1, create secret cloud-crentials-site1 from the file that is generated in step 3.
  6. Create secrets in the Velero operator namespace to access S3 storage on both the sites of Metro-DR. The S3 credentials in this secret are in the following Amazon Web Services format:
    
[default]
aws_access_key_id=<AWS_ACCESS_KEY_ID>
aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>
  7. Access details in the ibm-spectrum-fusion-ns (or your Fusion namespace) Velero secret:
    • For Velero secret to access Site 1 S3 storage, see isf-metrodr-minio-site1 secret.
    • For Velero secret to access Site 2 S3 storage, see isf-metrodr-minio-site2 secret.
    Example cloud-credentials-site1 secret content to access the S3 storage of Metro-DR sites:
    • Metro-DR Site 1.
      
Key: cloud
Value:
[default]
aws_access_key_id=minio
aws_secret_access_key=04ie0oO7x46g0i5
    • Metro-DR Site 2:
      
Key: cloud
Value:
[default]
aws_access_key_id=minio
aws_secret_access_key=84ie0oO9x46g0i8
  8. Enable kubeObjectProtection in ramen-dr-cluster-operator-config map of ibm-spectrum-fusion-ns namespace:
    1. To enable kubeObjectProtection, set disabled to false and specify the Velero installation namespace.
      
kubeObjectProtection:
      disabled: false
      veleroNamespaceName: openshift-adp
    2. In S3StoreProfiles section, for every S3StoreProfile, specify the Velero secret details for the corresponding site, including key name and secret name:

      Example for Site 1: 

      
veleroNamespaceSecretKeyRef:
        key: cloud
        name: cloud-credentials-site1
      Example for Site 2: 
      
veleroNamespaceSecretKeyRef:
        key: cloud
        name: cloud-credentials-site2
    3. From site 1, get the ingress certificate details by using the following command:
      oc get cm default-ingress-cert -n openshift-config-managed -o jsonpath="{.data.ca-bundle\.crt}"
    4. Repeat the previous step on site 2.
    5. On site1, edit the ConfigMap ramen-dr-cluster-operator-config in namespace ibm-spectrum-fusion-ns to add caCertificates under s3StoreProfiles. Make sure that you add the appropriate caCertificatesfor the S3 profile.
    6. Repeat the previous step on site 2.
    7. Configure ramen-dr-cluster-operator-config ConfigMap by using the following example:
      kind: ConfigMap
apiVersion: v1
metadata:
  name: ramen-dr-cluster-operator-config
  namespace: ibm-spectrum-fusion-ns
data:
  ramen_manager_config.yaml: |
    apiVersion: ramendr.openshift.io/v1alpha1
    drClusterOperator: {}
    health:
      healthProbeBindAddress: :8081
    kind: RamenConfig
    kubeObjectProtection:
      disabled: false
      veleroNamespaceName: openshift-adp
    leaderElection:
      leaderElect: true
      leaseDuration: 0s
      renewDeadline: 0s
      resourceLock: ""
      resourceName: dr-cluster.ramendr.openshift.io
      resourceNamespace: ""
      retryPeriod: 0s
    metrics:
      bindAddress: 127.0.0.1:9289
    ramenControllerType: dr-cluster
    s3StoreProfiles:
    - s3Bucket: isf-minio-site1
      s3CompatibleEndpoint: https://isf-minio-ibm-spectrum-fusion-ns.apps.rackag2.mydomain.com
      s3ProfileName: site1
      s3Region: site1
      s3SecretRef:
        name: isf-minio-site1
        namespace: ibm-spectrum-fusion-ns
      veleroNamespaceSecretKeyRef:
        key: cloud
        name: cloud-credentials-site1
      caCertificates: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURiRENDQWxTZ0F3SUJBZ0lIWjc3OHV4V3QrREFOQmdrcWhraUc5dzBCQVFzRkFEQW1NU1F3SWdZRFZRUUQKREJ0cGJtZHlaWE56TFc5d1pYSmhkRzl5UURFMk9ESXpNVGs1TlRBd0hoY05Nak13TkRJME1EY3dOVFE1V2hjTgpNalV3TkRJek1EY3dOVFV3V2pBbU1TUXdJZ1lEVlFRRERCc3FMbUZ3Y0hNdWNtRmphMkZuTWk1dGVXUnZiV0ZwCmJpNWpiMjB3Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRQ3BiaWI5dURTTTVjTk4KdzdKVGRSWGppd2JNdUlWRGhXbmhIMU9oejA4SEY4ck9McEplVEpiYXZJKzBPdjdnaHcwb2p3MWVoaW9adktraApVdWg2R1dyWUtiTk5DSm0xVkhrSWt6ZDhXYlpDcklkdTFXcCtnN21zaHpkcTBYZEtNOXVkblB3aG9VNURtV2RzCmNuRjU0bnl0bkhoOXJVM0ZNU1hHUGhyU3J0cnVEQ01EUkQzQVJ6UmlOL09VNVlyZHFtYk1teHM3RUJtZmRJY1QKWGlFQkoxb09TTC85enkzNnZiSFFkckptc2hPUEN6VG1LWkJ6MklVL3NvR2NraVBrT2tXL1gvRERUemZNTEJNWApab3NxWWRpNWxjOTR0ek9wUjNMOGU1bVhrR2k4cnNHamJpcURmTHpMZFFtN1pmT1BqYWdsQjE1M0JvdnRMM1hjCmhKQzJiWGJsQWdNQkFBR2pnWjR3Z1pzd0RnWURWUjBQQVFIL0JBUURBZ1dnTUJNR0ExVWRKUVFNTUFvR0NDc0cKQVFVRkJ3TUJNQXdHQTFVZEV3RUIvd1FDTUFBd0hRWURWUjBPQkJZRUZJVy9UdjdtY1BBS1JVWlVJNlRDMmVMdwpaZWhTTUI4R0ExVWRJd1FZTUJhQUZDNmpGaXJiZzdVeXdUeEc2UTFDWHMyMWVwVjJNQ1lHQTFVZEVRUWZNQjJDCkd5b3VZWEJ3Y3k1eVlXTnJZV2N5TG0xNVpHOXRZV2x1TG1OdmJUQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUEKdENDUGZkUWs2Ui9XeGZ3VC9SWGdlOFczWlFwNUJjL3A1eEJ0MG1QODcwbVBFc0ZmRGhBVVpiYStZeFJNWWp0SApOblBhemlKQkdmUVpFamRoblVGb0lVVjNYd0VENUQrSzZtVCtxVWgyNUNNazBuc1YveEZ0eUZuTlYyWElFcSthCmtjclowZTcvVVRpcVJsR3lYelgwSUpFYnlsYWxrL1lHb1pqMG10SDZ6aEVNVjNaczV1UVlJYmNJSmh0N1Z0UHkKR2lJQmdxaHBhS2NPU2VJWUpaUzlyVDE0NEdIL2lvc2xQTTNpUFJDQkxmUVNWSnJqMGRndVpKN0czQS9Vc3BEbAozOFp5QnNFZXZ1bXgxTDY5M0xoOElsN2NzcnBTVVBFZG8zc3ZZcXdtWVBlQzdVZTN3RGEyaCtjQkEyMW5CYW5zCllLT1ErYTV5eXFUR0MyKzBoVnpiaHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tQkVHSU4gQ0VSVElGSUNBVEUtLS0tLQpNSUlERERDQ0FmU2dBd0lCQWdJQkFUQU5CZ2txaGtpRzl3MEJBUXNGQURBbU1TUXdJZ1lEVlFRRERCdHBibWR5ClpYTnpMVzl3WlhKaGRHOXlRREUyT0RJek1UazVOVEF3SGhjTk1qTXdOREkwTURjd05UUTVXaGNOTWpVd05ESXoKTURjd05UVXdXakFtTVNRd0lnWURWUVFEREJ0cGJtZHlaWE56TFc5d1pYSmhkRzl5UURFMk9ESXpNVGs1TlRBdwpnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFEc1VhME9IOG96TFBtS3VxTVhSV1h4CnNVWFgyN0tuY2pqdnMzNFc5b2V0aGp2ZWh5U0NLMHJvYkpDQmt2enpHelpRaXRTT2NmUUNnV1BUQ2NIVWl5YVMKNWNKUUt1bGFSbjBYUTFrVld4MVQxbFo2NXZHcmliVnhYblhTUTVhVzR3T0w1bVJNT1hyVnkyZ0ZLK2NHMzZ2RgprQ2VBMmZXcXZONi8ycnZ3aEFpU2VYTHpOc3dEWHk1Q3puOGZiajl3K2Q0YWE2VFhUSG96VWZlQ21aaCtpK3MyCjlpQ0MzSzhjSGtSdTN4K0xwZldWK2MzNGd3c0JCbGtJYU1xWkpNY094WGhNemJ3dHJtRGJOeHpLc3ZUeXJ2bVoKcnhiWHVEblpyMExWZ3hvSS9qNjhvUEdreWpRdVQxd0VEcEdtZ1JHczhVdFRNaVdrcG1QY2VPTmVrTzVDbmltbgpBZ01CQUFHalJUQkRNQTRHQTFVZER3RUIvd1FFQXdJQ3BEQVNCZ05WSFJNQkFmOEVDREFHQVFIL0FnRUFNQjBHCkExVWREZ1FXQkJRdW94WXEyNE8xTXNFOFJ1a05RbDdOdFhxVmRqQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUEKMjRJN3JvYmJTZmtXdXZGVjZQNGNEeXdoQ0xWKzZjWHQzdnVLWnFCWmxjSlB4ODFpNnl3YktyTjZidXNQRWkxQwpDWkIzdVpJcTBrZUhVRGFoQndwM1BYVWlhcG9LMHRRNC9CcWpBd21DejZXVHc2VmVRRDlaZWErcFpINWlaNHZDClBIV1VkSHdId0ZUOUZWRkVxNWxRa3RUYVdqclRLdTE1YnpHT3RpNTE3RFkyaHNrMDJFcVJXdk1DUzZTeWhNb3IKbDN5MlNiMTFnVWMyZ1czaU1TSFo4ZzkzcHVIZGZRUXVEMElqbXFSMWp4ZHc1K05tcFlka3N1M1ArbGE4aW1lawpxa1U3eUp4UzhnRzZ1NmFrdXk0YU9YakhwVUFtSTVOTG0xdW9wNXdiUTlGV2RVZkNxQmd0cmxRblhhaVNXQ3lBCmJXTkxjR0V1a3RnZzUrcGVVVjJzMHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
    - s3Bucket: isf-minio-site2
      s3CompatibleEndpoint: https://isf-minio-ibm-spectrum-fusion-ns.apps.rackag3.mydomain.com
      s3ProfileName: site2
      s3Region: site2
      s3SecretRef:
        name: isf-minio-site1
        namespace: ibm-spectrum-fusion-ns
      veleroNamespaceSecretKeyRef:
        key: cloud
        name: cloud-credentials-site2
      caCertificates: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURiVENDQWxXZ0F3SUJBZ0lJVTc3MFdhck5pdVl3RFFZSktvWklodmNOQVFFTEJRQXdKakVrTUNJR0ExVUUKQXd3YmFXNW5jbVZ6Y3kxdmNHVnlZWFJ2Y2tBeE5qZ3lORGc0TXpFeE1CNFhEVEl6TURReU5qQTFOVEUxTVZvWApEVEkxTURReU5UQTFOVEUxTWxvd0pqRWtNQ0lHQTFVRUF3d2JLaTVoY0hCekxuSmhZMnRoWnpNdWJYbGtiMjFoCmFXNHVZMjl0TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFxN1JYaXA2WW04UlkKYVJEYVhZTGdZaWpTVjNaOEx6S3UvWDVnakR4NUtqd2tBUW5IeWZJazBpOWVrSnRMR2RabWgzbEFCZi9jRS9YNAoyUzlZLzFiRUhPN3J2cHpLVHhZajVtQ1BSRXRLT3BBY1B1bEY3SjJqcnBYODRMYzJJdmNtZHB1blR0TVhUVHBtCkxFS0JVdWMyVUZkOHdrcUpKV3pRRVlhVFpueDZxVVVYOEM2SkRYVzlNM1B6THJIOHBza2xxS0ZvWk9yaEtzNnYKYjhOeUJZMjd1eTNaZWFhS21KU2l2K1R4M2RURGR1amxwNTdoUnd3Y004WnNRdy93MjB2cTQ2S2JPcWZ2YmpmZApIZ0pEbHV3WjhBQzZGWld2OCtwRVk4YTlaVDVtUE1MTnZ5OUZnS2NRMzczdjZEd2VTSGZudWxsZjJVYldBTkJLCjJLOTNTTUwyc3dJREFRQUJvNEdlTUlHYk1BNEdBMVVkRHdFQi93UUVBd0lGb0RBVEJnTlZIU1VFRERBS0JnZ3IKQmdFRkJRY0RBVEFNQmdOVkhSTUJBZjhFQWpBQU1CMEdBMVVkRGdRV0JCUTVCelVrVkdic3BVUEU2cjhrSkZrZApxV01lTGpBZkJnTlZIU01FR0RBV2dCUW5nOHErVlFMSnRzUWtEbTk1S0x1RTJYYTRrVEFtQmdOVkhSRUVIekFkCmdoc3FMbUZ3Y0hNdWNtRmphMkZuTXk1dGVXUnZiV0ZwYmk1amIyMHdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUIKQUN3NmFVTkRNY0VtaVBMUWZWcTEzY0laZHZVaWE3Y252Q3hLNWR1WDdReVZKOU5jZ21QMUo0WDJ6Vmh0enJMawptNnAybjhCcHF0WXZGSDBSRkxJUHpCVTd6eVN0N0szUDJ5clhHM3laV1N6aXVIYURPdUQ5YXBmTVY3QnlrOVdYCk8xQnVxeU0ydzA3NDBPajU3bndybTNKd3pBK0dsVStVc0RTaSs0RlRuT0lBTHF6R05FWG5IMy80eHN5WVgwejUKNDNpNXdpbUJPYVE0ZWE4K1hlalNOeGJBVzRTNUF3VGlTQnVHaWNSN1RrR0NaNGo3WDZ2eEF0dWZ3Q2pnNElxZAo3RGNWcW11WEdxREhZTFBtVVhKMWVBaXYrRmRCTkQ3dE8xVWhQdXdMdEZxWlAyTERiNHpZSkF5Tnd4NHZPYjJGCnBNZ3dzVmNFZXdkbklmaGduM1hLZGZNPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tQkVHSU4gQ0VSVElGSUNBVEUtLS0tLQpNSUlERERDQ0FmU2dBd0lCQWdJQkFUQU5CZ2txaGtpRzl3MEJBUXNGQURBbU1TUXdJZ1lEVlFRRERCdHBibWR5ClpYTnpMVzl3WlhKaGRHOXlRREUyT0RJME9EZ3pNVEV3SGhjTk1qTXdOREkyTURVMU1UVXdXaGNOTWpVd05ESTEKTURVMU1UVXhXakFtTVNRd0lnWURWUVFEREJ0cGJtZHlaWE56TFc5d1pYSmhkRzl5UURFMk9ESTBPRGd6TVRFdwpnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFEWFp3SFVoTTJ0Z2pZMG5LOEJGZkFtCllMZkFTOHAwaEdKZ3B3SmtCZFllTUUyTlg2Q1RSeFU0RmZWMVRaZm8vQld3K3dpcHpzY1hzWE9OMUEzWDczVUIKcnhUOXM4OFJTZDJtWlI3VllPVThqbVNPZXR2N1NOeUlqYWtic3hJcmJJanRJOHVpZlpQNGlJT2VlMmFUaWJSYQorWEY4MURuLzdCNldwdmxrYUpxa0ZXekQ3Q2pQeFdUd1NqRjJRZllyOVVYYldYZWI2SnMwaHNwbGRGNDhzQ2pqClQrRXE4VnNxRmdjNFJaQWxjRFVMZEE3VUs3OE1YTnI2NkU4Z2NqdlhtaS9RdjJvZ3dHZGczeHZHTlNDUGw5YW0KQzFkSGJ4Uk9vSkZjNkx6THpiZ3BqeVJHbnRJblNTbCtSRmsvTk13cUUrci8rQkhpWXFpOTlReWxUR2lZY1JQbgpBZ01CQUFHalJUQkRNQTRHQTFVZER3RUIvd1FFQXdJQ3BEQVNCZ05WSFJNQkFmOEVDREFHQVFIL0FnRUFNQjBHCkExVWREZ1FXQkJRbmc4cStWUUxKdHNRa0RtOTVLTHVFMlhhNGtUQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUEKelRoOHhsS3FneWlPU2V4NS9ZblFob0ZycHN5c0x0UGJmMkE2QUVCSy9acGVORlc1T3VmZm5QU25EUEtqdGtORwp5UTlUOFJuS2FrVTRKK0NldGNlUTdKM0l4R3ZzdjNqVGZ3ZXlVMjEra1R2TTJhaGZXVmt3NTZEZlRUZmU4dm5sCnQrcVpqRnBmV2p4NnU1Nnp5S0t1NXcwTm12WllXSUdDOFFUWlJNOE9VRmhudjJFeCtnT2NRb2dPcEtNdXdnVWkKemEzWVRxUC93aVJ2Q2l6ZURXUmsxUmdnR0xVWkcvRGUxZUYxcVp0U0xaZHlFelQ3N3hxMEdCbVpXQjh6NktISAp1eW5VS0ZBa0tZSDJObTAydmlmdVN1MEU2TVN4a2lsVm5qM1ZJQUgrVnJTNU15OHh6MkJzcGdYVnVSSmtnL1lICnFmR1Ywb3VYenZCczV2SHJUZFdNdlE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
    volSync:
      disabled: true
    webhook:
      port: 9443
    8. Go to Workloads > Pods in the ibm-spectrum-fusion-ns namespace (or Fusion namespace) and then delete the pod with the prefix ramen-dr-cluster-operator- to reflect the previous Ramen configuration changes.

Step 2-enroll an application

  1. Create VolumeReplicationGroup (VRG) in the namespace of the Application with content similar to the following example:
    Note: You can also do the enrollment from the IBM Fusion user interface.
    apiVersion: ramendr.openshift.io/v1alpha1
kind: VolumeReplicationGroup
metadata:
  name: shioramen
  namespace: shioramen
spec:
  kubeObjectProtection: {}
  pvcSelector: {}
  replicationState: primary
  s3Profiles:
    - site2
    - site1
  sync: {}
  volSync:
    disabled: true
    Important: Make sure that all the required parameters are defined in spec section during the VRG creation. If you edit VRG parameters later on, it can lead to an inconsistent behavior. If changes to the VRG are required after creation, then you must recreate the VRG.
  2. If the Application is already enrolled by using the UI, then use the following command to patch the existing VRG:
    kubectl patch -n <namespace> vrg/<namespace> --type json -p'[{"op":"add", "path":"/spec/kubeObjectProtection", "value": {}}]'
  3. The VRG status must show ClusterDataProtected as true. If you see AnnotationFailed reason, then go to the PVC and remove the following annotation from it: 
    volumereplicationgroups.ramendr.openshift.io/vr-archived: archiveV1-0

Step 3-failover applications

  1. Failover applications. For steps to failover applications, see ../metrodr/sf_disaster_recovery.html#sf_disaster_recovery__failover.
  2. After the persistent volumes are recovered, recover the application. To enable Kubernetes resource protection and recover, update the VolumeReplicationGroup in the namespace of the application.

    Example: 

    kubectl patch -nmy-ns vrg/my-ns --type json -p'[{"op":"add", "path":"/spec/kubeObjectProtection", "value": {}}]'
  3. Failover must be successful for the Application, and the VRG must reflect the same in the status ClusterDataProtected as true. In case the condition is false and you can see the AnnotationFailed reason, then go to the PVC and remove the following annotation from it:
    volumereplicationgroups.ramendr.openshift.io/vr-archived: archiveV1-0

Step 4-disable Application DR

For each application that is DR enabled, disable Metro-DR for that application to clean it up, including deletion of its VolumeReplicationGroup.
  1. Disable DR for applications from the IBM Fusion HCI System. For the procedure to disable, see Disable disaster recovery point in Backing up your applications topic.
  2. Update Ramen ConfigMap and restart the Ramen controller:
    
kubectl -nibm-spectrum-fusion-ns patch cm/ramen-dr-cluster-operator-config --type json -p[{\"op\":add\,\"path\":/data/ramen_manager_config.yaml\,\"value\":\""$(kubectl -nibm-spectrum-fusion-ns get cm/ramen-dr-cluster-operator-config -ojsonpath='{.data.ramen_manager_config\.yaml}'|sed -n '/^kubeObjectProtection:/{:1;n;/^ /b1};p;$!s/$/\\n/'|tr -d '\n')"\"}]
sleep 120
kubectl -nibm-spectrum-fusion-ns rollout restart deploy/ramen-dr-cluster-operator;kubectl -nibm-spectrum-fusion-ns rollout status -w deploy/ramen-dr-cluster-operator
    Note: Update Ramen ConfigMap, wait for two minutes, and then restart the Ramen controller.