Configuring Regional-DR

Configure regional disaster recovery (Regional-DR) capability in the IBM Fusion HCI with Fusion Data Foundation to replicate and recover applications and virtual machines (VMs) across clusters.

Before you begin

If an external Red Hat® Advanced Cluster Management (RHACM) hub cluster is already available, you can use it for the Regional-DR configuration. If you don't have it, install the RHACM operator to create a hub cluster outside the IBM Fusion HCI sites.

For instructions, see the Install guide in the RHACM documentation set.

If hosted control plane (HCP) clusters deployed on IBM Fusion HCI sites are upgraded from a previous version and the RHACM operator is already installed on an HCP cluster, migrate the RHACM operator from the HCP cluster to a new cluster. For assistance with migration, contact IBM support.
It is recommended that the VMs should use the ocs-storagecluster-ceph-rbd-virtualization storage class.
Ensure that the storage class with the same name exists on the secondary-managed cluster.
Ensure that the version of the HCP clusters is 4.18 or higher.

Procedure

Deploy primary and secondary IBM Fusion HCI sites and configure them to have the network reachability between them.
Each site should have one IBM Fusion HCI. If only one IBM Fusion HCI site is present, install an IBM Fusion HCI in another site by following the instructions mentioned in Installing IBM Fusion HCI.
Important: The IBM Fusion HCI version number should be the same across both sites.
Open the following ports to ensure that the pods are reachable from other clusters:
Note: In provider mode, host networking is used for MONs and OSD.
- Ceph mon v1: 6789
- Ceph mon v2: 3300
- Ceph OSD: 6800–7300
- Ceph Manager: 9283
Optional: Set up an SSL connection between clusters deployed on IBM Fusion HCI sites.
For instructions, see Establishing an SSL connection between clusters.
Note: If a signed and valid set of certificates has been used for the deployment of base or HCP clusters in both IBM Fusion HCI sites, skip this step.
Install the Fusion Data Foundation in newly created clusters.
- For base clusters:
  Install the Fusion Data Foundation in provider mode by following the instructions mentioned in Installing Fusion Data Foundation.
- For HCP clusters:
  Install the Fusion Data Foundation client by following the instructions mentioned in Installing Fusion Data Foundation on a Hosted Control Plane cluster.
Important: The Fusion Data Foundation version number must be the same on base and HCP clusters deployed across IBM Fusion HCI sites.
Import the clusters of primary and secondary IBM Fusion HCI sites to the RHACM hub cluster.
- For base clusters:
  Import base clusters to the RHACM hub by following the instructions mentioned in Discovering multicluster engine operator hosted clusters in Red Hat Advanced Cluster Management of the RHACM documentation.
- For HCP clusters:
  Discover HCP clusters from the multicluster engine (MCE) operator by following the instructions mentioned in Discovering hosted clusters of the RHACM documentation.
  
  Discovered HCP clusters can be automatically imported into the RHACM hub if the spec.importAsManagedCluster field is set to true. You need to set this field in the individual discovered cluster's custom resource (CR). For instructions, see Configuring settings for automatic import of the RHACM documentation.
Connect the manged clusters and service networks by using the Submariner add-on of the RHACM.
Verify that the clusters have non-overlapping Classless Inter-Domain Routings (CIDRs) and cluster private networks. For overlapping CIDRs, enable Globalnet during the Submariner add-on installation to connect clusters.

Run the following command on each managed cluster to verify that the clusters have overlapping CIDRs or non-overlapping CIDRs:
```
oc get networks.config.openshift.io cluster -o json | jq .spec
```
The following example is for clusters with overlapping CIDRs, so the Globalnet needs to be enabled to connect clusters.

Example output for primary-managed cluster:
```
{ 
  "clusterNetwork": [ 
    { 
      "cidr": "10.5.0.0/16", 
      "hostPrefix": 23 
    } 
  ], 
  "externalIP": { 
    "policy": {} 
  }, 
  "networkType": "OVNKubernetes", 
  "serviceNetwork": [ 
    "10.15.0.0/16" 
  ] 
}
```
Example output for secondary-managed cluster:
```
{ 
  "clusterNetwork": [ 
    { 
      "cidr": "10.5.0.0/16", 
      "hostPrefix": 23 
    } 
  ], 
  "externalIP": { 
    "policy": {} 
  }, 
  "networkType": "OVNKubernetes", 
  "serviceNetwork": [ 
    "10.15.0.0/16" 
  ] 
}
```
Additionally, if Submariner and Fusion Data Foundation are already installed on the managed clusters, use the OpenShift Data Foundation command-line interface (CLI) tool to get additional information about the managed clusters. This information determines the need for enabling Globalnet during Submariner installation based on the cluster service and private networks. Download the OpenShift Data Foundation CLI tool from the customer portal.

Run the following command on one of the two managed clusters, where PeerManagedClusterName(ClusterID) is the name of the peer Fusion Data Foundation cluster:
```
odf get dr-prereq <PeerManagedClusterName(ClusterID)>
```
If Submariner is installed without Globalnet on managed clusters with overlapping services, the following output appears:
```
Info: Submariner is installed.
Info: Globalnet is required.
Info: Globalnet is not enabled.
```
Note: This requires Submariner to be uninstalled and then reinstalled with Globalnet enabled.

If Submariner is installed with Globalnet on managed clusters with overlapping services, the following output appears:
```
Info: Submariner is installed.
Info: Globalnet is required.
Info: Globalnet is enabled.
```
If Submariner is installed without Globalnet on managed clusters with non-overlapping services, the following output appears:
```
Info: Submariner is installed.
Info: Globalnet is not required.
Info: Globalnet is not enabled.
```
If Submariner is installed with Globalnet on managed clusters with non-overlapping services, the following output appears:
```
Info: Submariner is installed.
Info: Globalnet is not required.
Info: Globalnet is enabled.
```
For more information on Submariner add-ons and Globalnet configuration, see Submariner multicluster networking and service discovery of the RHACM documentation.
Install the OpenShift® Multicluster Orchestrator operator on the hub cluster.
For instructions, see Installing Multicluster Orchestrator operator.
To connect clusters with providerAPIServerServiceType: ClusterIP by using the Submariner add-on do one of the following methods:
If MetalLB is configured and the IP range is provided, do method 1. If MetalLB is not configured, do method 2.
- Method 1:
  If the ocs-provider-server-load-balancer service is present on both managed clusters, add an annotation to the StorageCluster on both managed clusters with the following key-value pair:
  - Key: ocs.openshift.io/api-server-exported-address
  - Value: Endpoint of the <LoadBalancer ip>:50051.
  To fetch the IP address of the ocs-provider-server-load-balancer service, run the following command:
```
oc get svc -n openshift-storage -o wide | grep provide
```
  Example output:
```
ocs-provider-server-load-balancer   LoadBalancer   172.30.213.146   172.200.88.109   50051:30506/TCP   6d   app=ocsProviderApiServer
```
  Once the annotation is added, the odf-info configMap will detect this endpoint, which the MCO will use to create the StorageClusterPeer.
  For example:
```
ocs.openshift.io/api-server-exported-address: '172.200.88.109:50051'
```
- Method 2:
  If the ocs-provider-server-load-balancer service is not present on both managed clusters, do as follows:
  1. Create a ServiceExport for the ocs-provider-server-load-balancer service.
```
apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceExport
metadata:
  name: ocs-provider-server
  namespace: openshift-storage
```
  2. Add an annotation to the StorageCluster on both managed clusters with the following key and value:
    
    Key: ocs.openshift.io/api-server-exported-address
    
    Value: <managerClusterName>.ocs-provider-server.openshift-storage.svc.clusterset.local:50051
    managedClusterName refers to the name used during cluster import into the hub cluster. For example, if rackm03 was the name provided, then:
    ocs.openshift.io/api-server-exported-address:rackm03.ocs-provider-server.openshift-storage.svc.clusterset.local:50051
    To add an annotation, run the following command:
    oc -n openshift-storage annotate storagecluster ocs-storagecluster 'ocs.openshift.io/api-server-exported-address="172.200.88.109:50051"'
    Where,
    
    openshift-storage: The namespace where Fusion Data Foundation is installed.
    
    ocs-storagecluster: The storage cluster name.
    
    key: ocs.openshift.io/api-server-exported-address
    
    value: "172.200.88.109:50051"
    Note: Either RHACM, MCE, or Submariner brings the ServiceExport CRD. So, you can create the ServiceExport after Submariner is installed for the clusters that do not have MCE installed.
    
    Once the annotation is added, the odf-info configMap will detect this endpoint, which the MCO will use to create the StorageClusterPeer.

Add the following YAML configuration on the RHACM hub cluster for both managed clusters:

For example:

apiVersion: agent.open-cluster-management.io/v1
kind: KlusterletAddonConfig
metadata:
name: <rack name>
namespace: <rack name>
spec:
clusterName: <rack name>
clusterNamespace: <rack name>
clusterLabels:
name: <rack name>
cloud: auto-detect
vendor: auto-detect
applicationManager:
enabled: true
policyController:
enabled: true
searchCollector:
enabled: true
certPolicyController:
enabled: true

Remember: In the YAML configuration, provide the same cluster name that you get in the output of the oc get managedcluster command.

Create a disaster recovery policy (DRPolicy) for the managed clusters.
For instructions, see Creating a disaster recovery policy on hub cluster.
Optional: Create a VolumeReplicationClass (VRC) for the DRPolicy that is configured with a schedule other than the default replication interval of 5 minutes (such as 10 or 15 minutes).
For instructions, see Creating a VRC.
Verify whether the Regional-DR is successfully configured.
For instructions, see Verifying Regional-DR configuration.

What to do next

Validate Regional-DR with applications by following the instructions mentioned in Validating Regional-DR with applications.