Preparing for fail back

If two storage clusters were originally configured only for one-way mirroring, in order to fail back, configure the primary storage cluster for mirroring in order to replicate the images in the opposite direction.

During failback scenario, the existing peer that is inaccessible must be removed before adding a new peer to an existing cluster.

Prerequisites

  • A running IBM Storage Ceph cluster.

  • Root-level access to the client node.

Procedure

  1. Log into the cephadm shell:

    Example 

    [root@rbd-client ~]# cephadm shell

  2. On the site-a storage cluster , run the following command:

    Example 

    [ceph: root@rbd-client /]# ceph orch apply rbd-mirror --placement=host01
  3. Remove any inaccessible peers.
    Important: This step must be run on the peer site which is up and running.
    Note: Multiple peers are supported only for one way mirroring.
    1. Get the pool UUID:
      Syntax 
      rbd mirror pool info POOL_NAME
      Example
      [ceph: root@host01 /]# rbd mirror pool info pool_failback
    2. Remove the inaccessible peer:
      Syntax 
      rbd mirror pool peer remove POOL_NAME PEER_UUID
      Example
      [ceph: root@host01 /]# rbd mirror pool peer remove pool_failback f055bb88-6253-4041-923d-08c4ecbe799a
  4. Create a block device pool with a name same as its peer mirror pool.
    1. To create an rbd pool, run the following:

      Syntax

      ceph osd pool create POOL_NAME PG_NUM
      ceph osd pool application enable POOL_NAME rbd
      rbd pool init -p POOL_NAME

      Example

      [root@rbd-client ~]# ceph osd pool create pool1
      [root@rbd-client ~]# ceph osd pool application enable pool1 rbd
      [root@rbd-client ~]# rbd pool init -p pool1

  5. On a Ceph client node, bootstrap the storage cluster peers.

    1. Create Ceph user accounts, and register the storage cluster peer to the pool:

      Syntax 

      rbd mirror pool peer bootstrap create --site-name LOCAL_SITE_NAME POOL_NAME > PATH_TO_BOOTSTRAP_TOKEN

      Example 

      [ceph: root@rbd-client-site-a /]# rbd mirror pool peer bootstrap create --site-name site-a data > /root/bootstrap_token_site-a
      Note: This example bootstrap command creates the client.rbd-mirror.site-a and the client.rbd-mirror-peer Ceph users.

    2. Copy the bootstrap token file to the site-b storage cluster.

    3. Import the bootstrap token on the site-b storage cluster:

      Syntax 

      rbd mirror pool peer bootstrap import --site-name LOCAL_SITE_NAME --direction rx-only POOL_NAME PATH_TO_BOOTSTRAP_TOKEN

      Example 

      [ceph: root@rbd-client-site-b /]# rbd mirror pool peer bootstrap import --site-name site-b --direction rx-only data /root/bootstrap_token_site-a
      Note: For one-way RBD mirroring, you must use the --direction rx-only argument, as two-way mirroring is the default when bootstrapping peers.

  6. From a monitor node in the site-a storage cluster, verify the site-b storage cluster was successfully added as a peer:

    Example 

    [ceph: root@rbd-client /]# rbd mirror pool info -p data
Mode: image
Peers:
  UUID                                 NAME   CLIENT
  d2ae0594-a43b-4c67-a167-a36c646e8643 site-b client.site-b