Failing back

You can fail back to a Passive instance and let the current Primary instance become the new Passive instance after a failover to the current Primary instance has occurred.

Failing back to the original primary instance

Consider the following scenario:
  • Primary configuration A is deployed as Primary instance A.
  • Passive configuration B is deployed and attached to Primary instance A, forming a Primary A/ Passive B topology.
  • Later, problems arise in Primary instance A, and a failover from A to B takes place, in which Primary instance A becomes a new Passive instance A, and Passive instance B becomes the new Primary instance B.
  • While B has acted as the Primary instance, additional required shared volumes have been added by using the Add Disk operation.

Considering the example scenario, suppose that the problems on Passive instance A are later fixed, and as a result you want to fail back from B to A, returning A to its Primary role and B to its original Passive role.

Given these conditions, the administrator completes the following general steps:
  1. At the location where the Primary instance B is deployed, run the Prepare Primary for Takeover operation. This operation shuts down the servers on the Primary instance and detaches shared volumes.
  2. At the location where the Primary instance B is deployed, navigate to the Block Storage Replication page of the console, select the replication instance, and for each shared volume pair, run the Failover operation. As part of this operation the administrator can choose to switch roles and enable replication in the reverse direction. If you choose to change the direction of the replication, you must wait until the state is synchronized for each shared volume pair.
  3. At the location where the Passive instance A is deployed, from the console bring up the Manage > Operations page of the Instance Console for Passive instance A, and run the Passive Takeover operation. This operation restarts the servers on the new Primary instance A, and clients can now be served from this cluster.
  4. At the location where the Primary instance A is deployed, run the Add New Member operation, select the Passive member type, and enter the IP address of the IBM Storage Scale Manager virtual machine for Passive instance B.

Failing back to a new passive instance

Considering the example scenario again, suppose that the problems on Passive instance A cannot be fixed, and as a result you want to create a new Passive instance C and establish a new Primary / Passive configuration by using Primary instance B and Passive instance C.

Given these conditions, the administrator completes the following general steps:
  1. Re-create as many block shared storage volumes as are used in Primary instance B.
  2. Configure block storage replication between the storage volumes for Primary instance B and the corresponding new storage volumes that will be added to Passive configuration C when it is deployed. As part of this process, create a replication profile between the locations where Primary instance B and Passive configuration C are deployed.
  3. Deploy Passive configuration C to create Passive instance C. During this deployment, you must specify the same file system names as those that exist on Primary instance B, and add the block shared storage volumes that correspond to those used in Primary instance B. You can use the List Attached Volumes operation on Primary instance B to determine which volumes are associated with a given file system.
    Example:
    2014-06-18 20:21:04.363788
GPFS VM=172.20.176.68:
Volume Name=insurancev1, Size=1024, Mount=/gpfs/insurance, uid=7b9fcc7f-8c93-4397-ac89-3430f174978a lunId=6005076802858B5938000000000001C8

    In this example, the volume insurancev1 is associated with the file system named insurance (as seen in the Mount=/gpfs/insurance output).

    When creating the corresponding file system on Passive instance C you can use the block storage replication profile information to match the volumes on Passive instance C with the corresponding volumes on Primary instance B. Be sure to accept the replication for any target volumes in the block replication relationship (see the System > Block Storage Replication page of the console). Keep in mind the following considerations:
    • All storage volumes used for your application must be in the same block storage replication relationship.
    • All storage volumes must be configured for the same synchronous or asynchronous replication setting.
    • Any volumes that you attach must be newly created and cannot have been previously used or partitioned by other entities.
  4. Add additional file systems as needed by using the Attach Storage Volumes operation. Ensure that you specify the same file system name and the correct shared volumes.
  5. At the location where the Primary instance B is deployed, run the Add New Member operation, select the Passive member type, and enter the IP address of the Manager virtual machine for Passive instance C.
  6. At the location where the Primary instance B is deployed, run the Prepare Primary for Takeover operation. This operation shuts down the servers on the Primary instance and detaches shared volumes.
  7. At the location where the Primary instance B is deployed, navigate to the Block Storage Replication page of the console, select the replication instance, and for each shared volume pair, run the Failover operation. As part of this operation the administrator can choose to switch roles and enable replication in the reverse direction. If you choose to change the direction of the replication, you must wait until the state is synchronized for each shared volume pair.
  8. At the location where the Passive instance C is deployed, from the console bring up the Manage > Operations page of the Instance Console for Passive instance C, and run the Passive Takeover operation. This operation restarts the servers on the new Primary instance C, and clients can now be served from this cluster.

At this point, you can failover from Primary instance B to the newly created Passive instance C by using the steps outlined above.

Unplanned failover

You can fail back to a passive instance after an unrecoverable failure of primary instance.

Consider the following scenario:
  1. The primary configuration A is deployed as primary instance A.
  2. The passive configuration B is deployed and attached to the primary instance A, forming a Primary A/Passive B topology.
  3. Later, problems arise in the primary instance A: either the system where A is deployed cannot be recovered or instance A cannot be recovered.
To continue working, the administrator completes the following steps:
  1. On the system where the passive instance B is deployed, go to the Block Storage Replication page of the console and select the replication instance. For each shared volume pair, run the failover operation. The replication between the Primary A/Passive B block storage is removed.
  2. At the location where the Passive instance B is deployed, run the passive takeover operation. This operation starts the passive instance B. The passive instance B becomes the new primary instance. The clients can now be served from this cluster.
If you need to set the active-passive topology again, in a new rack or the rack where the primary configuration A was created, complete the steps that are described in Failing back to a new passive instance.