Manual failover and failback

Edit online

You can complete manual failover and failback with the coordinator service APIs. You can also simulate failover and failback with the haproxy.cfg file.

API method

The coordinator service enables manual failback and failover with its APIs. For the manual failover, primary and backup service states must be in Primary Maintenance and Acting Primary No Auto Failback states. Complete the following steps for manual failover:
  1. Set the primary state to Primary Maintenance by running the command:
    curl -u <api_username>:<api_password> -X POST https://<primary_coordinator_route_url>/coordinator/operationalState?state=PRIMARY_MAINTENANCE
    Verify the state by running the command:
    curl -X -u <api_username>:<api_password> GET http://<primary_coordinator_route_url>/coordinator/operationalState
  2. Set the backup state to Acting Primary No Auto Failback by running the command:
    
curl -u <api_username>:<api_password> -X POST https://<backup_coordinator_route_url>/coordinator/operationalState?state=ACTING_PRIMARY_NO_AUTO_FAILBACK
    Verify the state by running the command:
    curl -X -u <api_username>:<api_password>  GET http://<backup_coordinator_route_url>/coordinator/operationalState

For manual failback, the primary and backup service states should be in Primary and Backup state.

Complete the following steps for manual failback:
  1. Set the backup state to Backup by running the command:
    
curl -u <api_username>:<api_password> -X POST https://<backup_coordinator_route_url>/coordinator/operationalState?state=BACKUP
    and verify the state by running the command:
    curl -X -u <api_username>:<api_password>  GET http://<backup_coordinator_route_url>/coordinator/operationalState
  2. Set the primary state to Primary by running the command:
    curl -u <api_username>:<api_password> -X POST https://<primary_coordinator_route_url>/coordinator/operationalState?state=PRIMARY
    and verify the state by running the command:
    curl -X -u <api_username>:<api_password>  GET http://<primary_coordinator_route_url>/coordinator/operationalState

Health Check

The coordinator service health can be checked by running the curl command:
curl -X  GET http://<primary/backup_coordinator_route_url>/coordinator/status/health

Stateful set method

You can simulate a failover and failback with the haproxy.cfg file by scaling down the coordinator stateful set. Run the following command:
oc scale sts <dr_coorindator_sts> --replicas=0
Running this command simulates a site failure and causes the cluster to failover. The dashboard displays a banner stating that the clusters have failed over.