Safe Cluster Shutdown and Restart Automation Guidelines

Edit online

This section provides guidelines for automating a safe cluster shutdown procedure and restart procedure described in the section Safe Cluster Shutdown and Restart Procedures.

Note that these guidelines apply to restarting a healthy cluster only, one where all servers in the cluster are up and there is only one active server per stripe.

Safe cluster shutdown automation

Safe cluster shutdown involves the following steps:

  • STEP 1. Shut down the clients.
  • STEP 2. Shut down the passive servers.
  • STEP 3. Shut down the active servers.

The details for these steps are as follows:

  • STEP 1. Shut down the clients

    The Terracotta client will shut down when you shut down your application.

  • STEP 2. Shut down the passive servers

    Use the script stop-tc-server with the option --stop-if-passive to shut down a server only if it is in passive mode.

    Use a procedure indicated by the following pseudocode for shutting down all passive servers: 

    for each <server> in <running servers> {
   stop-tc-server --stop-if-passive <server> <args>
}

  • STEP 3. Shut down the active servers

    Use the script stop-tc-server with the option --stop-if-active to shut down a server only if it is in active mode.

    Use a procedure indicated by the following pseudocode for shutting down all active servers: 

    for each <server> in <running servers> {
   stop-tc-server --stop-if-active <server> <args>
}

Safe cluster restart automation

A safe cluster restart involves the following steps:

  • STEP 1. Start all servers in safe mode.
  • STEP 2. Make the previous active servers exit from safe mode.
  • STEP 3. Make the previous passive servers exit from safe mode.

The details for these steps are as follows:

  • STEP 1. Start all servers in safe mode

    The script start-tc-server with the option --safe-mode starts a server in safe mode.

    See the section Start and Stop Server Scripts (start-tc-server, stop-tc-server) in the Administration Guide for more information.

    Use a procedure indicated by the following pseudocode start all servers in safe mode: 

    for each <server> in <servers> {
   start-tc-server -f <config-file> -n <server> --safe-mode
}

    Wait for all servers to reach SAFE_MODE_STATE. The server state can determined by using the server-stat script.

    See the section Server Status (server-stat) in the Administration Guide for related information.

  • STEP 2. Make previous active servers exit from safe mode

    Use the exit-safe-mode script to make a server exit from safe mode.

    See the section Exit Safe Mode (exit-safe-mode) in the Administration Guide for related information.

    All previous active servers can be determined using the server-stat script. The server-stat script provides the state of a server prior to shutdown in the initialState field.

    See the section Server Status (server-stat) in the Administration Guide for related information.

    Use a procedure indicated by the following pseudocode to make previous active servers exit from safe mode: 

    <previous active servers> = []
for each <server> in <servers> {
   server-stat -s <server>:<management-port>
   add <server> to <previous active servers> if the initialState is Active state
}

for each <server> in <previous active servers> {
   exit-safe-mode -s <server>:<management-port>
}

  • STEP 3. Make previous passive servers exit from safe mode

    Use a procedure indicated by the following pseudocode to make previous passive servers exit from safe mode: 

    for each <server> in <previous passive servers> {
   exit-safe-mode -s <server>:<management-port>
}