Central manager redundancy

Edit online

Use Central Manager Redundancy or Backup Central Manager to configure a secondary or backup central manager in case the primary central manager becomes unavailable.

Central Manager Redundancy in Guardium® supports the following:

  • Backup central manager access to the Make Primary CM link is available after the primary central manager loses connection.
  • User layouts are retained.
  • User and roles are in the sync backup and do not rely on Portal User Sync.
  • User Group Roles Data is retained.
  • The API make_primary_cm allows you to switch the central manager from the CLI.
  • Data is retained from the Audit Process Builder processes after switching from the primary central manager to the backup central manager.
  • Central Management backup includes all the definitions (such as reports, queries, alerts, policies, and audit processes), users, and roles as it did before.
  • Schedules for enterprise reports, distributed reports, and LDAP are included.
  • Schedules for all audit processes, schedules and settings for data management processes (such as archive, export, backup, and import), and populate group members from query are included.
  • Settings for Alerter and Sender are included.
  • User GUI customization, custom classes, and uploaded JDBC drivers are included.
Note:
  • Data, either collected data, audit results, or custom tables data, is not included.
  • The top risky users list and threat cases are not copied to the backup central manager.
  • To list the status of cm_sync_file(s) on the backup central manager, use the following CLI command: show local_cm_sync_file.
  • To list the value of the backup central manager IP for each managed unit, use the following GuardAPI command: grdapi show_backup_cm_ip. This API command can only run on a central manager.
  • Failover with Central Manager load balancing: After failover, if the new managed units connect and then disconnect right away, the correct DB_USER is not sent until the failover message is received.
  • Switching to a backup central manager interrupts communication with collectors and might generate the following message: "Central manager experienced failed data transfer from collector." The issue is visible in the Scheduled Jobs Exceptions report and should clear within 24 hours.
  • 12.2 After switching from the primary central manager to the backup central manager, the cruise control functionality present in Kafka Cluster Management will not work on the new central manager.
  • 12.2.x and later After switching from the primary central manager to the backup central manager, you must manually restart Kafka Cluster and Kafka Cruise Control by using the Kafka Cluster Management UI.

Perform the following steps on your development or secondary servers and test. If successful, then perform these steps on your primary or live Guardium servers.

Install patches on the central manager
  1. From the current primary central manager, log in as CLI.
  2. Install patches with the following CLI command: store system patch install scp
  3. The CLI command copies the files over to your Guardium server so you can install them.
  4. To watch the patch installation, use the following CLI command: show system patch install
  5. Wait until the patch status displays the message “DONE: Patch installation Succeeded” for both patches.
Install patches on the backup central manager
  1. Log in to the now primary central manager GUI as admin.
  2. Navigate to Manage > Central Management > Central Management.
  3. Click the check box for the backup central manager managed unit only on the central manager.
  4. Click Patch Distribution and install all of the patches that you just installed onto the primary central manager.

Patch installation example:

  1. Click Patch Distribution.
  2. Click Install Patch Now.
  3. Wait approximately 15 minutes to be sure the patch is installed on all managed servers.
  4. To verify, log in as CLI on the backup central manager and run the CLI command show system patch install from the backup central manager server.
Optional: Install patches on all other managed servers
  1. Repeat the previous steps to install patches on all managed servers.
  2. Verify that all patches are installed before starting the next procedure.
After all patches are installed on the central maanger and managed servers
  1. Log in as admin onto the now primary central manager.
  2. Navigate to Manage > Central Management > Central Management.
  3. Click Designate Backup CM.
  4. Select the backup central manager server from the list of eligible backup central manager candidates.
  5. Click Apply.
  6. Wait approximately two minutes for the backup central manager to sync and the new backup central manager file to be created and copied to the backup central manager.
  7. Wait for two complete rounds of backups to complete (approximately one hour). Two backup central manager sync files will be copied to the backup central manager.
  8. To view the progress of the creation of the backup central manager sync files, click the Guardium Monitor tab and select Aggregation/Archive Log Report.
  9. Verify that the activity backup started and the cm_sync_file.tgz file was created from the Aggregation/Archive Log Report.
    1. Log in as admin from the GUI.
    2. Click the Guardium Monitor tab.
    3. Select Aggregation/Archive Log Report.
    4. Look for the backup types.
The process is complete when:
  • The patches are installed on the central manager.
  • The patches are installed on the backup central manager.
  • Optional: The patches are installed on all other managed units.
  • Two backup central manager sync files are complete, as indicated in the Aggregation/Archive Log Report.
Important:
  • Wait approximately one hour to be sure at least two of the backup central manager sync files supporting the backup central manager are complete.
  • The backups schedule for backup central manager sync files is approximately every 30 minutes.
  • The process runs on the central manager to create a backup central manager file and copies that file to the directory on the backup central manager.

The following section outlines the process to convert the now primary central manager and its managed nodes to the backup central manager.

Start the backup central manager process after two sync file process are complete
  1. Shut down the primary central manager Guardium server.
    Note: If you do not have access to shut down the primary central manager, go directly to the backup central manager and log in as admin. Navigate to Manage > Central Management > Central Management and click Make Primary CM. Then, skip to the Steps to start the backup central manager configuration to become the primary central manager section of this document.
  2. Wait approximate five minutes and log in again as admin in the GUI of the backup central manager.
  3. After the primary central manager is shut down completely, you can continue to the steps in the next section.
Note: If you are logged into the primary central manager and it goes down, you receive a message indicating that the connection timed out.
Steps to start the backup central manager configuration to become the primary central manager

The secondary central manager will not be responsive for approximately five minutes. When you log in as admin after five minutes, the Make Primary CM link will be available at Manage > Central Management > Central Management.

When the primary server shuts down, you receive the following message on the backup central manager: “Unable to connect to Remote Manager, consider switching to (the name of the backup CM)".

If you decide to switch:

  1. Log in as admin.
  2. Navigate to Manage > Central Management > Central Management.
  3. Click Make Primary CM.
    Note: Do not click Make Primary CM more than once. Also stay on the screen and do not select anything else during the running of this process. A log file will be created that you can view to see the progress and completion of this process. Be patient as this process will take awhile to complete. There is a safeguard that if you click Make Primary CM more than once, nothing will change with the currently running process.
  4. Within seconds, you receive the following message: "Are you sure you want to make this unit the primary CM?" Click OK.
  5. Within a few seconds more, you receive a message that states "This may take a few minutes." Click OK to create the load_secondary_cm_sync_file.log file, which shows you the progress of the backup central manager switch process. (See the following section, View the progress log file while the central manager backup process is running to learn how to view the file from your GUI.)
    Note: The amount of time it takes for the backup central manager to become the primary central manager depends on the amount of data backed up from the backup central manager sync file and the amount of managed nodes that switch to the backup central manager, which will become the primary central manager.
  6. The last message before the backup central manager switch is complete will take awhile to appear on the screen: “GUI will restart now. Try to log in again in a few minutes and the Backup CM will now become the Primary CM”. When it appears, click OK and wait a few minutes for the backup central manager to become primary and for all the managed nodes to complete the switch over to the new primary central manager.
View the progress log file while the central manager backup process is running

While the Make Primary CM process is running, you can view the progress of the backup central manager becoming the primary central manager. To do so, you need the IP of the server you are connected to in order to view the log files.

  1. Login as CLI from your backup central manager server from a Putty.exe session
  2. From the CLI, run fileserver <IP> “enter your IP number” 3600". For example: fileserver 9.70.32.122 3600
  3. From the GUI, enter the value http://yourserver.x.x.x.com (the value will display in the CLI screen after you enter the command). For example: http://joe.server.guardium.com (the server name will be the backup central manager server). The Fileserver window on the UI will open to select file – Select Sqlguard logs.
  4. Select the file load_secondary_cm_sync_file.log (the file will display in a list of files from Step 3). This allows you to view the progress of the backup central manager becoming the primary central manager. Locating the log file to view the central manager backup process is complete when you see the following line in the load_secondary_cm_sync_file.log file: Import CM sync info - DONE.
  5. Wait approximately 10 minutes for all of the managed units to become available to the new primary central manager.
After the backup central manager becomes the primary and all managed nodes are now managed by the backup central manager server

You can now bring up the old primary central manager server. After it is up and running, you can add it as the backup central manager server.

  1. Reboot the old primary central manager.
  2. After the server is up, log in as CLI.
  3. To delete the manager unit type, enter delete unit type manager. After it completes, you receive an OK message from the CLI.
  4. Important: Wait approximately five minutes for the GUI to completely restart even after the deleted unit type displays a successful message and the GUI restart message.
  5. After you wait five minutes for the GUI to completely restart, log in to the new primary central manager to register the old primary central manager as a managed unit.
    1. Log in as admin on the new primary central manager.
    2. Navigate to Manage > Central Management > Central Management.
    3. Click Register New.
    4. Enter the IP of the old primary central manager that you just rebooted.
    5. Enter 8443 as Port.
    6. Click Save.
      Important: Be patient, do not click the Save button twice.
    7. Wait a minute for the old primary central manager to register.
  6. To make the old primary central manager a new backup central manager:
    1. Click Designate Backup CM.
    2. Select the old primary central manager server.
    3. Click Apply. The old primary central manager server is now the new backup central manager server.
  7. Refresh the Central Management screen to see the new unit type Backup CM defined.
This task is complete.
Report data after the Backup Central Manager process is complete

The following data is missing after the Backup Central Manager process is complete. This is related to only the "first" switch from the primary to the secondary central manager.

  • Audit Process Results
  • Custom Table Data
  • Custom Report Data
  • VA Results
  • Classifier Results
  • DSD Results
  • CAS results
  • Datamart Data
  • Collected Data
  • Entitlement Data

The reports are repopulated after you run the reports again on the new primary central manager. If you switch back to the old primary central manager, the report data is presented.