Restoring the management subsystem on OpenShift and Cloud Pak for Integration

You can restore your Management subsystem in your OpenShift and Cloud Pak for Integration environments.

Before you begin

You must have backups of the management subsystem in order to restore. See Configuring backup settings for a fresh install of the Management subsystem on OpenShift or Cloud Pak for Integration and Generating a manual backup of the management subsystem on OpenShift and Cloud Pak for Integration.

About this task

To trigger a restore, you will create a ManagementRestore custom resource. When the ibm-apiconnect operator detects the new CR, it starts the restore process.

  • Ensure that the backup method that you used is complete before you attempt a restore. The backup data that is stored in the remote server is used for restoring the Management subsystem data back to a previous state. Ensure that these backups are on your remote server before you attempt a restore, and that the backup ID in the selected backup exists in remote storage.
  • Restoring a backup restores the registration credentials (client_ID, client_secret) that were in use at the time that the selected backup was created. For information on the registration credentials, see Changing the registration client_id and client_secret for applications.
  • If you have to perform a restore, you must complete the restoration of the Management Service first, and then immediately restore the Developer Portal. The backups of the Management and Portal must be taken at the same time to ensure that the Portal sites are consistent with Management database.
Note: There is a known issue where sometimes a restore from the Management subsystem's SFTP backup succeeds but the data is not restored. If this happens, run the restore again.

Procedure

  1. Use the Form UI or the oc command to obtain a list of the available backups, and decide which one to restore your Management subsystem:
    oc get mgmtb -n <APIC_namespace>

    For example:

    $ oc get mgmtb
    NAME                  STATUS   ID                 CLUSTER      TYPE   CR TYPE   AGE
    management-3aa3bebf   Ready    20200929-152150F   management   full   record    10h
    management-3c4ca8df   Ready    20200929-115520F   management   full   record    20h
    management-5tbxj      Ready    20200929-224349F   management   full   create    17h
    management-f10af337   Ready    20200925-133703F   management   full   record    5d2h
    management-jtlcd      Ready    20200929-224349F   management   full   create    25h
    management-zxjtz      Ready    20200929-224349F   management   full   create    28h
    
    Attention: You can restore the Management subsystem from any backup where the STATUS is Ready; however, you should avoid restoring to the initial system backup. During installation, the Management database is used for an initial system backup before certain database schema jobs are complete. Restoring to this backup will result in an unstable system.

    You will use one of the values in the NAME column.

  2. Use the instructions that apply to your release of API Connect:
    • Version 10.0.1.1 or later
      1. Use the following code as an example:
        apiVersion: management.apiconnect.ibm.com/v1beta1
        kind: ManagementRestore
        metadata:
          generateName: management-
        spec:
          backupName: management-3aa3bebf
          
        Note that backupName: is the name of the backup custom resource you want to restore to. For example, using the output from the previous step:
        backupName: management-3aa3bebf
      2. Use the Form UI or oc to create the ManagementRestore custom resource in the namespace of the Management Subsystem, to trigger the restore process:
        $ oc create -f mgmtrestore_cr.yaml -n <APIC_namespace>
        
      3. You can list current restores using the following command, or if using OCP UI you can list all ManagementResource list:
        $ oc get managementrestore -n <APIC_namespace> --sort-by=.metadata.creationTimestamp

        Example output:

        • SFTP:
          NAME                 STATUS      BACKUP              CLUSTER      MESSAGE                                            PITR     AGE
          mgmt-restore-ptx6d   Complete    mgmt-backup-w4h8c   m1           Restore process completed (DB Restore + DRR job)            6m31s        
        • S3
          NAME                 STATUS      BACKUP              CLUSTER       MESSAGE                                            PITR                    AGE
          mgmt-restore-hr9zx   Complete    mgmt-backup-xnjkl   m1            Restore process completed (DB Restore + DRR job)   2020-11-23 23:08:32+00  9m8s
    • Version 10.0.1.0
      1. Use the following command to obtain the Point-in-Time-Recovery target for the selected backup:
        oc get mgmtb <backup_name> -o jsonpath="{..status.info.range}"

        For example:

        $ oc get mgmtb management-3aa3bebf -o jsonpath="{..status.info.range}"
        2020-09-29 15:21:50 +0000 UTC - 2020-09-29 15:30:56 +0000 UTC

        The target for recovery is the second half the range value in the backup custom resource. In the prior example: 2020-09-29 15:30:56 +0000.

        The custom resource uses the pitrTarget setting to specify the Point-in-Time-Recovery target. You must set pitrTarget to be later than the value obtained above (such as by one second newer). Use the format YYYY-MM-DD HH:MM:SS+Z.

        Using the example above, where 2020-09-29 15:30:56 +0000 is the range, the pitrTarget is 2020-09-29 15:30:57+00:

          pitrTarget: "2020-09-29 15:30:57+00"
      2. Use the following code as an example:
        apiVersion: management.apiconnect.ibm.com/v1beta1
        kind: ManagementRestore
        metadata:
          generateName: management-
        spec:
          backupName: management-3aa3bebf 
          pitrTarget: "2020-09-30 13:53:21+00" ## Not required in 10.0.1.1-eus and greater  
        • backupName - The name of the backup custom resource you want to restore to. For example, using the output from the previous step, management-3aa3bebf.
        • pitrTarget: - The time you want to restore to, in format YYYY-MM-DD HH:MM:SS+Z.
      3. Use the Form UI or oc to create the ManagementRestore custom resource in the namespace of the Management Subsystem, to trigger the restore process:
        $ oc create -f mgmtrestore_cr.yaml -n <APIC_namespace>
        
      4. You can list current restores using the following command, or if using Form UI you can list all ManagementResource lists:
        $ oc get managementrestore -n <APIC_namespace> --sort-by=.metadata.creationTimestamp

        Example output:

        NAME                 STATUS      BACKUP              CLUSTER      PITR                     AGE
        management-6nzjr     Complete    mgmt-backup-dg4s4   m1           2020-01-05 19:05:00+01   14m
      5. The operator creates an upgrade CR when the postgres database is restarted during the restore:
        managementdbupgrade.management.apiconnect.ibm.com/m1-up-zqkk5   Complete   Fresh install is Complete (DB Schema/data are up-to-date)
        63s5s