Recovering from a disaster on Cloud Pak for Integration

Prepare your Cloud Pak for Integration environment for installation of the replacement API Connect deployment. Install API Connect. Restore the database backups of each subsystem.

About this task

To recover from a disaster, re-create your API Connect deployment, referring to the backup files and artifacts you took when you followed the backup process: Backing up your Cloud Pak for Integration API Connect deployment.

Note: Your recovery installation of Cloud Pak for Integration must include the Platform UI if your original deployment used it. In addition, the CR must use the same settings as in the original deployment.

Procedure

Preparing the Cloud Pak for Integration environment.

Prepare your Cloud Pak for Integration environment for deploying API Connect as explained in the appropriate version of the Cloud Pak for Integration documentation.

After you prepare your environment, return to this procedure to configure the installation CR, deploy API Connect, and then restore the subsystem database backups to your API Connect subsystems.
Create management and portal encryption secrets from your backups of these secrets.
1. Create the management database encryption secret.
```
oc -n <namespace> apply -f management_enc_key_backup.yaml
```
  where management_enc_key_backup.yaml is the encryption secret backup that you created in step 2.c of Taking a management infrastructure configuration backup
2. Create the portal database encryption secret.
```
oc -n <namespace> apply -f portal_enc_key_backup.yaml
```
  where portal_enc_key_backup.yaml is the encryption secret backup that you created in step 2.c of Taking a portal infrastructure configuration backup.
Create the Cloud Pak for Integration credentials secret.
```
oc -n <namespace> apply -f cp4i_credentials_secret_backup.yaml
```
where cp4i_credentials_secret_backup.yaml is the credentials backup file that you created in step 1 of Backing up your Cloud Pak for Integration API Connect deployment.
Generate the Kubernetes secret for each subsystem's database backups, by using the backup files that you created for each subsystem.
1. ```
oc -n <namespace> apply -f mgmt-backup-secret_backup.yaml
```
  where mgmt-backup-secret_backup.yaml was created in step 3 on Taking a management infrastructure configuration backup.
2. ```
oc -n <namespace> apply -f ptl-backup-secret_backup.yaml
```
  where ptl-backup-secret_backup.yaml was created in step 3 on Taking a portal infrastructure configuration backup.
3. ```
oc -n <namespace> apply -f analytics-backup-secret_backup.yaml
```
  where analytics-backup-secret_backup.yaml was created in Configuring analytics backups.

Create each of the management client application credential secrets.

Run the following command to apply each secret, by using the backup YAML files that you created in step 5 of Taking a management infrastructure configuration backup.

oc -n <namespace> apply -f <secretName>.yaml

Example:

atmCredentialSecret: management-atm-cred
consumerToolkitCredentialSecret: management-ccli-cred
consumerUICredentialSecret: management-cui-cred
designerCredentialSecret: management-dsgr-cred
discoveryCredentialSecret: management-discovery-cred
governanceCredentialSecret: management-governance-cred
juhuCredentialSecret: management-juhu-cred
toolkitCredentialSecret: management-cli-cred
uiCredentialSecret: management-ui-cred

Prepare draft API Connect installation YAML file.

Log in to the IBM Cloud Pak Platform UI and create a draft API Connect YAML.

In this step, you complete the initial steps for deploying API Connect in Cloud Pak for Integration, but you do not begin the actual installation until step 8. Instead, you update the draft API Connect YAML to match the API Connect YAML you backed up in Backing up your Cloud Pak for Integration API Connect deployment.
1. Log in to the Platform UI.
2. On the home page, click Create an instance.
3. Select the API management tile and click Next.
4. On the Create an instance of API management page, select the deployment type, and click Next.
  
  Choose the same deployment type that you used when you originally deployed API Connect.
5. On the deployment settings page, click the YAML tab to edit the installation CR in YAML format.

Update draft API Connect YAML with values from your CR backup files.

Refer to your CR backup files apic_cr_backup.yaml and <subsystem>_cr_backup.yaml to update the properties in your draft YAML.
1. Set metadata.name to the value from your backup apic_cr_backup.yaml.
2. Set the management and portal encryption secrets.
  Set spec.<subsystem>.encryptionSecret.secretName with the secrets you created in step 2, as shown in the following example:
```
spec:
  management:
    encryptionSecret:
      secretName: mgmt-enc-key
  portal:
    encryptionSecret:
      secretName: portal-enc-key
```
3. Set each of the credential secrets in spec.management section, to the secrets you created in step 5.
  Example:
```
spec:
  management:
    customApplicationCredentials:
    - name: atm-cred
      secretName: management-atm-cred
    - name: ccli-cred
      secretName: management-ccli-cred
    - name: cui-cred
      secretName: management-cui-cred
    - name: dsgr-cred
      secretName: management-dsgr-cred
    - name: governance-cred
      secretName: management-governance-cred
    - name: juhu-cred
      secretName: management-juhu-cred
    - name: cli-cred
      secretName: management-cli-cred
    - name: ui-cred
      secretName: management-ui-cred
 
```
4. If you are restoring your deployment in a different data center, explicitly set your endpoints to prevent auto-generation of different endpoints.
  
  If you are restoring your deployment in a different data center, the endpoints that are used in your original deployment must be the same in your new data center. The Platform UI in Cloud Pak for Integration auto-generates the endpoints if they are left empty in the YAML, therefore ensure that you explicitly set the endpoints in the CR to match what is set in your <subsystem>_cr_backup.yaml files.
  For example:
```
apiManagerEndpoint:
      annotations:
        cert-manager.io/issuer: prod-ingress-issuer
        haproxy.router.openshift.io/timeout: 240s
      hosts:
        - name: prod-api-manager.example.com
        secretName: prod-be787dd3-api-manager
```
5. Set the subsystem database backup and restore settings to match what you have in apic_cr_backup.yaml.
  
  The spec.<subsystem name>.databaseBackup sections for each subsystem contain the database backup settings.

Start API Connect deployment.

On the deployment page, install API Connect by clicking Create.
Wait for all subsystems to be created and ready. Monitor the installation progress with the command:
```
oc -n <namespace> get ApiconnectCluster
```
Restore the subsystems' database backups in the following sequence:
1. Restoring the management database with a backup ID.
  Verify that the restoration completed successfully and that the management subsystem is Ready, then proceed to the next step and restore the portal subsystem.
2. Restoring from a portal database backup.
  Use the restore type all to ensure that you restore the complete subsystem and all portal sites.
3. Restoring the analytics database.
  Use the restore type replace to ensure that you restore all analytics data.
Force the configurator to run again.

The cluster will still not be ready after restore at this stage because the configurator has yet to successfully complete. For the configurator to run again, delete the associated job so that a new pod starts running:
1. Run the following command to get the list of jobs:
```
oc get jobs -n <APIC_namespace>
```
2. Run the following command to determine the name of your API Connect instance:
```
oc get apiconnectcluster -n <APIC_namespace>
```
3. Run the following command to delete the configurator job:
```
oc -n <APIC_namespace> delete job <instance_name>-configurator
```
Verify that the recovery was successful:
1. Ensure that you can log in to the Cloud Manager UI.
2. Verify that your provider organizations exist.
3. Ensure that you can log in to each portal site.
4. Ensure that the analytics dashboard contains your restored analytics data.