Upgrading Db2 or Db2 Warehouse HADR deployments- Failure scenarios

When upgrading Db2 or Db2 Warehouse HADR deployments, may come across some failure scenarios on your primary or standby database instances.

1. On your primary database instance, the db2ckupgrade commands runs the Kubernetes upgrade job. If the upgrade job fails, review the logs in the ${SUPPORTDIR}/upgrade_update.log file to identify the issue.
2. Exec into the Db2 engine pod as the db2inst1 user and run the db2_update_upgrade command to re-run the instance upgrade. If the re-run operation fails, you must failover to the standby instance by redirecting all your applications.
3. Open a support case with IBM to review the primary database instance.
4. If high availability is not available and you do not have access to your primary and standby databases:
   1. Deploy a new Db2 instance for the version prior to your upgrade. For more information, see Deploying Db2 or Db2 Warehouse.
   2. Restore your database from a backup image. For more information, see Backing up and restoring Db2 or Db2 Warehouse.
   3. On the standby database instances, Run the manage_hadr -stop command from the Db2 engine pod to stop HADR.
   4. Run the su - db2inst1 -c "db2 rollforward db <DBNAME> complete" command to rollforward the database to complete. For more information, see ROLLFORWARD DATABASE command.
   5. Perform all the steps in Configuring and starting HADR.
5. On your primary database instance, if the db2_update_upgrade --databases command fails, review the logs in the ${SUPPORTDIR}/upgrade_update.log file to identify the issue.
6. Exec into the Db2 engine pod as the db2inst1 user and run the db2_update_upgrade --databases command to re-run the database upgrade. If the re-run operation fails, you must failover to the standby instance by redirecting all your applications.
7. Follow instructions in Step 3 and Step 4.
8. On your primary database instance, if the db2_update_upgrade --post-upgrade command fails, review the logs in the ${SUPPORTDIR}/upgrade_update.log file to identify the issue.
9. Exec into the Db2 engine pod as the db2inst1 user and run the db2_update_upgrade --post-upgrade command to re-run the instance upgrade. If the re-run operation fails, you must failover to the standby instance by redirecting all your applications.
10. Follow instructions in Step 3 and Step 4.
11. On your standby database instance, the db2ckupgrade commands runs the Kubernetes upgrade job. If the upgrade job fails, review the logs in the ${SUPPORTDIR}/upgrade_update.log file to identify the issue.
12. Exec into the Db2 engine pod as the db2inst1 user and run the db2_update_upgrade command to re-run the instance upgrade. If the re-run operation fails, you must failover to the standby instance by redirecting all your applications.
13. Open a support case with IBM to review the standby database instance.
14. If high availability is not available and you do not have access to your primary and standby databases:
    1. Deploy a new Db2 instance for version s12.1.0.0. For more information, see Deploying Db2 or Db2 Warehouse.
    2. On the primary database instances, Run the manage_hadr -stop command from the Db2 engine pod to stop HADR.
    3. Run the su - db2inst1 -c "db2 rollforward db <DBNAME> complete" command to rollforward the database to complete. For more information, see ROLLFORWARD DATABASE command.
    4. Perform all the steps in Configuring and starting HADR.
15. On your standby database instance, if the db2_update_upgrade --databases command fails, review the logs in the ${SUPPORTDIR}/upgrade_update.log file to identify the issue.
16. Exec into the Db2 engine pod as the db2inst1 user and run the db2_update_upgrade --databases command to re-run the database upgrade. If the re-run operation fails, you must failover to the standby instance by redirecting all your applications.
17. Follow instructions in Step 3 and Step 4.
18. On your standby database instance, if the db2_update_upgrade --post-upgrade command fails, review the logs in the ${SUPPORTDIR}/upgrade_update.log file to identify the issue.
19. Exec into the Db2 engine pod as the db2inst1 user and run the db2_update_upgrade --post-upgrade command to re-run the instance upgrade. If the re-run operation fails, you must failover to the standby instance by redirecting all your applications.
20. Follow instructions in Step 3 and Step 4.