When you maintain your Kubernetes cluster that contains Automation Decision Services, you might need to upgrade the Kubernetes
cluster.
Before you begin
Before you begin the upgrade of the Kubernetes cluster, make sure that all the standby EDB
cluster members are up and running. Under normal circumstances, an EDB Postgres cluster can manage
if one member is down, but it is not ideal just before you start a Kubernetes upgrade. The status of
the cluster can be checked in the EDB Postgres CR.
About this task
The Procedure steps provide steps to consider when you are upgrading your Kubernetes cluster that
contains Automation Decision Services.
Procedure
-
Various Postgres deployments have a
PodDisruptionBudget (PDB) with minimum
available set to 1. If the replica size is also set to 1 for a small environment, then this prevents
node restarts without manual intervention if they contain the 1 pod for the deployment. If node
restart is blocked, it can prevent the Kubernetes upgrade or other cluster configuration
tasks.
To mitigate the problem, you can do one of these actions:
- Scale down the
postgresql-operator deployment to replica
0.
- Delete the PDB objects from the Automation Decision Services namespace with min available set to 1. The
PDB objects must be related to deployments owned by the cluster CR objects.
- Complete the Kubernetes upgrade or other task that requires a Kubernetes
restart.
- Scale the
postgresql-operator operator back up to replica 1 after the
upgrade.
- When the nodes attempt to restart during the upgrade or configuration task, delete the
pods from the Automation Decision Services namespace that are
stuck. The pods must be related to deployments owned by the cluster CR objects.