StatefulSet pods with local storage are stuck in Pending state

StatefulSet pods using local storage remain bound to failed nodes.


If you are using local storage and a node in the cluster goes down, then StatefulSet pods remain bound to that node. These pods are unable to restart on another node and are stuck in Pending state because they have a Persistent Volume on the node that is down.


The persistent volumes and persistent volume claims for these pods must be removed to allow the pods to be reassigned to other nodes. There is a script here that can be used to do this.

Run the script with the following command:
Draft comment: LOUISERoberts
#4856 Needs triage/work