Cleaning up an instance after provisioning or deletion fails (Streams)

Deleting an IBM Streams service instance that failed to provision or failed to delete completely can leave backend resources running that must be cleaned up.

Symptoms

You might experience the following symptoms:

You provisioned a service instance and the provision failed.
You used the Cloud Pak for Data web user interface to delete the Streams service instance:
- The delete request hangs and the Kubernetes pods are not deleted.
- The delete request fails and the instance is still displayed on the Instances page.

Causes

A service instance that failed to be provisioned or deleted leaves resources running in the background that must be cleaned up.

Resolving the problem

Note: The sample commands in the following procedure assume that you provisioned an IBM Streams service instance called sample in the samplens namespace.

Delete the Streams service instance from the Cloud Pak for Data web user interface.
1. From the navigation menu, click Services > Instances.
2. From the service instance options menu for the sample service instance, click Delete.
If the deletion fails, proceed with the following steps:

Delete any Helm releases for this Streams service instance. Use the exec command to access the operations pod and view and delete any Helm releases:

$ oc exec -it $(oc get pods -l app.kubernetes.io/name=streams-addon-service-provider --no-headers --namespace samplens | awk '{print $1}') -- /bin/bash
$ helm list --tls | grep sample
build-sample    1     Mon Apr 13 19:09:07 2020      DEPLOYED     ibm-streams-build-1.3.0         5.3.0    samplens
sample          1     Mon Apr 13 19:09:08 2020      FAILED       ibm-streams-instance-1.3.0      5.3.0    samplens
$ helm delete sample --purge --tls --no-hooks
release "sample" deleted
$ helm delete build-sample --purge --tls --no-hooks
release "build-sample" deleted

Remove any jobs for the Streams service instance.

Use the following Kubernetes command to view the jobs. In this example, there are three jobs.

$ oc get job -l release=sample && oc get job -l release=build-sample
sample-default-security              1/1           21s        40m
sample-mkinstance                    0/1           39m       39m
build-sample-default-security        1/1           23s        40m

Use the following Kubernetes commands to delete any existing jobs. In this example, all instance and build jobs are deleted.

$ oc delete job -l release=sample
$ oc delete job -l release=build-sample

Remove any Streams application and builder pods for this Streams service instance.

Use the following Kubernetes command to view any Streams application and builder pods. In this example, there is one of each.

$ oc get pods --namespace samplens -l streams-instance=sample,streams-app=active && oc get pods --namespace samplens -l streams-build=build-sample,streams-builder=active
sample-app-0                         1/1       Running     0          3m
build-sample-builder-0               1/1       Running     0          2d

Use the following Kubernetes commands to delete any Streams application and builder pods that exist. In this example, all active application and builder pods are deleted.

$ oc delete pods -l streams-instance=sample,streams-app=active
$ oc delete pods -l streams-build=build-sample,streams-builder=active

Delete any remaining Helm release objects for this Streams service instance.
- For instances, use the following command to list any statefulset, service, or job objects:
```
oc get all --namespace samplens -l release=sample
```
  Use the following commands to delete any existing statefulset, service, or job objects, in the following order:
```
oc delete statefulset -l release=sample
oc delete service -l release=sample
```
- For the build service, use the following command to list any statefulset, service, or job objects:
```
oc get all --namespace samplens -l release=build-sample
```
  Use the following commands to delete any existing statefulset, service, or job objects, in the following order:
```
oc delete statefulset -l release=build-sample
oc delete service -l release=sample
```
Delete any configmap objects for this Streams service instance.
- For instances, use the following command to list any configmap objects:
```
$ oc get cm --namespace samplens -l release=sample
```
  Use the following command to delete any existing configmap objects:
```
$ oc delete cm --namespace samplens -l release=sample
```
- For the build service, use the following command to list any configmap objects:
```
$ oc get cm --namespace samplens -l release=build-sample
```
  Use the following command to delete any existing configmap objects:
```
$ oc delete cm --namespace samplens -l release=build-sample
```
Delete any secret objects for this Streams service instance.
- For instances, use the following command to list any secret objects:
```
$ oc get secrets --namespace samplens -l release=sample
```
  Use the following command to delete any existing secret objects:
```
$ oc delete secrets --namespace samplens -l release=sample
```
- For the build service, use the following command to list any secret objects:
```
$ oc get secrets --namespace samplens -l release=build-sample
```
  Use the following command to delete any existing secret objects:
```
$ oc delete secrets --namespace samplens -l release=build-sample
```
Delete any persistent volume claims (PVCs) that were manually and dynamically created when this Streams service instance was provisioned.
- For instances, use the following command to list any PVCs:
```
$ oc get pvc --namespace samplens -l release=sample
```
  Use the following command to delete any existing PVCs:
```
$ oc delete pvc --namespace samplens -l release=sample
```
- For the build service, use the following command to list any PVCs:
```
$ oc get pvc --namespace samplens -l release=build-sample
```
  Use the following command to delete any existing PVCs:
```
$ oc delete pvc --namespace samplens -l release=build-sample
```
Try again to delete the Streams service instance from the Cloud Pak for Data web user interface.
1. From the navigation menu, click Services > Instances.
2. From the service instance options menu for the sample service instance, click Delete.