ACM observability label-based deployment

Diagnose and resolve issues encountered during observability stack deployment to managed clusters.

The observability stack deploys to managed clusters when the following label is added to the cluster:

observability.sovereign.cloud.ibm.com/enabled=true

Resolution:

Follow these steps in sequence to troubleshoot deployment issues on the managed cluster:

Confirm that the observability label was added to the managed cluster.
```
oc get managedcluster <cluster-name> --show-labels | grep observability
```
Replace cluster-name with your managed cluster name.
Verify that the cluster appears in the placement decisions.
```
oc get placementdecisions -n observability -o jsonpath='{range .items[*].status.decisions[*]}{.clusterName}{"\n"}{end}' | grep <cluster-name>
```
If the cluster is not listed:
- Wait 1-2 minutes for the placement to update automatically.
- If the cluster is still not listed, check cluster availability:
```
oc get managedcluster <cluster-name>
```
Review the status of observability policies:
```
oc get policies -n observability
```
Find the policies with a status of NonCompliant. To examine a specific policy:
```
oc describe policy <policy-name> -n observability
```

Retrieve detailed information about policy compliance and violations:

oc get policy <policy-name> -n <cluster-namespace> -o jsonpath='{.status.compliant}
oc get policy <policy-name> -n <cluster-namespace> -o jsonpath='{.status.details}'

Replace policy-name and cluster-namespace with the appropriate values.

Connect to the managed cluster and verify policy status directly:
```
oc get policy -A
oc describe policy <policy-name> -n <namespace>
```
If the previous steps do not resolve the issue, reset the observability label:
1. Remove the observability label from the managed cluster.
2. Wait 2 minutes for the system to process the change.
3. Add the observability label again:
```
observability.sovereign.cloud.ibm.com/enabled=true
```