On 15 November 2022, IBM introduced the enhanced Ingress Status for IBM Cloud Kubernetes Service and Red Hat OpenShift on IBM Cloud clusters.

The enhanced Ingress Status provides granular status reports for your Ingress components, including ALBs, subdomains and secrets. 

Additionally, a set of new step-by-step troubleshooting guides is available in our documentation that will help narrow down and resolve issues reported by the Ingress Status. To make Ingress Status more flexible and suitable for all of your use cases, we also introduced new configuration options.

Where can I see the status report?

You can use the ibmcloud ks ingress status-report get CLI command to see the Ingress Status of your cluster. We grouped up the reports to provide a clear and readable view of your resources:

➜  ~ ibmcloud ks ingress status-report get -c example
OK

Ingress Status:   warning
Message:          Some Ingress components are in warning state check `ibmcloud ks ingress status-report get` command.

Cluster                        Status
ingress-controller-configmap   healthy
alb-healthcheck-ingress        The ALB health ingress resource is not found on the cluster (ERRAHINF). See: https://ibm.biz/ts-ingress-errahinf
                               The ALB health service is not found on the cluster (ERRAHSNF). See: https://ibm.biz/ts-ingress-errahsnf

Subdomain                                                  Status
example-subdomain-0000.region.containers.appdomain.cloud   healthy
example-subdomain-0001.region.containers.appdomain.cloud   The subdomain has DNS resolution issues (ERRDRISS). See: https://ibm.biz/ts-ingress-errdriss
                                                           The subdomain has TLS secret issues (ERRDSISS). See: https://ibm.biz/ts-ingress-errdsiss

ALB                     Status
public-crexample-alb1   The ALB is unable to respond to health requests (ERRAHCF). See: https://ibm.biz/ts-ingress-errahcf
public-crexample-alb2   One or more ALB pod is not in running state (ERRADRUH). See: https://ibm.biz/ts-ingress-erradruh
                        The ALB is unable to respond to health requests (ERRAHCF). See: https://ibm.biz/ts-ingress-errahcf

Secret             Namespace        Status
example-secret-1   ibm-cert-store   healthy
example-secret-1   kube-system      healthy

What exactly can be seen here?

The first group at the top is the overall state of the Ingress Status. We added a few new states compared to the previous Ingress states. Now the possible states are the following:

• healthy: All Ingress components are looking good, or you chose to ignore the reported problems.
• warning: Some Ingress components have issues.
• critical: Some Ingress components are not functioning at all.
• unsupported: The cluster runs a version that is not supported for Ingress Status.
• disabled: Ingress Status reporting is disabled for the corresponding cluster.

We provide reports on more Ingress-related general components than before. This includes the Ingress operator in the case of Red Hat OpenShift clusters and, for Kubernetes clusters, the ALB health checker and configuration resources.

Now we have reports on the subdomains that are registered for the cluster. It might include warnings like: DNS registration issues, IP or hostname mismatch problems and many more.

We improved our Ingress report by breaking it down to each managed Ingress Controller (ALB or router) found on the cluster. The reports now can contain warnings like the following:

• The Ingress Controller is not running or has high availability issues.
• The Ingress Controller has an invalid configuration.
• The Ingress Controller does not respond to HTTPS requests.

We also added the managed secrets-related reports for this output. The following are a few of the warning you can see:

• The managed secrets will expire soon or already expired.
• The cluster has communication problems with the IBM Cloud Secrets Manager instances.
• The cluster has secret synchronization issues.

Improved troubleshooting documentation

Each warning that Ingress Status might report has a step-by-step troubleshooting guide. The Ingress Status printout gives a short description of the problem that was found, and next to the description is a URL that takes you to the appropriate troubleshooting guide to solve the issue.

Each troubleshooting page has a detailed description of What’s happening, Why it’s happening and How to fix it. The Why it’s happening section gives more information about the reported issue or misconfiguration. By following the steps of How to fix it, you will be able to resolve the problem or misconfiguration of your Ingress components and make your Ingress Status healthy again.

Once you have executed the troubleshooting steps, you might need to wait 10-15 minutes for the Ingress Status to be updated and to reflect the current state of your cluster.

The troubleshooting guides are also available from the Checking the status of Ingress components documentation page.

Report configuration

The Ingress Status report gives a detailed list of all Ingress-related components and their health state.  Although it is always a good idea to fix the reported errors for the best experience with our managed solution, there can be scenarios where you would like to ignore specific warnings. The warnings might not be relevant to your business needs, so we made it possible to configure the Ingress Status.

Ignoring Ingress Status errors

To configure your status report and choose which warnings you want to ignore for a cluster, use the ibmcloud ks ingress status-report ignored-errors CLI commands. You can find all warning codes in our documentation:

➜  ~ ibmcloud ks ingress status-report ignored-errors --help
NAME:
        ibmcloud ks ingress status-report ignored-errors - View and configure ignored warnings for a cluster.
USAGE:
        ibmcloud ks ingress status-report ignored-errors command [arguments...] [command options]

COMMANDS:
    add   Add warnings to be ignored by Ingress status for a cluster.
    ls    List warnings that are currently ignored by Ingress status for a cluster.
    rm    Remove warnings that are currently ignored by Ingress status for a cluster.

To see the list of ignored errors, run ibmcloud ks ingress status-report ignored errors ls. To add errors to the list to be ignored, run ibmcloud ks ingress status-report ignored-errors add. To remove an error from list so that it is no longer ignored, run ibmcloud ks ingress status-report ignored errors rm. You can specify more errors by using the --code flag multiple times with different codes.

Ignoring specific Ingress-related errors can be useful if Ingress Status is reporting issues on components that you don’t use. Ignored warnings don’t affect the cluster’s overall Ingress Status. Cluster Ingress Status state shows as healthy if all the reported errors are configured to be ignored.

The in-cluster HTTPS health checking

Now you can harden your cluster even further. With new in-cluster HTTP health checks, you no longer need to add IBM Cloud control plane IP addresses to your allowlists. The HTTPS health checks will be performed by components that run on your cluster. For clusters running on VPC infrastructure, might need to adjust the VPC security group configuration to allow incoming requests to the VPC load balancer from the VPC Public Gateway.

In the case of Red Hat OpenShift clusters, this mechanism is already present and is called canary checks. This component will periodically send HTTPS requests to the Ingress subdomain of your router.

In the case of Kubernetes clusters, we developed a new component for health checking. Similarly to canary checks, this component will periodically send HTTPS requests to the public address or addresses of your ALBs. It will be deployed to your cluster if you are using the IBM-managed ALB solution. This feature runs in your cluster by default, but you can configure it to opt-out if you wish. You can find the in-cluster health checker commands under the ibmcloud ks ingress alb health-checker CLI command.

Disabling the report

If you are not interested in viewing your cluster’s Ingress Status, you can disable it with the ibmcloud ks ingress status-report disable command. If you later want to re-enable Ingress Status reporting, you can use the ibmcloud ks ingress status-report enable command.

By disabling the Ingress Status, you no longer receive detailed health state reports of your Ingress-related components. Disabling Ingress Status reporting can be useful in case you don’t use any of the IBM-managed Ingress components. such as cluster subdomains, default routers, ALBs or managed secrets.

More information

For more information, check out our official documentation.

Learn more about IBM Cloud Kubernetes Service and Red Hat OpenShift on IBM Cloud.

Contact us

If you have questions, engage our team via Slack by registering here and join the discussion in the #general channel on our public IBM Cloud Kubernetes Service Slack.

Kudos

We (the blog authors) would like to appreciate our coworkers who helped develop the enhanced Ingress Status.

• Attila Fábián
• Jared Hayes
• Lucas Copi
• Marcell Pünkösd
• Sándor Szombat