Checking Kubernetes services status

The Kubernetes infrastructure is used to run the PowerAI Vision application. The kubectl command can be used to check the status of these underlying services, using the --namespace kube-system option.

Using kubectl get pods to check kube-system

The kubectl command is used to show the detailed status of the Kubernetes pods deployed to run the PowerAI Vision application.

Example output
# kubectl get pods --namespace kube-system
NAME                                    READY     STATUS    RESTARTS   AGE
default-http-backend-77c86f88b4-bm9nq   1/1       Running   0          4d
kube-dns-c5b9d46b-fffn7                 3/3       Running   0          4d
nginx-ingress-lb-ppc64le-djkt6          1/1       Running   0          4d
tiller-deploy-5f954f4845-9sr64          1/1       Running   0          4d
Interpreting the output
  • When the Kubernetes system is running correctly, each of the pods should have:
    • In the READY column all pods should be counted - for example, "1/1" or "3/3".
    • A value of "Running" in the STATUS column.
  • A STATUS value other than "Running" indicates an issue with the Kubernetes infrastructure.
  • A non-0, and growing, value in the RESTARTS column indicates an issue with that Kubernetes pod.

Using kubectl describe pods to check kube-system

The kubectl describe pods command provides detailed information about each of the pods that provide Kubernetes infrastructure. If the output from a specific pod is desired, run the command kubectl describe pod pod_name --namespace kube-system.

Example output

The output from the command is verbose, so sample output from only one pod is shown:

# kubectl describe pods --namespace kube-system
Name:           tiller-deploy-5f954f4845-9sr64
Namespace:      kube-system
Start Time:     Mon, 17 Sep 2018 12:26:33 -0500
Labels:         app=helm
Status:         Running
Created By:     ReplicaSet/tiller-deploy-5f954f4845
Controlled By:  ReplicaSet/tiller-deploy-5f954f4845
    Container ID:   docker://f049f19b4180d0406c04fa7d5ca8993ac1ef596a29d8d0096a54eb504182dd0b
    Image:          ibmcom/tiller-ppc64le:v2.6.0
    Image ID:       docker-pullable://
    Port:           44134/TCP
    State:          Running
      Started:      Mon, 17 Sep 2018 12:26:38 -0500
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:44135/liveness delay=1s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:44135/readiness delay=1s timeout=1s period=10s #success=1 #failure=3
      TILLER_NAMESPACE:  kube-system
      /var/run/secrets/ from default-token-xs2b4 (ro)
  Type           Status
  Initialized    True 
  Ready          True 
  PodScheduled   True 
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-xs2b4
    Optional:    false
QoS Class:       BestEffort
Tolerations: for 300s
        for 300s
Events:          <none>

Interpreting the output

Significant fields providing status of the Kubernetes pods include:

  • The Status field should be "Running" - any other status will indicate issues with the environment.
  • In the Conditions section, the Ready field should indicate "True". Any other value indicates that there are issues with the environment.
  • If there are issues with any pods, the Events section of the pod should have information about issues the pod encountered.