Kubernetes metrics for thresholds

As soon as you deploy the Kubernetes data collector, incidents are generated for any native Kubernetes events. You can also define your own Kubernetes thresholds that, when breached, open events and generate incidents.

This topic lists the metrics for each Kubernetes resource type that you can use in a threshold definition and provides usage examples.

Kubernetes Cluster

These are the metrics that are available for use in Kubernetes Cluster thresholds:
  • Cluster Name
  • CPU: Allocatable Nanocores, Capacity Nanocores, Usage Core Nanoseconds, Usage Millicores
  • Deployment Availability Percent
  • Ephemeral-Storage: Allocatable Bytes, Capacity Bytes
  • File System: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • Hugepages-2Mi: Allocatable Bytes, Capacity Bytes
  • Memory: Allocatable Bytes, Available Bytes, Capacity Bytes, Major Page Faults, Page Faults, Rss Bytes, Usage Bytes, Usage with Cache Bytes
  • Name
  • Pods: Allocatable, Capacity, Hosted
  • Rlimit: Curproc, Maxpid
  • Runtime Image File System: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • Stateful Set: Availability, Availability Percent
  • Total: Deployments, Deployments Available, Stateful Sets, Stateful Sets Available
  • Type

Kubernetes Container

These metrics are available for use in Kubernetes Container thresholds:
  • Cluster Uid
  • Container Id
  • CPU: Limits Nanocores, Requests Nanocores, Usage Core Nanoseconds
  • InitContainer
  • Logs: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • Memory: Limits Bytes, Major Page Faults, Page Faults, Requests Bytes, Rss Bytes, Usage Bytes, Usage with Cache Bytes
  • Name
  • Namespace
  • Node Id
  • Pod Id
  • Restart Count
  • Rootfs: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • Type

Kubernetes Daemon Set, Kubernetes Deployment, Kubernetes Job, Kubernetes Replication Controller, Kubernetes Replica Set, and Kubernetes Stateful Set

Use these metrics in thresholds for monitoring the Kubernetes daemon set, deployment, job, replication controller, replica set, or stateful set:
  • Annotations
  • Available Replicas
  • Back off Limit
  • Cluster Uid
  • Collision Count
  • Completion Time
  • Completions
  • Concurrency Policy
  • Creation Timestamp
  • Current: Number Scheduled, Replicas, Revision
  • Desired Number Scheduled
  • Failed (available for Kubernetes Job only)
  • Failed Jobs History Limit
  • Fully Labeled Replicas
  • Generation
  • Labels
  • Name
  • Namespace
  • Node Selector (available for Kubernetes Daemon Set only)
  • Number: Available, Misscheduled, Ready, Unavailable
  • Observed Generation
  • Parallelism
  • Pod Management Policy
  • Ready Replicas
  • Replicas
  • Revision History Limit
  • Schedule
  • Service Name
  • Start Time
  • Succeeded
  • Successful Jobs History Limit
  • Suspend
  • Template Generation
  • Update: Revision, Strategy
  • Updated: Number Scheduled, Replicas

Kubernetes Node

These metrics are available for use in thresholds for monitoring Kubernetes nodes:
  • Allocatable
  • Annotations
  • Architecture
  • Boot Id
  • Capacity
  • Cluster Uid
  • Container Runtime Version
  • CPU: Allocatable Nanocores, Capacity Nanocores, Usage Core Nanoseconds, Usage Millicores
  • Creation Timestamp
  • Ephemeral-Storage: Allocatable Bytes, Capacity Bytes
  • External Id
  • File System: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • Hostname
  • Hugepages-2Mi: Allocatable Bytes, Capacity Bytes
  • Internal Ip
  • Kernel Version
  • Kube Proxy Version
  • Kubelet: Port, Version
  • Labels
  • Machine Id
  • Memory: Allocatable Bytes, Available Bytes, Capacity Bytes, Major Page Faults, Page Faults, Rss Bytes, Usage Bytes, Usage with Cache Bytes
  • Name
  • Node Role
  • Operating System
  • Os Image
  • podCIDR
  • Pods: Allocatable, Capacity, Hosted
  • Rlimit: Curproc, Maxpid
  • Runtime Image File System: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • System Uuid
  • Type
  • Unschedulable

Kubernetes Pod

The following metrics are available for use in thresholds for monitoring Kubernetes pods:
  • Annotations
  • Cluster Uid
  • CPU Usage Core Nanoseconds
  • Creation Timestamp
  • Dns Policy
  • Ephemeral-Storage: Available Bytes, Capacity Bytes, Inodes, Inodes Free, Inodes Used, Used Bytes
  • Generate Name
  • Host: Ip, Network, Pid
  • Hostname
  • Image Pull Secrets
  • Labels
  • Memory: Major Page Faults, Page Faults, Rss Bytes, Usage Bytes, Usage with Cache Bytes
  • Name
  • Namespace
  • Network: Received Bytes, Received Errors, Transmitted Bytes, Transmitted Errors
  • Node: Id, Name, Selector
  • Num Containers
  • Phase
  • Pod Ip
  • Qos Class
  • Restart: Count, Policy
  • Scheduler Name
  • Service: Account, Account Name
  • Start Time
  • Subdomain
  • Termination Grace Period Seconds
  • Type

Kubernetes Service

You can use the following metrics to define a Kubernetes Service threshold:
  • Annotations
  • Browser: Load Time (ms), Type, Version
  • Cluster: IP, Uid
  • Content Loading Time (ms)
  • Creation Timestamp
  • Domain Name
  • Error Count per Interval
  • External Traffic Policy
  • Labels
  • Latency (ms)
  • Load Balancer
  • Name
  • Namespace
  • Page Transfer Time (ms)
  • Pod Name
  • Ports
  • Real User Latency (ms)
  • Request: Name, Type
  • Resolve Time (ms)
  • Selector
  • Service Type
  • Session Affinity
  • Status
  • Status Code
  • Transaction: Name, Type
Some metrics cannot be used in a threshold definition with multiple AND conditions:
  • Request Name and Latency (ms) with Cluster Ip, Creation Timestamp, or Error Count per Interval.
  • Labels, Latency (ms), Load Balancer, Name, Namespace, Ports, Request Name