Monitoring vLLM on Kubernetes

Monitor vLLM pods running in Kubernetes clusters with Instana by deploying an OpenTelemetry Collector as a Helm-managed StatefulSet. The collector scrapes Prometheus metrics from vLLM pods, enriches them with Kubernetes metadata, and forwards them to the Instana backend.

Inspect a vLLM pod:

kubectl describe pod <vllm-pod-name> -n <namespace>

Note the following values from the output:


Field to find	Location in pod describe	Example value
Namespace	`Namespace:`	`text-generation`
App label	`Labels: app=`	`vllm-server`
Component label	`Labels: component=`	`huggingface-vllm`
Metrics port	`Port:` under Containers	`8000/TCP`

Before you deploy the collector, verify that vLLM is exposing metrics:
```
kubectl exec -n <namespace> <vllm-pod-name> -- \
  curl http://localhost:<metrics-port>/metrics
```
The output contains Prometheus-formatted metrics such as vllm:*, http_*, and process_*. If the metrics endpoint does not respond, verify that vLLM started correctly and that the port number matches the pod description.
Retrieve the cluster name:
```
kubectl config current-context
```
The collector tags every metric with cluster metadata. Collect the following details about your cluster.
Retrieve the Kubernetes version:
```
kubectl version --short
```
Retrieve the cluster UID, which is derived from the kube-system namespace UID:
```
kubectl get namespace kube-system -o jsonpath='{.metadata.uid}'
```

Create a values.yaml file based on the following template. Replace every placeholder (marked with <…>) with the values you collected in the previous steps.

mode: statefulset
replicaCount: 1
image:
  repository: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s
extraEnvs:
  - name: K8S_NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName
presets:
  kubernetesAttributes:
    enabled: true
    extractAllPodLabels: true
    extractAllPodAnnotations: true
  kubeletMetrics:
    enabled: false
  logsCollection:
    enabled: false
  hostMetrics:
    enabled: false
  kubernetesEvents:
    enabled: true
  clusterMetrics:
    enabled: true
config:
  extensions:
    health_check:
      endpoint: ${env:MY_POD_IP}:13133

  receivers:
    k8s_cluster:
      collection_interval: 10s
    kubeletstats:
      collection_interval: 10s
      auth_type: serviceAccount
      endpoint: "https://${env:K8S_NODE_NAME}:10250"
      insecure_skip_verify: true
      metric_groups:
        - pod
        - node
        - container
    prometheus:
      config:
        scrape_configs:
          - job_name: "prometheus-scrape"
            scrape_interval: 10s
            metrics_path: /metrics
            tls_config:
              insecure_skip_verify: true
            kubernetes_sd_configs:
              - role: pod
                namespaces:
                 names:
                  - <vllm-namespace-name>     # your vLLM namespace, e.g. text-generation
            relabel_configs:
              - source_labels: [__meta_kubernetes_pod_label_component]
                regex: <component-label-value> # your component label, e.g. huggingface-vllm
                action: keep
              - source_labels: [__meta_kubernetes_pod_phase]
                regex: Running
                action: keep
              - source_labels: [__meta_kubernetes_pod_ip]
                regex: (.+)
                target_label: __address__
                replacement: $1:<vllm-metrics-port>  # your metrics port, e.g. 8000
              - source_labels: [__meta_kubernetes_pod_name]
                target_label: k8s_pod
              - source_labels: [__meta_kubernetes_namespace]
                target_label: k8s_namespace

  processors:
    resource/vllm:
      attributes:
        - key: service.name
          value: vllm
          action: upsert
        - key: service.namespace
          from_attribute: k8s.namespace.name
          action: upsert
        - key: service.instance.id
          from_attribute: k8s.pod.uid
          action: insert
        - key: INSTANA_PLUGIN
          value: vllm
          action: upsert
        - key: vllm.entity.type
          value: vllm
          action: upsert
    resource/k8s:
      attributes:
        - key: k8s.cluster.name
          value: <cluster-name>            # e.g. prod-cluster-01
          action: insert
        - key: k8s.cluster.distribution
          value: <cluster-distribution>    # openshift | eks | gke | aks | kubernetes
          action: insert
        - key: k8s.cluster.managedBy
          value: <cluster-managed-by>      # e.g. CNF, EKS, GKE, AKS
          action: insert
        - key: k8s.cluster.shortName
          value: <cluster-short-name>
          action: insert
        - key: k8s.cluster.fullName
          value: <cluster-full-name>
          action: insert
        - key: k8s.cluster.version
          value: <cluster-version>         # e.g. v1.29.0
          action: insert
        - key: k8s.cluster.uid
          value: <cluster-uid>             # kube-system namespace UID
          action: insert
        - key: INSTANA_PLUGIN
          value: k8s
          action: upsert
    memory_limiter:
      check_interval: 1s
      limit_percentage: 50
      spike_limit_percentage: 10
    k8sattributes:
      auth_type: serviceAccount
      passthrough: false
      extract:
        metadata:
          - k8s.namespace.name
          - k8s.pod.name
          - k8s.pod.uid
          - k8s.deployment.name
          - k8s.node.name
        labels:
          - tag_name: k8s.pod.labels.app
            key: app
            from: pod
          - tag_name: k8s.pod.labels.component
            key: component
            from: pod
    resourcedetection/env:
      detectors:
        - env
    filter/cluster:
      metrics:
        metric:
          - 'resource.attributes["k8s.cluster.uid"] == nil'
    filter/vllm:
      metrics:
        include:
          match_type: regexp
          resource_attributes:
            - key: k8s.pod.labels.component
              value: <component-label-value>  # same value as the Prometheus relabel_configs regex
    batch: {}

  exporters:
    otlphttp:
      endpoint: <otlp-endpoint>  # e.g. https://otlp-grpc-<tenant>.instana.io:4318
      tls:
        insecure: false
      headers:
        x-instana-key: <your-instana-agent-key>
    debug:
      verbosity: detailed

  service:
    extensions:
      - health_check
    pipelines:
      metrics:
        receivers:
          - prometheus
        processors:
          - k8sattributes
          - filter/vllm
          - resource/k8s
          - resource/vllm
          - batch
        exporters:
          - debug
          - otlphttp
      metrics/k8s:
        receivers:
          - k8s_cluster
          - kubeletstats
        processors:
          - k8sattributes
          - resource/k8s
          - memory_limiter
          - batch
          - filter/cluster
        exporters:
          - debug
          - otlphttp

clusterRole:
  rules:
    - apiGroups: ['']
      resources:
        - pods
        - namespaces
        - nodes
        - nodes/stats
        - nodes/proxy
        - nodes/metrics
        - endpoints
        - services
        - events
        - resourcequotas
        - replicationcontrollers
      verbs: ['get', 'list', 'watch']

Review the configuration reference to confirm that you have all required values. For the full list of placeholders and how to obtain each value, see vLLM Kubernetes configuration reference.
Validate that your values.yaml file renders correctly:
```
helm template my-otel-collector open-telemetry/opentelemetry-collector \
  -f values.yaml \
  --validate
```
Fix any reported YAML syntax errors before you continue.

Install the collector:

helm install my-otel-collector open-telemetry/opentelemetry-collector \
  -f values.yaml \
  -n monitoring \
  --create-namespace

Verify that the collector pod is running:

kubectl get pods -n monitoring -l app.kubernetes.io/name=opentelemetry-collector

Inspect the collector logs to confirm that the collector is scraping and exporting vLLM metrics:
```
kubectl logs -n monitoring -l app.kubernetes.io/name=opentelemetry-collector --tail=100
```
Look for log entries that contain Scrape succeeded or Metrics exported successfully. If you see errors, see Troubleshooting vLLM monitoring on Kubernetes.

The OpenTelemetry Collector is deployed and scraping Prometheus metrics from your vLLM pods. Metrics are now being forwarded to the Instana backend.

After the collector is running and exporting data, view vLLM metrics in Instana:

In the Instana UI, go to Infrastructure > Analyze Infrastructure.
Select OTel vLLMonitor from the entity types list.
Click an OTel vLLMonitor instance to open its dashboard.

The dashboard shows token throughput, running and waiting requests, and other vLLM metrics collected from your pods.

Go to the Kubernetes pod to view resource utilization. The pod summary shows CPU usage, memory, and filesystem metrics.

Use Upstream/downstream on the pod to inspect the upstream services that call the vLLM pod and the applications that they belong to.