Enabling the NGINX monitoring interface

If you want to use the NGINX monitoring by deploying the Unified Agent, you must confirm that the NGINX monitoring interface is enabled.

Procedure

  • Run the following command on the machine where you want to deploy the Unified Agent:
    http://pod_ip_or_nginx_host:18080/nginx_status
    where pod_ip_or_nginx_host is the fully qualified host name of the NGINX server.

    If no status is returned, the NGINX monitoring interface is not enabled.

    The NGINX monitoring interface requires loading the ngx_http_stub_status_module module. This module helps in collecting basic performance metrics. IBM® Cloud Private provides the NGINX Docker image ibmcom/nginx-ingress-controller, which has ngx_http_stub_status module enabled. If the workloads are using this image, there is no need to do any further configuration. You need only to get the Kubernetes SERVICE or POD IP and verify that the management interface is enabled. For example, http://NGINX_service_or_node_or_pod_ip:18080/nginx_status. Some command examples are shown below to help you determine if NGINX workloads are running and to confirm that their management interfaces are enabled.
    Note: Log into IBM Cloud Private to continue with the following section:
    > cloudctl login -a <cluster> -u <username> 
    > kubectl get po -n kube-system -o wide |grep nginx
    
    The command returns output similar to the following
    
    nginx-ingress-controller-jx8vb      1/1     Running   0     5h   
    10.1.253.201   9.42.75.39
    
    Verify the NGINX management interface status
    
    > curl http://10.1.253.201:18080/nginx_status
    
    Active connections: 9
    server accepts handled requests
     5372 5372 22532
    Reading: 0 Writing: 2 Waiting: 7
    Because the pod IP address can change, you can optionally create a service that points to the POD to get a static IP address. The following commands help create a service configuration:
    > kubectl describe po nginx-ingress-controller-jx8vb -n kube-system 
    
    Name:              nginx-ingress-controller-jx8vb 
    Namespace:         kube-system 
    Priority:          0 
    PriorityClassName: <none> 
    Node:              9.42.75.39/9.42.75.39 
    Start Time:        Fri, 28 Sep 2018 23:46:39 -0400 
    Labels:            app=nginx-ingress-controller
    Note: Here the selector is app=nginx-ingress-controller. It may differ in your IBM Cloud Private environment
    Create a service resource file (nginx-status.yaml) as shown here:
    {
      "apiVersion": "v1",
      "kind": "Service",
      "metadata": {
        "name": "nginx-status",
        "namespace": "kube-system",
        "labels": {
          "app": "nginx-status"
        }
      },
      "spec": {
        "ports": [
          {
            "name": "nginx-status",
            "protocol": "TCP",
            "port": 18080,
            "targetPort": 18080
          }
        ],
        "selector": {
          "app": "nginx-ingress-controller"
        },
        "type": "ClusterIP",
        "sessionAffinity": "None"
      }
    }
    Note: Open port 18080 for NGINX status access.
    Create the Kubernetes service resource using the file that you created above and obtain the service ip
    > kubectl create -f nginx-status.yaml
    > kubectl describe svc nginx-status -n kube-system
    
    Name:              nginx-status
    Namespace:         kube-system
    Labels:            app=nginx-status
    Annotations:       <none>
    Selector:          name=nginx-ingress-controller
    Type:              ClusterIP
    IP:                10.0.0.243
    Port:              nginx-status  18080/TCP
    TargetPort:        18080/TCP
    Endpoints:         9.37.22.210:18080
    Session Affinity:  None
    Events:            <none>
     curl http://10.0.0.243:18080/nginx_status 

    You can give the NGINX management interface URL http://10.0.0.243:18080/nginx_status to the Helm Chart configuration. In on-premises installations, this module is not enabled by default. It must first be built and then enabled with the configuration parameter --with-http_stub_status_module. Please see the NGINX documentation for enablement.

  • If you want to monitor NGINX in IBM Cloud Private 3.2.0 or 3.2.1, you need to do extra steps to ensure NGINX monitoring can run successfully.
    1. Find the NGINX pod on IBM Cloud Private.
      # kubectl get po -n kube-system|grep nginx
      nginx-ingress-controller-ph8t6                                 1/1     Running     0          18m
    2. Export nginx.tmpl.
      # kubectl cp kube-system/nginx-ingress-controller-ph8t6:template/nginx.tmpl nginx.tmpl
    3. Remove the line deny all in nginx.tmpl.
      location /nginx_status {
                  {{ if $all.Cfg.EnableOpentracing }}
                  opentracing off;
                  {{ end }}
      
                  {{ range $v := $all.NginxStatusIpv4Whitelist }}
                  allow {{ $v }};
                  {{ end }}
                  {{ if $all.IsIPV6Enabled -}}
                  {{ range $v := $all.NginxStatusIpv6Whitelist }}
                  allow {{ $v }};
                  {{ end }}
                  {{ end -}}
      ###This line should be removed , or comment out
                  deny all;
      ###End
                  access_log off;
                  stub_status on;
              }
    4. Create nginx template configmap.
      kubectl create configmap nginx-template -n kube-system --from-file=nginx.tmpl
    5. Modify nginx daemonset to use this configmap.
      kubectl edit daemonset nginx-ingress-controller -n kube-system
      Snippet to add to here:
      spec:
            containers:
            - args:
              ...
              ...
      ### Here begin the config map setting to copy ###
              volumeMounts:
              - mountPath: /etc/nginx/template
                name: nginx-template-volume
                readOnly: true
            volumes:
            - name: nginx-template-volume
              configMap:
                name: nginx-template
                items:
                - key: nginx.tmpl
                  path: nginx.tmpl
      ### End ###
    6. Do the verification. After finishing the daemonset edit, the Nginx controller pod will be automatically restarted. If not, manually delete it to take effect.
      # kubectl get po -n kube-system -o wide|grep nginx
      nginx-ingress-controller-ph8t6                                 1/1     Running     0          3m    10.1.13.68     9.46.67.224   <none>           <none>
      The nginx status is exposed on port 80 according to the nginx config. So, curl this url for verification:
      #curl http://10.1.13.68/nginx_status
      Active connections: 48
      server accepts handled requests
       729017 729017 896362
      Reading: 0 Writing: 14 Waiting: 33