Knative Monitoring with Sysdig on IBM Cloud

By: Vidyasagar Machupalli

IBM Cloud Monitoring with Sysdig

After learning on how to deploy an app to Knative on IBM Cloud and how to do Knative log analysis using LogDNA, it’s time to monitor our Knative Kubernetes cluster using the IBM Cloud Monitoring with Sysdigservice.

IBM Cloud Monitoring with Sysdig is a third-party cloud-native and container-intelligence management system that you can include as part of your IBM Cloud architecture.

You can use IBM Cloud Monitoring with Sysdig to gain operational visibility into the performance and health of your applications, services, and platforms. It offers administrators, DevOps teams, and developers full stack telemetry with advanced features to monitor and troubleshoot, define alerts, and design custom dashboards. IBM Cloud Monitoring with Sysdig is operated by Sysdig in partnership with IBM.

Provision a Sysdig instance

Before provisioning the service, you must manage user access.

With your Knative node app running on IBM Cloud, let’s provision a Sysdig instance to start monitoring the app:

  • Navigate to the observability dashboard and under Overview, click Create monitoring instance.

  • Give a service name, choose a region/location (us-south), select a resource group, and choose an appropriate plan based on your requirement.

  • Click Create to provision the instance.

Configure the Sysdig agent

After you provision an instance, you must configure a Sysdig agent for each metric source that you want to monitor. A metric source is a cloud resource that you want to monitor and control its performance and health. In our case, the metric source is a Kubernetes cluster.

Follow these steps to configure

Follow these steps to configure:

  • Under Monitoring on the observability dashboard, click on Edit sources next to the Sysdig instance you just created in the above step.

  • Click Kubernetes on the left pane, and as you have already set the KUBECONFIG, you can skip to the “Install Sysdig agent to your cluster” step.

  • Run the command which looks something similar to this (with your sysdig access key):

curl -sL https://raw.githubusercontent.com/draios/sysdig-cloud-scripts/master/agent_deploy/IBMCloud-Kubernetes-Service/install-agent-k8s.sh 
| bash -s -- -a <ACCESS_KEY> -c ingest.us-south.monitoring.cloud.ibm.com -ac 'sysdig_capture_enabled: false'

Alternatively, you may deploy the Sysdig agent manually by following the instructions here.

the instructions

To confirm the successful creation of Sysdig agent on every worker node in the cluster, run the following command:

kubectl get pods -n ibm-observe
command

Monitor your cluster

Let’s monitor our Kubernetes cluster running your app on Knative:

  • Click View Sysdig. You should see the sysdig monitor wizard.

  • On the welcome page, click Next.

  • Choose Kubernetes as your installation method under set up environment.

  • Once your nodes are connected, you are ready to go. Your cluster will be scanned for infrastructure and integrations.

Monitor your cluster

You can monitor your cluster in the Explore view that is available through the Web UI. This view is the starting point to troubleshoot and monitor your infrastructure. It is the default homepage of the Web UI for users.

On the Explore view, you can see all the namespaces in your cluster and check various dashboards and metrics with a bundle of information, including the health of your nodes, deployments, pods, services. etc.:

Explore

Explore

We need to generate some traffic/load, and there are many open source libraries/tools available. In the post, we will be using Vegeta, an HTTP load-testing tool and library. You can also use hey.

Before generating the load, let’s quickly check whether the IP_ADDRESS and HOST_URL are set properly while deploying the app:

echo $IP_ADDRESS
echo $HOST_URL

After installing Vegeta, run the following command:

 echo “GET http://${IP_ADDRESS}" | vegeta attack -duration=300s -header “Host: ${HOST_URL}” | tee results.bin | vegeta report
 vegeta report -type=json results.bin > metrics.json
 cat results.bin | vegeta plot > plot.html
 cat results.bin | vegeta report -type=”hist[0,100ms,200ms,300ms]”

You can open the plot.html file in the folder you have run the above command to see the Vegeta plot:

Vegeta plot:

Once you start sending a load of HTTP requests, it’s time to check the dashboards.

Use the dashboards to monitor your infrastructure, applications, and services. You can use pre-defined dashboards. You can also create custom dashboards through the Web UI or programmatically. You can backup and restore dashboards by using Python scripts.

A dashboard shows groups of metrics that report on the health, performance, and state of your infrastructure, applications, and services for a single host or a group of hosts. Dashboards offer specialized insight into network data, application data, topology, services, hosts, and containers:

  • Click Dashboards on the left pane and select HTTP overview under My Shared Dashboards. You should see the Request Count and other metrics:

Dashboards

  • Feel free to change the way you want to see metrics:

metrics

  • Here’s an overview by Container:

overview by Container

What we have seen is just the tip of the iceberg. Click here to dig deeper into dashboards.

You can also learn about alerts. For more information, see Working with alerts.

Autoscaling in action

Let’s check the autoscaling feature of Knative in a Sysdig dashboard real-time. Before making a Vegeta attack, this is the deployment state of Kubernetes default namespace. If you observe closely, there are no knative-node-app pods: No. of Pods = 3 logdna pods.

Don’t forget to set on the refresh time to 10 S on the LIVE panel in the bottom of the page:

Autoscaling in action

Once we make a Vegeta attack for 60 seconds, this is what you will see—a new knative-node-app-00001-deployment scaled to 3 pods to take the concurrent requests. After the load goes down, it will return to this:

knative-node-app-00001-deployment

Questions or concerns? Please don’t hesitate to reach out to me on Twitter: VidyasagarMSC

Learn more

Be the first to hear about news, product updates, and innovation from IBM Cloud