February 17, 2022 By Vidyasagar Machupalli 3 min read

Learn how to set up log forwarding and collect audit logs that are passed through the Kubernetes API server to IBM Log Analysis to check who initiated a request and when they did so.

As a cluster administrator, by following the simple steps in this blog post, you should be able to answer questions about Kubernetes audit logs, like who initiated a request to delete a Kubernetes resource? When did it happen? On what did it happen?

What are audit logs?

Audit logs allow you to better understand the operations that are initiated by users in your cluster, which can help you troubleshoot issues or report compliance to industry and internal standards.

Although the Kubernetes API server for your cluster is enabled for auditing by default, no auditing data is available until you set up log forwarding. You can forward audit logs for the IBM Cloud Kubernetes Service, the Kubernetes API server and the worker nodes to a logging instance on IBM Cloud.

Prerequisites

Query and decode the logs

You can always enable and launch logging from your Kubernetes cluster’s overview page. By now, you should see the audit logs on the IBM Log Analysis view:

  1. To test, set the context for your cluster without the --admin flag: 
    ibmcloud ks cluster config --cluster <CLUSTER ID or NAME>

    Note: Using the --admin flag will show the cluster-admin context and may not reveal the IAM user overriding the RBAC.

  2. Create a namespace with the following command: 
    kubectl create namespace test123
  3. In the Log Analysis UI, enter the following query: 
    verb:create objectRef.name:test123 objectRef.resource:namespaces

    So, by now you know which IAM user (who) created the namespace (what) and when was it created. Note: The objectRef.resources is optional and can be any Kubernetes resource (e.g., secrets, configmaps, services, etc.).

  4. Similarly, you can delete the namespace with the command kubectl delete namespace test123. Then, from the audit logs, you can quickly find out who (IAM user) deleted the namespace and at what time. The query to decode who deleted the namespace will be as follows: 
    verb:delete objectRef.name:test123 objectRef.resource:namespaces
  5. To create a custom view out of the query:
    • Click Unsaved View > Save as new view.
    • Enter a Name for your view. 
    • Optionally, you can select a Category and Alert value.
    • Click Save View. Your view is listed under your selected category. If you didn’t select a category, it will be listed under UNCATEGORIZED.
  6. To add an alert to your custom view, check the add alert to custom view section of IBM Cloud Kubernetes documentation.

Conclusion

Following the steps in the post, you learned what audit logs are, what the audit logs capture and how to forward and collect the audit logs in IBM Log Analysis to query and decode the logs to understand the operations that are initiated by users in your cluster. 

You can always control user access with IBM Cloud IAM and Kubernetes RBAC. To understand more about Kubernetes auditing and the audit policy, refer to the Kubernetes documentation.

If you have any queries, feel free to reach out to me on Twitter or on LinkedIn

Was this article helpful?
YesNo

More from Cloud

A clear path to value: Overcome challenges on your FinOps journey 

3 min read - In recent years, cloud adoption services have accelerated, with companies increasingly moving from traditional on-premises hosting to public cloud solutions. However, the rise of hybrid and multi-cloud patterns has led to challenges in optimizing value and controlling cloud expenditure, resulting in a shift from capital to operational expenses.   According to a Gartner report, cloud operational expenses are expected to surpass traditional IT spending, reflecting the ongoing transformation in expenditure patterns by 2025. FinOps is an evolving cloud financial management discipline…

IBM Power8 end of service: What are my options?

3 min read - IBM Power8® generation of IBM Power Systems was introduced ten years ago and it is now time to retire that generation. The end-of-service (EoS) support for the entire IBM Power8 server line is scheduled for this year, commencing in March 2024 and concluding in October 2024. EoS dates vary by model: 31 March 2024: maintenance expires for Power Systems S812LC, S822, S822L, 822LC, 824 and 824L. 31 May 2024: maintenance expires for Power Systems S812L, S814 and 822LC. 31 October…

24 IBM offerings winning TrustRadius 2024 Top Rated Awards

2 min read - TrustRadius is a buyer intelligence platform for business technology. Comprehensive product information, in-depth customer insights and peer conversations enable buyers to make confident decisions. “Earning a Top Rated Award means the vendor has excellent customer satisfaction and proven credibility. It’s based entirely on reviews and customer sentiment,” said Becky Susko, TrustRadius, Marketing Program Manager of Awards. Top Rated Awards have to be earned: Gain 10+ new reviews in the past 12 months Earn a trScore of 7.5 or higher from…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters