How-tos

Kubernetes Log Forwarding with Syslog

Share this post:

Kubernetes Log Forwarding with Syslog

Logs help you troubleshoot issues with your clusters and apps. Sometimes, you might want to send logs somewhere for processing or long-term storage. On a Kubernetes cluster in the IBM Cloud Container Service, you can enable log forwarding for your cluster and choose where your logs are forwarded.

Using the Container Service CLI, you can forward your container logs to a syslog server with one command:

bx cs logging-config-create mycluster \
    --hostname mysyslog.example.com \
    --type syslog \
    --namespace default

The above command creates a logging configuration to send all container standard output and error logs from the default Kubernetes namespace. These logs are sent using the syslog protocol to mysyslog.example.com.

Try it out

In this tutorial, you will forward your logs to an rsyslog instance running in the same cluster.

Create a Kubernetes cluster on the IBM Cloud Container Service and wait for it to become ready. Next, connect kubectl commands to your cluster with the following command:

eval `bx cs cluster-config mycluster --export`

Next, create an rsyslog service we can forward logs to.

Start by creating deploy-rsyslog.yaml with the following contents:

apiVersion: v1
kind: Service
metadata:
  name: rsyslog-service
spec:
  selector:
    app: rsyslog
  ports:
  - name: tcp-syslog
    port: 514
    targetPort: 514
    protocol: TCP
  - name: udp-syslog
    port: 514
    targetPort: 514
    protocol: UDP
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: rsyslog
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rsyslog
  template:
    metadata:
      name: rsyslog
      labels:
        app: rsyslog
    spec:
      containers:
      - name: rsyslog
        image: voxxit/rsyslog:latest
        imagePullPolicy: "Always"
        ports:
        - name: incoming-logs
          containerPort: 514

Then run the following:

kubectl create -f deploy-rsyslog.yaml

Then configure your logs to go to the rsyslog service.

bx cs logging-config-create mycluster \
    --hostname rsyslog-service.default \
    --type syslog \
    --namespace default

Finally, deploy a container to your cluster that generates logs. I like using a noisy pod to verify that log forwarding is working. Make a deploy-noisy.yaml file with the following contents:

apiVersion: v1
kind: Pod
metadata:
  name: noisy
spec:
  containers:
  - name: noisy
    image: ubuntu:16.04
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 10; echo 'Hello world!'; done"]
    imagePullPolicy: "Always"

Finally, create the noisy pod.

kubectl create -f deploy-noisy.yaml

Now take a look inside the rsyslog instance to see the logs.

export rsyslog_pod_name=`kubectl get pods -l app=rsyslog -o jsonpath='{range .items[*]}{.metadata.name}'`
kubectl exec -it "$rsyslog_pod_name" -- tail -f /var/log/messages

If you see some Hello world! lines, then you have successfully forwarded logs to your rsyslog service.

To learn more, continue reading about log forwarding or IBM’s Kubernetes offering.

Add Comment
No Comments

Leave a Reply

Your email address will not be published.Required fields are marked *

More How-tos stories

IBM Cloud Functions: we’re doubling the time limit on executing actions

If you work with Cloud Functions, you know that building serverless applications mainly happens by creating actions–small pieces of code that each do one thing well. You can of course sequence actions together or create event-driven workflows. Through our work with customers, we have realized that some scenarios (for example, image processing, file processing, data pipelines) require more time […]

Continue reading

Plan, create and update deployment environments with IBM Cloud CLI and Terraform

Developers do not like to write the same thing twice. The DRY principle is one example of this. Similarly they don't like having to go through tons of clicks in a user interface to setup an environment. Consequently shell scripts have been long used by system administrators and developers to automate repetitive, error-prone and uninteresting tasks.

Continue reading

Access Groups

IBM Cloud introduces access groups in April 2018 as the platform’s newest feature that makes organizing users and streamlining the assignment of access easier than ever! Previously, IBM Cloud Identity Access and Management (IAM) required access policies to be set for individual users and service IDs. Now with access groups, you can organize users and service IDs into a group and manage access by assigning policies to the group. This will save you time and effort and make your IBM Cloud security efforts more efficient!

Continue reading