How-tos

Kubernetes Log Forwarding with Syslog

Share this post:

Kubernetes Log Forwarding with Syslog

Logs help you troubleshoot issues with your clusters and apps. Sometimes, you might want to send logs somewhere for processing or long-term storage. On a Kubernetes cluster in the IBM Cloud Container Service, you can enable log forwarding for your cluster and choose where your logs are forwarded.

Using the Container Service CLI, you can forward your container logs to a syslog server with one command:

bx cs logging-config-create mycluster \
    --hostname mysyslog.example.com \
    --type syslog \
    --namespace default

 

The above command creates a logging configuration to send all container standard output and error logs from the default Kubernetes namespace. These logs are sent using the syslog protocol to mysyslog.example.com.

Try it out

In this tutorial, you will forward your logs to an rsyslog instance running in the same cluster.

Create a Kubernetes cluster on the IBM Cloud Container Service and wait for it to become ready. Next, connect kubectl commands to your cluster with the following command:

eval `bx cs cluster-config mycluster --export`

 

Next, create an rsyslog service we can forward logs to.

Start by creating deploy-rsyslog.yaml with the following contents:

apiVersion: v1
kind: Service
metadata:
  name: rsyslog-service
spec:
  selector:
    app: rsyslog
  ports:
  - name: tcp-syslog
    port: 514
    targetPort: 514
    protocol: TCP
  - name: udp-syslog
    port: 514
    targetPort: 514
    protocol: UDP
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: rsyslog
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rsyslog
  template:
    metadata:
      name: rsyslog
      labels:
        app: rsyslog
    spec:
      containers:
      - name: rsyslog
        image: voxxit/rsyslog:latest
        imagePullPolicy: "Always"
        ports:
        - name: incoming-logs
          containerPort: 514

 

Then run the following:

kubectl create -f deploy-rsyslog.yaml

 

Then configure your logs to go to the rsyslog service.

bx cs logging-config-create mycluster \
    --hostname rsyslog-service.default \
    --type syslog \
    --namespace default

 

Finally, deploy a container to your cluster that generates logs. I like using a noisy pod to verify that log forwarding is working. Make a deploy-noisy.yaml file with the following contents:

apiVersion: v1
kind: Pod
metadata:
  name: noisy
spec:
  containers:
  - name: noisy
    image: ubuntu:16.04
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 10; echo 'Hello world!'; done"]
    imagePullPolicy: "Always"

 

Finally, create the noisy pod.

kubectl create -f deploy-noisy.yaml

 

Now take a look inside the rsyslog instance to see the logs.

export rsyslog_pod_name=`kubectl get pods -l app=rsyslog -o jsonpath='{range .items[*]}{.metadata.name}'`
kubectl exec -it "$rsyslog_pod_name" -- tail -f /var/log/messages

 

If you see some Hello world! lines, then you have successfully forwarded logs to your rsyslog service.

To learn more, continue reading about log forwarding or IBM’s Kubernetes offering.

More How-tos stories
May 3, 2019

Kubernetes Tutorials: 5 Ways to Get You Building Fast

Ready to start working with Kubernetes? Want to build your Kubernetes skills? The five tutorials in this post will teach you everything you need to know about how to manage your containerized apps with Kubernetes.

Continue reading

May 3, 2019

Using Portworx to Deploy and Manage an HA MySQL Cluster on IBM Cloud Kubernetes Service

This tutorial is a walkthrough of the steps involved in deploying and managing a highly available MySQL cluster on IBM Cloud Kubernetes Service.

Continue reading

May 2, 2019

Kubernetes v1.14.1 Now Available in IBM Cloud Kubernetes Service

We are excited to announce the availability of Kubernetes v1.14.1 for your clusters that are running in IBM Cloud Kubernetes Service. IBM Cloud Kubernetes Service continues to be the first public managed Kubernetes service to support the latest upstream versions from the community.

Continue reading