How-tos

Kubernetes Log Forwarding with Syslog

Share this post:

Kubernetes Log Forwarding with Syslog

Logs help you troubleshoot issues with your clusters and apps. Sometimes, you might want to send logs somewhere for processing or long-term storage. On a Kubernetes cluster in the IBM Cloud Container Service, you can enable log forwarding for your cluster and choose where your logs are forwarded.

Using the Container Service CLI, you can forward your container logs to a syslog server with one command:

bx cs logging-config-create mycluster \
    --hostname mysyslog.example.com \
    --type syslog \
    --namespace default

 

The above command creates a logging configuration to send all container standard output and error logs from the default Kubernetes namespace. These logs are sent using the syslog protocol to mysyslog.example.com.

Try it out

In this tutorial, you will forward your logs to an rsyslog instance running in the same cluster.

Create a Kubernetes cluster on the IBM Cloud Container Service and wait for it to become ready. Next, connect kubectl commands to your cluster with the following command:

eval `bx cs cluster-config mycluster --export`

 

Next, create an rsyslog service we can forward logs to.

Start by creating deploy-rsyslog.yaml with the following contents:

apiVersion: v1
kind: Service
metadata:
  name: rsyslog-service
spec:
  selector:
    app: rsyslog
  ports:
  - name: tcp-syslog
    port: 514
    targetPort: 514
    protocol: TCP
  - name: udp-syslog
    port: 514
    targetPort: 514
    protocol: UDP
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: rsyslog
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rsyslog
  template:
    metadata:
      name: rsyslog
      labels:
        app: rsyslog
    spec:
      containers:
      - name: rsyslog
        image: voxxit/rsyslog:latest
        imagePullPolicy: "Always"
        ports:
        - name: incoming-logs
          containerPort: 514

 

Then run the following:

kubectl create -f deploy-rsyslog.yaml

 

Then configure your logs to go to the rsyslog service.

bx cs logging-config-create mycluster \
    --hostname rsyslog-service.default \
    --type syslog \
    --namespace default

 

Finally, deploy a container to your cluster that generates logs. I like using a noisy pod to verify that log forwarding is working. Make a deploy-noisy.yaml file with the following contents:

apiVersion: v1
kind: Pod
metadata:
  name: noisy
spec:
  containers:
  - name: noisy
    image: ubuntu:16.04
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 10; echo 'Hello world!'; done"]
    imagePullPolicy: "Always"

 

Finally, create the noisy pod.

kubectl create -f deploy-noisy.yaml

 

Now take a look inside the rsyslog instance to see the logs.

export rsyslog_pod_name=`kubectl get pods -l app=rsyslog -o jsonpath='{range .items[*]}{.metadata.name}'`
kubectl exec -it "$rsyslog_pod_name" -- tail -f /var/log/messages

 

If you see some Hello world! lines, then you have successfully forwarded logs to your rsyslog service.

To learn more, continue reading about log forwarding or IBM’s Kubernetes offering.

More How-tos stories
October 15, 2018

Add Custom Domain and TLS Certificate to Your Secure Cloud App

Secure your cloud app end to end. Use a custom domain with TLS certificate for apps deployed on IBM Cloud Kubernetes Service.

Continue reading

October 12, 2018

Cloud Functions Package Design and Best Practices

Whether they're JARs, libs, Gems, or modules, successful programming languages provide an ability to share and re-use code. IBM Cloud Functions is no different—it enables developer productivity through installable packages. But what makes a good package?

Continue reading

October 5, 2018

Deploying a Microservices-Based Solution to Cloud Foundry Enterprise Environment (CFEE)

With Cloud Foundry Enterprise Environment (CFEE), you can instantiate multiple, isolated, enterprise-grade Cloud Foundry platforms on demand. Instances of the CFEE service run within your own account in IBM Cloud.

Continue reading