Community

Transitioning Your Service Mesh From IBM Cloud Kubernetes Service Ingress to Istio Ingress

Share this post:

Migrating a service mesh from Kubernetes Ingress resources to Istio’s ingress gateway

Through a tremendous collaborative effort between IBM, Google, Lyft, Red Hat, and other members of the open source community, Istio is officially ready for production. Istio adds additional layers of service mesh management on top of those available in Kubernetes and allows developers to connect, secure, and manage microservices.

This tutorial will provide steps for migrating a service mesh from Kubernetes Ingress resources to Istio’s ingress gateway in an IBM Cloud Kubernetes Service environment.

Prerequisites

Using IBM Kubernetes Service ALB and Kubernetes Ingress resources to access a service

IBM Cloud Kubernetes Service exposes applications deployed within your cluster using Kubernetes Ingress resources. The application load balancer (ALB) accepts incoming traffic and forwards the requests to the corresponding service. Your cluster ALB is assigned a URL to expose the deployed services using the name format <cluster_name>.<region>.containers.appdomain.cloud. Retrieve your cluster’s ingress subdomain and configure this value as an environment variable for the steps that follow:

ibmcloud cs cluster-get <cluster_name>
export INGRESS_HOST=<ingress_subdomain>

For this tutorial, we will use a simple HTTP request and response service called httpbin. Begin by deploying a copy of this service to your space in IBM Cloud Kubernetes Service:

kubectl apply -f samples/httpbin/httpbin.yaml

Next, apply a Kubernetes Ingress resource to access the httpbin service and expose the /headers API:

cat <<EOF | kubectl apply -f -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: simple-ingress
  namespace: default
spec:
  rules:
    - host: $INGRESS_HOST
      http:
        paths:
          - path: /headers
            backend:
              serviceName: httpbin
              servicePort: 8000
EOF

The serviceName and servicePort match those specified in the Kubernetes service definition in samples/httpbin/httpbin.yaml. Verify that the httpbin service is now accessible using curl:

curl -i http://$INGRESS_HOST/headers
HTTP/1.1 200 OK
Date: Mon, 13 Aug 2018 21:43:04 GMT
Content-Type: application/json
Content-Length: 304
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
{
  "headers": {
    "Accept": "*/*", 
    "Host": "istio-integration.us-south.containers.appdomain.cloud", 
    "User-Agent": "curl/7.54.0", 
    "X-Forwarded-Host": "istio-integration.us-south.containers.appdomain.cloud", 
    "X-Global-K8Fdic-Transaction-Id": "ba234fb0a36e57a20ee68e9da27ae6fd"
  }
}

Chain ALB to Istio ingress gateway

Install Istio using Helm

First, install Istio’s custom resource definitions:

kubectl apply -f https://raw.githubusercontent.com/IBM/charts/master/stable/ibm-istio/templates/crds.yaml

Configure Tiller in your cluster:

kubectl apply -f install/kubernetes/helm/helm-service-account.yaml
helm init --service-account tiller

Next, install the Helm chart to your cluster:

helm repo add ibm-stable https://registry.bluemix.net/helm/ibm
helm install ibm-stable/ibm-istio --name=istio --namespace istio-system

Verify Istio is up and running:

kubectl get pods -n istio-system
 NAME                                       READY     STATUS      RESTARTS   AGE
 istio-citadel-748d656b-pj9bw               1/1       Running     0          2m
 istio-egressgateway-6c65d7c98d-l54kg       1/1       Running     0          2m
 istio-galley-65cfbc6fd7-bpnqx              1/1       Running     0          2m
 istio-ingressgateway-f8dd85989-6w6nj       1/1       Running     0          2m
 istio-pilot-5fd885964b-l4df6               2/2       Running     0          2m
 istio-policy-56f4f4cbbd-2z2bk              2/2       Running     0          2m
 istio-sidecar-injector-646655c8cd-rwvsx    1/1       Running     0          2m
 istio-statsd-prom-bridge-7fdbbf769-8k42l   1/1       Running     0          2m
 istio-telemetry-8687d9d745-mwjbf           2/2       Running     0          2m
 prometheus-55c7c698d6-f4drj                1/1       Running     0          2m

Inject the httpbin service with Istio

Enable the default Kubernetes namespace for autoinjection:

kubectl label namespace default istio-injection=enabled
kubectl get namespace -L istio-injection
NAME             STATUS    AGE       ISTIO-INJECTION
default          Active    19d       enabled
ibm-cert-store   Active    19d       
ibm-system       Active    19d       
istio-system     Active    15m       
kube-public      Active    19d       
kube-system      Active    19d

With this configured, your Kubernetes services will automatically get deployed with Istio’s sidecar proxy. Since the httpbin service is already deployed, deleting the httpbin pod will cause it to get redeployed with the sidecar in place:

kubectl delete pod -l app=httpbin
kubectl get po
NAME                       READY     STATUS        RESTARTS   AGE
httpbin-77647f7b59-bcp4q   2/2       Running       0          8s
httpbin-77647f7b59-gm7cc   0/1       Terminating   0          2h

The httpbin service now has two containers in the pod: httpbin app and istio-proxy sidecar. Verify the new httpbin pod is running and curl the httpbin service again.

curl -i http://$INGRESS_HOST/headers
HTTP/1.1 200 OK
Date: Tue, 14 Aug 2018 00:19:47 GMT
Content-Type: application/json
Content-Length: 533
Connection: keep-alive
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 3
x-envoy-decorator-operation: httpbin.default.svc.cluster.local:8000/*
{
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "0", 
    "Host": "istio-integration.us-south.containers.appdomain.cloud", 
    "User-Agent": "curl/7.54.0", 
    "X-B3-Sampled": "1", 
    "X-B3-Spanid": "f1a011b53ba9cc4b", 
    "X-B3-Traceid": "f1a011b53ba9cc4b", 
    "X-Envoy-Internal": "true", 
    "X-Forwarded-Host": "istio-integration.us-south.containers.appdomain.cloud", 
    "X-Global-K8Fdic-Transaction-Id": "f9327e92e4aec58c04c19451a50967c8", 
    "X-Request-Id": "612a99ee-cf5e-94b7-b2ae-cbe1f98ef4a9"
  }
}

Observe the additional headers returned in the response this time around. The Envoy proxy running in the Istio sidecar captures all traffic to the httpbin service, applies any traffic management policies, and then routes the request to the httpbin service.

Note: If Istio was deployed with –set mtls.enabled=true, the curl command above will fail since the Kubernetes Ingress resource does not have mTLS configured between the ALB and httpbin service. This option will be addressed in a follow-up blog.

Convert the Kubernetes Ingress resource to Istio Gateway and VirtualService rules

The ALB relies on Kubernetes Ingress resources to control how traffic is routed to services deployed in your cluster. In Istio, ingress traffic is configured via Gateways and VirtualServices. Gateways and VirtualServices provide a super set of the functionality provided by the Kubernetes Ingress resource in a more flexible format. In the same way that the routing behavior of the ALB is configured by Kubernetes Ingress resources, Istio uses Gateways and VirtualServices to configure Istio’s ingress gateway. Traffic from outside the Istio service mesh is routed by the ingress gateway:

kubectl get pods -n istio-system -l istio=ingressgateway
NAME                                   READY     STATUS    RESTARTS   AGE
istio-ingressgateway-f8dd85989-h5sb9   1/1       Running   0          44m

To make the transition easier for users who may already have Kubernetes Ingress resources defined for their services, Istio provides a converter tool as part of istioctl for migrating Ingress resource definitions to corresponding VirtualServices:

kubectl get ingress simple-ingress -o yaml > ingress.yaml
istioctl experimental convert-ingress -f ingress.yaml > vservice.yaml

Then, apply the generated VirtualService along with a standard Gateway which opens port 80 for incoming connections:

kubectl apply -f vservice.yaml
cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-autogenerated-k8s-ingress
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
EOF

Finally, the original Kubernetes Ingress resource needs to be deleted and recreated so that it points to the Istio ingress gateway rather than the httpbin service directly:

kubectl delete ingress simple-ingress
cat <<EOF | kubectl apply -f -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: simple-ingress
  namespace: istio-system
spec:
  rules:
    - host: $INGRESS_HOST
      http:
        paths:
          - path: /headers
            backend:
              serviceName: istio-ingressgateway
              servicePort: 80
EOF

Perform the curl command once more to verify that traffic is now routed from the ALB to the Istio ingress gateway and finally the httpbin service.

curl -i http://$INGRESS_HOST/headers

Replace IBM Cloud Kubernetes Service ALB with Istio ingress gateway

IBM Cloud Kubernetes Service provides a bring-your-own-ingress option if developers want to use their own ingress solution over the ALB. The steps below will enable the Istio ingress gateway to accept web traffic directly instead of first going through the ALB.  The first step is to disable the current ALB for your cluster.

To get the ALB ID for your cluster, run the following kubectl command:

kubectl get svc -n kube-system | grep alb

Then, disable the ALB deployment using the ID returned:

ibmcloud ks alb-configure --albID <ALB_ID> --disable-deployment

Disabling the ALB deployment may take a few minutes to complete. Verify the corresponding ALB pods are deleted from the kube-system namespace before continuing:

kubectl get pods -n kube-system | grep alb
public-cr7591595a4fb54d839cf3a77d12147ba4-alb1-9d858db58-8k65p   2/2       Terminating   0          17m
public-cr7591595a4fb54d839cf3a77d12147ba4-alb1-9d858db58-qn8jh   2/2       Terminating   0          17m

Next, delete the current Istio ingress gateway so that it can be recreated in the kube-system namespace:

kubectl delete deployments -n istio-system istio-ingressgateway
kubectl delete svc -n istio-system istio-ingressgateway

Use Helm to deploy only the ingress gateway to the kube-system namespace:

helm install -f install/kubernetes/helm/istio/values-istio-gateways.yaml ibm-stable/ibm-istio \
--name=istio-ingress \
--namespace kube-system \
--set global.omitSidecarInjectorConfigMap=true

Update the ALB service to point to the new ingress. Under spec/selector, remove the ALB ID from the app label and add the label for istio-ingressgateway:

kubectl edit svc <ALB_ID> -n kube-system
spec:
...
  selector:
    app: istio-ingressgateway
  sessionAffinity: None
  type: LoadBalancer
...

Now, you can curl the Istio ingress gateway directly using the ALB URL provided for your cluster:

curl -i http://$INGRESS_HOST/headers

Conclusions

With these instructions, existing services can be migrated from the default IBM Cloud Kubernetes Service ALB to using the Istio ingress gateway. After migration is complete, you can begin exploring the additional ingress configuration options for your service mesh available with Istio Gateways and VirtualServices.

Related links

Staff Software Engineer

More stories
September 20, 2018

Deploying to IBM Cloud Private 2.1.0.3 with IBM Cloud Developer Tools CLI

IBM Cloud Developer Tools CLI version 2.1.4 adds deployment support for IBM Cloud Private 2.1.0.3. This version of IBM Cloud Private uses a more secure Helm for Kubernetes deployments and simplifies the cluster configuration for the client compared to prior IBM Cloud Private releases.

Continue reading

September 19, 2018

Serverless Functions vs. Virtual Machines: A Total Cost of Ownership Comparison

Explore relevant costs, performance, and availability issues for a Total Cost of Ownership comparison of virtual machine and serverless functions.

Continue reading

September 19, 2018

Tutorial: Apply End-to-End Security to Cloud Applications

A new tutorial will show you how to use IBM Cloud services to secure your cloud application. Capture and review security-related events, encrypt storage, integrate authentication, and more.

Continue reading