Service discovery (kube-dns)

Kubernetes expects that a service is running within the pod network mesh that performs name resolution and acts as the primary name server within the cluster.

In IBM® Cloud Private, this service is implemented by using CoreDNS , which runs on the master nodes. CoreDNS resolves names for all services that are running in Kubernetes and forwarding name lookups against upstream name servers on behalf of containers. The DNS service itself runs as a ClusterIP service that is backed by one or more containers for high availability.

Kubernetes service names are resolved to ClusterIPs representing one or more pods that match a label selector. The cluster is assigned a cluster domain that is specified at installation time by using cluster_domain to distinguish between names local to the cluster and external names. The default value of cluster_domain is cluster.local.

Each Kubernetes cluster is logically separated into namespaces and each namespace acts as a subdomain for name resolution. If you examine the /etc/resolv.conf file in a container, you see that the nameserver line points to an IP address that is internal to the cluster, and the search suffixes are generated in a particular order, as shown in the following example:

# cat /etc/resolv.conf 
nameserver <kube-dns ClusterIP>
search <namespace>.svc.<cluster_domain> svc.<cluster_domain> <cluster_domain> <additional ...>
options ndots:5

The <additional ...> is a list of search suffixes obtained from the /etc/resolv.conf file of the worker node.

By default, a short host name like account-service has <namespace>.svc.<cluster_domain> appended to it so a pod that matches the label selector that is running in the same namespace as the running pod is selected. A pod can look up the ClusterIP of a pod in a different namespace by appending the namespace to the host name. For example, account-service.prod targets account-service that is running in the prod namespace, since the search suffix svc.<cluster_domain> is appended to the end.

Note the last line in /etc/resolv.conf, which is options ndots:5. The line indicates to the system resolver of the container that host names that are being resolved and have fewer than five dots in the name must have the search domain suffixes appended to them. This configuration can affect performance since lookups of external network resources with fewer than five dots in the name result in lookups for every entry in the search line. For example, a lookup of www.ibm.com results in lookups of www.ibm.com.<namespace>.svc.<cluster_domain>, www.ibm.com.svc.<cluster_domain>, www.ibm.com.<cluster_domain>, and so on, before it tries www.ibm.com. To resolve this issue, adding . to the end of fully qualified domain names that are used in the application configuration prevents the system resolver from cycling through the list of suffixes in the name lookups. For example, www.ibm.com..

Headless services

In some cases, it is desirable to not create a ClusterIP service at all. You can do so by specifying a service type of None. This specification creates A records for each pod that matches the label selector in the DNS, but no ClusterIP. This specification is typically used with StatefulSets where each of the pods needs to have resolvable names for communication between all pods in the set. For example, a clustered database, such as MongoDB. When a headless service is created, the A record of each pod is in the format <pod-name>.<service-name>.<namespace>.svc.<cluster-domain>.

External services

It is possible to have Kubernetes proxy endpoints outside of the cluster. You can do so by creating a Service resource with no label selector, and by either manually creating Endpoints resources that contain the IP addresses that are outside of the cluster to proxy, or by creating a Service resource with the type ExternalName that contains an external DNS name. This action creates a CNAME record in the DNS of the cluster. By using these functions, the cluster DNS can be used as service discovery for services both inside and outside of the cluster.