In this post, we discuss the development and packaging of operators.
With all the hoopla about operators in Kubernetes, what are they? A quick Google search retrieved this definition:
“A Kubernetes operator is a method of packaging, deploying and managing a Kubernetes application. A Kubernetes application is both deployed on Kubernetes and managed using the Kubernetes API (application programming interface) and kubectl tooling.”
For a closer look at Kubernetes operators, check out the following video:
So, an operator it is a mechanism to do something (install, build, manage an application, etc.) in Kubernetes. In my previous post, “Demystifying Operator Deployment in OpenShift,” I discussed the OperatorHub and the deployment process of an operator. In this post, I will discuss operator development and packaging.
An operator runs in a deployment-based pod and manages a specific Custom Resource Definition (CRD). It runs in a loop, continuously checking on the assigned CRDs. For each of the CRDs, the operator pod runs the reconcile() method. The method will be invoked for each CRD that is discovered in the namespace. The typical action of the method is a flow of reading the spec of the CRD, performing actions or checking the cluster based on that spec information and then writing the evaluation results to the status section of the CRD.
Now, let’s look into more detail on how to build an operator.
Let’s start with how operators are constructed. For that, you will most likely start with Operator SDK, a complete software development tool to build operators that are based on Go, Ansible or Helm. This article is focused on the Go-based operator, as it is the one that provides most functions.
First, you initialize a template scaffolding for your operator. For this, you must provide a domain (the qualifier for your operator, similar to your base DNS) and a GIT-like repository:
The result of the init is a skeleton of configuration files with
kustomization.yaml files, such as the following:
With that scaffolding created, you then must define the main content of the operator (i.e., the operator API). The API consists of the Custom Resource Definition (CRD) and the operator controller program:
As indicated in the output above, the generated files are as follows:
api/<version>/<kind>_types.go: Defines the structure of the CRD
controllers/<kind>_controller.go: Defines the processing logic
The CRD is defined in the
api/<version>/<kind>_types.go file. The main object of the operator called
Operator1 is defined as follows:
The development activity mainly adds the CRD fields in the
Operator1Status constructs, which coincide with the
status: sections of the CRD in the YAML definition. The fields definition can be qualified using the
kubebuilder directives to perform generation, validation and processing specifics to the fields (see here for more info).
The following is an example for the fields:
Once the CRD structure is finalized, you can validate and generate the YAML file to define the CRD:
The manifest that is generated is a YAML file in
config/crd/bases/<group>.<domain>_<type>s.yaml. It is the generated Custom Resource Definition YAML that you can load to Kubernetes, and it allows you to create objects to be managed by the operator.
Define the controller
The controller is a Go program, and the program resides in
controllers/<kind>_controller.go. In the code, the following block is the main content that you must modify — the Reconcile function:
As discussed above, the Reconcile function is called for each occurrence of the Custom Resource. The logic flow of the Reconcile function is to read in the Custom Resource (primarily from the spec field), perform its processing and write out to the status field. A simple Reconcile function that checks the
action field and writes to the
stage field is shown below:
With the controller logic defined, the container image that runs the operator controller can be built. Another set of make commands can be run by specifying the target image name:
The result is stored in the container image in your local machine registry. You can push it up to a Docker registry so that you can run your operator.
To package the operator, you use a combination of the operator-sdk and the operator package manager tool (see here for more info).
To clarify this process, let’s evaluate the following images terminology:
- Operator runtime: The runtime image you create for running the operator controller process(created using the
- Operator bundle: The container image that contains the manifests to install and activate the operator in a Kubernetes cluster, including roles, role-binding, manager, CRD and others.
- Operator catalog: The container image that has pointers and lists all operator bundles that are provided in this catalog.
To generate the operator bundle, run the
make bundle-build command:
The bundle contains the content of the bundle path YAMLs, which is created using the kustomize tool from the config directory:
Once you have the operator bundle image created and pushed into an image repository, you add the bundle into an operator catalog (which you can then import into OperatorHub):
When the operator catalog image has been built successfully and pushed to a registry, you can add a CatalogSource entry to OperatorHub and start installing your operator. See “Demystifying Operator Deployment in OpenShift” for further instructions.
For example, CatalogSource entry can be entered from the OpenShift console under Administration > Cluster Settings > Global Configuration > OperatorHub > Sources and the clicking Create Catalog Source:
Once the Catalog Source is
READY, you can see the operator bundle from the OperatorHub:
In this article, a very simple operator is built, defined and loaded into a registry, and allowing it to be shown in OperatorHub on a RedHat OpenShift environment.