A Kubernetes operator is not an Ops specialist or team, but an automated operators framework that can be used on Kubernetes (or OpenShift). The operators framework was introduced by CoreOS, now part of RedHat and (as of 2019) IBM, in 2016. A great way to manage complex Kubernetes operations, the operators framework is quickly becoming a popular, and core, element of cloud-native, DevOps architectures.
In this article, we’ll explain the role operators perform in scalable Kubernetes-based application architecture. And why they have become such a key part of DevOps automation. Much of the text in this section is inspired by IBM’s great explainer video on Kubernetes operators, which you can also watch as a general introduction to the concepts:
We’ll then guide you through a step-by-step tutorial to building an operator using Operator SDK and Ansible.
What is the Kubernetes control loop?
As a core part of the operators framework, it makes sense to first explain what the Kubernetes control loop is. The K8s control loop observes the state of what’s in a cluster. The next thing Kubernetes does is compare the actual state of a cluster with its desired state. That comparative process is called a diff.
The final phase of a Kubernetes control loop is resolving any diff by acting on it. This phase of the control loop is, predictably, termed the act phase.
The control loop is core to how Kubernetes works. And there’s a controller that acts on that loop for every default resource that Kubernetes comes with.
Deploying a Kubernetes application without operators
As an end user, the first step is to write up some YAML – the spec for the application. For example, we’re doing a deployment, which involves defining some configurations. eg.
What’s the image?
assorted other configurations
We now have a Kubernetes resource to be deployed into the cluster. The control loop now kicks in and checks the difference between what is wanted in the cluster and its actual state. Kubernetes will notice, for example, there are no pods. It will then act on that diff and create pods.
A more complex application may have more than one YAML. There could be a second for the backend. That will also be deployed into a cluster, with a pod subsequently deployed using the controllers and control loop.
Now, if we want to scale up the application, make some changes, add secrets and environment variables etc., we’ll have to either set up new Kubernetes resources each and every time, or go back and edit the existing ones. That can start to become complex and time consuming.
Deploying a Kubernetes application with operators
How would we deploy the same application using an operator? The first step is installing the operator itself. That, of course, means you’ll need an operator that will do the job you want it to. You can either have someone custom build a Kubernetes operator for you, do it yourself if you have the relevant technical background, or find a suitable option among the growing library available on OperatorHub.
The first thing now needed in the Kubernetes cluster is the Operator Lifecycle Manager (OLM). The OLM manages the operators that have been installed. Next, the operator is deployed into the cluster.
A Kubernetes operator is made up of two major components:
The CRD (custom resource definition) is something which, as opposed to default Kubernetes resources like deployments and pods, is something defined in Kubernetes by either the user or operator, so that YAML is created to work against the custom configuration.
The controller is a custom Kubernetes control loop, which runs as a pod in the cluster, and the control loop against the CRD.
If the operator has been created for the same custom application deployment as per the first example of deploying a K8s application without an operator, what differs? Instead of having to write up multiple deployments, config maps, secrets etc., just one YAML will need to be deployed.
Custom configurations could be assigned to the operator, or use the set defaults. The operator is then deployed directly into the cluster. The operator then takes over and is responsible for running the Kubernetes control loop and figuring out exactly what needs to be running.
The operator would, for example, realise a couple of deployments and pods are needed if our application is the same as that deployed without Kubernetes operators in the first example.
Why use Kubernetes operators?
Kubernetes operators are an approach to managing complex applications that is inherently more scalable (and just easier) than deploying Kubernetes clusters without an operator. The end-user only has to worry about the config that’s been exposed to them. The operator manages the control loop and the state of the application – how it needs to look.
Operators can be used for automating a large variety of processes. For example, after creating and deploying a config file in a few seconds, we can get a configured environment or service like Cache service (Redis, Memchace), Proxy (NGNX, HAProxy), Databases (Mysql), etc.
Building your own operators
There are now many fantastic operators out there, like the Prometheus Operator for Kubernetes native deployment and management of Prometheus and related monitoring components. You can check out our blog post dedicated to the step-by-step set-up of the Prometheus Operator for Kubernetes monitoring.
Before considering custom creating an operator, it always makes sense to look carefully at what is already out there. As already mentioned, OperatorHub is a great resource where the Kubernetes community shares operators.
But what if we want to develop a custom operator for something native to a specific application architecture? There’s a number of ways to do that.
Operator SDK allows us to start building out operators ourselves. The easiest way to get started with an operator is to use the Helm operator.
The Helm approach
The Helm approach allows us to take a Helm chart and apply that towards an operator. This gets us close to a pretty mature operator for a pre-existing chart. If you aren’t particularly familiar with Helm, you might be interested in our blog post covering the fundamentals of how Helm contributes to a Kubernetes architecture.
Levels of Operator maturity
Operator maturity is broken down into five levels:
Basic install – allows for the provisioning of the resources required.
Upgrades – supports minor and patched upgrades to whatever is defined in the operator.
Full lifecycle support – storage lifecycle, app lifecycle, backup and failure recovery.
Insights – deep metrics, analysis, logging etc.
Autopilot – automated horizontal and vertical scaling, config. tuning, acting on diffs.
Helm itself hits the first two levels of maturity.
For operators that meet maturity levels 3-5, Go and Ansible are the most popular relevant technologies. Operator SDK allows us to build operators using Helm, Go, Ansible and other technologies. You might also be interested in our Ansible tutorial, which will take you through the step-by-step process of installing and setting up this incredibly useful open-source RedHat tool.
A step-by-step guide to building a Kubernetes operator with Operator SDK and Ansible
The example operator we are going to create here using Operator SDK and Ansible will perform two core functions:
Create a namespace and apply Limit Ranges and Resource Quota to it
Create a deployment Nginx in a namespace
We are not going to focus on the installation process here. You can find a step-by-step guide on how to install Kubernetes Operator SDK at GitHub. Here, we’ll focus on the process of writing an operator.
Create a new project
We first need to create a new project, which we’ll do via the CLI (command-line interface):
operator-sdk new my-first-operator --api-version=krusche.io/v1alpha1 --kind=/v1alpha1 --kind=ResourcesAndLimits --type=ansible
This command will create a project with the operator that will subscribe to the resource ResourcesAndLimits with APIVersion krusche.io/v1alpha1 and Kind ResourcesAndLimits.
The directory should be structured in the following way:
Directory/File – Goal
build/ – Contains scrips using which operator-SDK will be assembled and initialized
deploy/ – Contains a set of Kubernetes manifests using which operator will be deployed in the cluster
roles/ – сontains Ansible roles
watches.yaml – Contains Group, Version, Kind, and method of launch Ansible
The file watches contains:
group: The group in Custom Resource to which our operator subscribes.
version: The version of Custom Resource to which our operator subscribes.
kind: The type of Custom Resource to which our operator subscribes.
role (default): The path to our Ansible roles.
playbook: The path to Ansible playbook. It is required for the case if we will use a playbook instead of a role.
vars: Described in the form of key-value. Will be passed as extra_vars
reconcilePeriod (optional): The negotiation interval that defines how often the role will run for this CR.
manageStatus (optional): If the value is set to true (default), the operator will manage the state of CR. If the value is set to false, then the status of CR is managed elsewhere with the help of the specified role/playbook of a separate controller.
kubectl -n nginx-namespace get pod
NAME READY STATUS RESTARTS AGE
kc-nginx-1-17-6-nginx-859d7dcf99-ddbzm 1/1 Running 0 24s
kc-nginx-1-17-6-nginx-859d7dcf99-j884q 1/1 Running 0 24s
kc-nginx-1-17-6-nginx-859d7dcf99-mvj5s 1/1 Running 0 24s
kc-nginx-1-17-6-nginx-859d7dcf99-wpcfj 1/1 Running 0 24s
kc-nginx-1-17-6-nginx-859d7dcf99-x7szg 1/1 Running 0 53s
That’s it! You’ve built your own Kubernetes operator!
Kubernetes Operators – The Bottom Line
The operator we created in this guide subscribes to K8s resources through the Kubernetes api and automates their management, without the need for manual intervention. Ultimately, Kubernetes operators help you automate routine manual work. That’s usually always beneficial when possible
What tech stack has been used? All we needed to build our operator were the operator SDK framework and Ansible.