Have you ever wondered how controllers work in Kubernetes and wished you could make your own?
In this post, I will show you how to use the Python client-kubernetes library to interact with Kubernetes and build your own custom controller interacting with Custom Resource Definitions (CRD).
Requisites
- A running Kubernetes/OpenShift cluster (
oc cluster up
is fine for that matter) - Clone my samplecontroller repo if you want to follow the code review
Context
Our custom controller will be watching for Guitar Objects. Upon creation, it will check their brand and put a corresponding comment. Note that the algorithm I use for reviewing guitars is fairly simple, based on my own tastes (and some guitars I own).
Custom Resource Definitions
Those Guitar objects are defined as Custom Resource Definitions, which are a way to provide an abstraction layer on top of etcd and to easily define new objects on top of Kubernetes. They are the successor to ThirdPartyResources (TPRs), though they conceptually provide similar benefits.
This is what a Custom Resource Definition looks like:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: guitars.kool.karmalabs.local
spec:
group: kool.karmalabs.local
version: v1
scope: Namespaced
names:
plural: guitars
singular: guitar
kind: Guitar
shortNames:
- guit
You can inject this definition with oc create -f
and you will then be able to create your own guitar "instances". For instance, to define a stratocaster, you can use the following yaml:
apiVersion: "kool.karmalabs.local/v1"
kind: Guitar
metadata:
name: stratocaster
spec:
brand: fender
review: false
We can populate whatever fields we want in the spec part of the definition, although the custom controller part is needed to make any use of it
Running the Controller
The following commands will:
- Create a dedicated project to hold the controller pod.
- Give enough privileges to this pod (this is needed only because I am checking guitars cluster wide).
- Deploy the custom controller (using the image I created for that matter).
oc new-project guitarcenter
oc adm policy add-cluster-role-to-user cluster-admin -z default -n guitarcenter
oc new-app karmab/samplecontroller
About the Code
Let's review relevant parts of the code:
- By checking whether
KUBERNETES_PORT
environment variables are defined or not, we detect whether code is being run within a pod or outside. This allows us to use the same code to run manually (which is quite handy during development). At this step, we are authenticated and can talk to the Kubernetes API.
if 'KUBERNETES_PORT' in os.environ:
config.load_incluster_config()
else:
config.load_kube_config()
- We check whether the Custom Resource Definition associated to the Guitar kind exists, or otherwise create it using a yaml file containing the same exact content indicated above.
definition = 'guitar.yml'
v1 = client.ApiextensionsV1beta1Api()
current_crds = [x['spec']['names']['kind'].lower() for x in v1.list_custom_resource_definition().to_dict()['items']]
if 'guitar' not in current_crds:
print("Creating guitar definition")
with open(definition) as data:
body = yaml.load(data)
v1.create_custom_resource_definition(body)
- We enter a loop using a watch to monitor guitar objects. We also wrap the watch with an additional
while True
to prevent unwanted timeouts.
I'm usingcrds.list_cluster_custom_object
, instead of methods limited to the current namespace, which is why my controller needs extra permissions.
Once we get a new object, we check its specs to see if it's been reviewed and if not, and launch an auxiliar method to actually process it.
crds = client.CustomObjectsApi()
DOMAIN = "kool.karmalabs.local"
resource_version = ''
while True:
stream = watch.Watch().stream(crds.list_cluster_custom_object, DOMAIN, "v1", "guitars", resource_version=resource_version)
for event in stream:
obj = event["object"]
operation = event['type']
spec = obj.get("spec")
if not spec:
continue
metadata = obj.get("metadata")
resource_version = metadata['resourceVersion']
name = metadata['name']
print("Handling %s on %s" % (operation, name))
done = spec.get("review", False)
if done:
continue
review_guitar(crds, obj)
How to Use
Create some guitars using the provided yaml files and see the review made for you (notice how the comment field of the spec gets updated).
oc create -f crd/stratocaster.yml
oc create -f crd/lespaul.yml
oc get guitars -o yaml
Running the Additional UI
To ease testing, you can also use the provided UI so you can list, create, and delete guitars.
The UI uses flask and the same client. For that, we deploy an additional pod and expose its route.
oc new-app karmab/sampleui
oc expose svc sampleui
SAMPLEUI=$(oc get route sampleui -o jsonpath='{.spec.host}{"\n"}')
Now you can access the URL defined as $SAMPLEUI
and watch existing guitars and their corresponding review.
You can also create and delete additional guitars through the UI.
Running Your Own Code
To build your own controller using the same library, you can build your container on top of my karmab/client-python-kubernetes docker image
Conclusion
Custom controllers give you the ability to interact easily upon creation/deletion/update of objects in your cluster.
Combined with Custom Resource Definitions, they provide a nice abstraction to etcd and ease the design of your application (did someone say microservices?) within the container platform.
If you felt like all of this required learning a bunch of golang, you now know you have a possible alternative to get started. So "go" for it! (Or Python it!)
About the author
More like this
Browse by channel
Automation
The latest on IT automation that spans tech, teams, and environments
Artificial intelligence
Explore the platforms and partners building a faster path for AI
Open hybrid cloud
Explore how we build a more flexible future with hybrid cloud
Security
Explore how we reduce risks across environments and technologies
Edge computing
Updates on the solutions that simplify infrastructure at the edge
Infrastructure
Stay up to date on the world’s leading enterprise Linux platform
Applications
The latest on our solutions to the toughest application challenges
Original shows
Entertaining stories from the makers and leaders in enterprise tech
Products
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Cloud services
- See all products
Tools
- Training and certification
- My account
- Developer resources
- Customer support
- Red Hat value calculator
- Red Hat Ecosystem Catalog
- Find a partner
Try, buy, & sell
Communicate
About Red Hat
We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Select a language
Red Hat legal and privacy links
- About Red Hat
- Jobs
- Events
- Locations
- Contact Red Hat
- Red Hat Blog
- Diversity, equity, and inclusion
- Cool Stuff Store
- Red Hat Summit