The question “What’s the difference between Kubernetes and OpenShift?” comes up every now and then, and it is quite like asking: “What’s the difference between an engine and a car?”
To answer the latter, a car is a product that immediately makes you productive: it is ready to get you where you want to go. The engine, in return, won’t get you anywhere unless you assemble it with other essential components that will form in the end a … car.
As for the first question, in essence, you can think of it as Kubernetes being the engine that drives OpenShift, and OpenShift as the complete car (hence platform) that will get you where you want to.
This question comes up every now and then, so the goal of this blog post is to remind you that:
- at the heart of OpenShift IS Kubernetes, and that it is a 100% certified Kubernetes, fully open source and non-proprietary, which means:
- The API to the OpenShift cluster is 100% Kubernetes.
- Nothing changes between a container running on any other Kubernetes and running on OpenShift. No changes to the application.
- OpenShift brings added-value features to complement Kubernetes, and that’s what makes it a turnkey platform, readily usable in production, and significantly improving the developer experience, as will be shown throughout the post. That’s what makes it both the successful Enterprise Platform-as-a-Service (PaaS) everyone knows about from a developer perspective, but also the very reliable Container-as-a-Service from a production standpoint.
OpenShift IS Kubernetes, 100% Certified by the CNCF
Certified Kubernetes is at the core of OpenShift. Users of `kubectl` love its power, once they are done with the learning curve. Users transitioning from an existing Kubernetes Cluster to OpenShift frequently point out how much they love redirecting their kubeconfig to an OpenShift cluster and have all of their existing scripts work perfectly.
You may have heard of the OpenShift CLI tool called `oc`. This tool is command compatible with `kubectl`, but adds a few extra special helpers that help get your job done. But first, let’s see how oc is just kubectl:
|kubectl commands||oc commands|
kubectl get pods
oc get pods
kubectl get namespaces
oc get namespaces
kubectl create -f deployment.yaml
oc create -f deployment.yaml
Here are the results of using kubectl commands against an OpenShift API:
- kubectl get pods => well, it returns … pods
- kubectl get namespaces => well, it returns… namespaces
kubectl create -f mydeployment.yaml => it creates the kubernetes resources just like it would on any other Kubernetes platform. Let's verify that with the following video:
In other words, the Kubernetes API is fully exposed in OpenShift, 100% compliant to the upstream. That’s why OpenShift is a Certified Kubernetes distribution by the CNCF.
OpenShift brings added-value features to complement Kubernetes
While the Kubernetes API is 100% accessible within OpenShift, the kubectl command-line lacks many features that could make it more user-friendly, and that’s why Red Hat complements Kubernetes with a set of features and command-line tools like OC (OpenShift client) and ODO (OpenShift DO, targeting developers).
1 - “oc” complements “Kubectl” with extra power and simplicity
OC for instance is the OpenShift command-line that adds several features over kubectl, like the ability to create new namespaces, easily switch context, and commands for developers such as the ability to build container images and deploy applications directly from source code or binaries, also known as the Source-to-image process, or s2i.
Let’s take a look at a few instances of where oc has built-in helpers and additional functionality to make your day to day life easier.
First example is namespace management. Every Kubernetes cluster has multiple namespaces, usually to provide environments from development to production, but also for every developer that will need sandbox environments for instance. This means you’re going to switch between them frequently, since kubectl commands are contextual to your namespace. If you’re using kubectl, you will frequently see folks use helper scripts to do this, but with oc you just say oc project foobar to switch to foobar.
And if you can’t remember your namespace name? You can just list it out with oc get projects. What if you only had access to a subset of the namespaces on the cluster? That command should list them out right? Not so with kubectl, unless you have RBAC access to list all namespaces on the cluster, which is not frequently granted on larger clusters. But with oc, you easily get a list of your namespaces. A small way Openshift is enterprise-ready and designed to scale with both your human users and applications.
2 - ODO improves the developer experience over kubectl
Another tool that Red Hat brings with OpenShift is ODO, a command-line that streamlines the developer experience, allowing them to quickly deploy local code to a remote OpenShift cluster, and have an efficient inner loop where all their changes can instantly be synced with the running container in the remote OpenShift, avoiding the burden of rebuilding the image, pushing it to a registry then deploying it again.
Here are a few examples where “oc” or odo command makes life easier to work with containers and Kubernetes.
In the following section, let's compare a kubectl-based workflow to using oc or odo.
- Deploying code to OpenShift without being a YAML native-speaker:
|Kubernetes / kubectl||
$> git clone https://github.com/sclorg/nodejs-ex.git
1- Create a Dockerfile that builds the image from code
COPY package*.json ./
COPY index.js ./
COPY ./app ./app
RUN npm install
CMD [ "npm", "start" ]
2- build the image
$> podman build ...
3- login to a registry
podman login ...
4- Push the image to a registry
5- Create yaml files that will help deploy the app (deployment.yaml, service.yaml, ingress.yaml) are the bare minimum
6- deploy the manifest files:
Kubectl apply -f .
|OpenShift / oc||
$> oc new-app https://github.com/sclorg/nodejs-ex.git --name myapp
|OpenShift / odo||
$> git clone https://github.com/sclorg/nodejs-ex.git
$> odo create component nodejs myapp
$> odo push
Switching contexts: changing working namespace or working cluster
|Kubernetes / kubectl||
1- create context in kubeconfig for “myproject”
2- kubectl set-context ...
|OpenShift / oc||
oc project “myproject”
Quality assurance process: “I have coded a new alpha feature, should we ship it to production?”
When you try a prototype car and they say: “We’ve put in some new types of brakes, honestly I’m not sure if they’re safe yet… but GO AHEAD and try it, and during the championship!”, do you blindly do so? I guess NO, and we feel the same way at Red Hat :)
That’s why we might refrain from alpha features until they mature, and we have battle-tested them during our qualification process and feel it is safer to use. Usually it goes through a Dev Preview phase, then a Tech Preview phase, then a General Availability Phase when they’re stable for production.
Why is that so ? Because, like in any other software craftsmanship, some initial concepts in Kubernetes might never make it in the final release, or they could functionally but in a very different implementation than what was initially delivered as an alpha feature. Because Red Hat is supporting more than a thousand customers for business-critical missions with OpenShift, we believe in delivering a stable and long-supported platform.
Red Hat is actively delivering frequent OpenShift releases, and updates the Kubernetes version within OpenShift, for instance OpenShift 4.3 which is the current GA release embeds Kubernetes 1.16, just one version behind upstream Kubernetes version 1.17; this is on-purpose in order to deliver production-grade Kubernetes and do extra Quality Assurance within the OpenShift release cycle.
The Kubernetes escalation flaw: “There is a critical Kubernetes bug in Production, do I need to upgrade all my production up to 3 releases to get the fix?”
In the Kubernetes upstream project, fixes are usually delivered in the next release, and sometimes contributed to 1 or 2 minor releases, spanning a 6 months time frame.
Red Hat has a proven track record of fixing critical bugs earlier than others, and on a much longer time frame. Take a look at the Kubernetes privilege escalation flaw (CVE-2018-1002105) that was discovered in kubernetes 1.11 and backward fixed upstream only until 1.10.11, leaving all previous kubernetes 1.x until 1.9 subject to the flaw.
On the opposite, Red Hat patched OpenShift until version 3.2 (or Kubernetes 1.2), spanning 9 OpenShift releases backwards, showing it would actively support its customers in these difficult situations. (See this blog for further information).
Kubernetes upstream benefits from OpenShift and Red Hat’s contributions to code
Red Hat is the second largest code-contributor to Kubernetes behind Google and currently employs 3 of the top 5 Kubernetes code contributors. Little known is that many critical features of upstream Kubernetes have been contributed by Red Hat. Some major examples of these are:
- RBAC: for some time, Kubernetes didn’t implement RBAC features (ClusterRole, ClusterRoleBinding), until Red Hat engineers decided to implement them in Kubernetes itself rather than as an added-value feature of OpenShift. So is Red Hat afraid of improving Kubernetes? Of course not, that’s what makes it Red Hat and not just any other Open Core software provider. Improvements that are made in the upstream communities mean more sustainability and broad adoption, which ultimately is the goal, to make these open source projects drive benefits for the customers.
- Pod Security Policies: Initially, these concepts that allow secure execution of applications within pods were present in OpenShift and known as Security Context Constraints or SCC. Again, Red Hat decided to contribute them upstream, and now everyone using Kubernetes or OpenShift benefits from it.
There are many more examples, but these are simple illustrations that Red Hat is committed to make Kubernetes an even more successful project.
So now the real question: what is the difference between OpenShift and Kubernetes? :)
By now, I hope you understand that Kubernetes is a core component of OpenShift, but nonetheless ONE component among MANY others. That means that just installing Kubernetes to have a production grade platform is not enough: you’ll need to add authentication, networking, security, monitoring, logs management, … That means you will also have to pick your tools among everything available (see CNCF landscape to get an idea of the complexity of the ecosystem), and maintain the cohesion of all of them as a whole; but also do updates and regression tests whenever there is a new version of one of these components. That finally means you are turning into a software editor, except that you are spending effort on building and maintaining a platform rather than investing on business value that will differentiate you from your competitors.
With OpenShift, Red Hat has decided to shield this complexity and deliver a comprehensive platform, including not only Kubernetes at its core, but also all the essential open source tools that make it an enterprise-ready solution to confidently run your production. Of course, in case you already have your own stacks, then you can opt-out and plug into your existing solutions.
Let’s look at Figure 1: surrounding Kubernetes are all the areas where Red Hat adds features that are not in Kubernetes by design, among which:
1- A trusted OS foundation: RHEL CoreOS or RHEL
Red Hat has been the leading provider of Linux for business-critical applications for over 20 years, and is putting its experience in delivering a SOLID and TRUSTED foundation for running containers in production. RHEL CoreOS shares the same kernel as RHEL, but is optimized for running containers and managing Kubernetes clusters at scale: and takes lower footprint and its immutable nature makes it easier to install clusters, adds auto-scaling, auto-remediation for workers, etc. All these features makes it the perfect foundation to deliver the same OpenShift experience, anywhere from bare-metal to private clouds and public clouds
2- Automated operations
Automated installation and day-2 Operations are OpenShift key features that make it easier to administrate, upgrade, and provide a first-class container platform. The usage of operators at the core of OpenShift 4 architecture is a strong foundation to make this possible.
OpenShift 4 also includes an extremely rich ecosystem of operators based solutions, developed by Red Hat and by 3rd party partners (see the operators catalog for Red Hat hosted operators, or operatorhub.io, a public marketplace created by Red Hat, to see community operators too).
OpenShift 4 gives you access to over 180 operators from the integrated catalog
3- Developer services
Since 2011, OpenShift has been a PaaS or Platform-as-a-Service, meaning that its goal is to make developer’s daily life easier, allowing them to focus on delivering code and offering a rich set of out-of-the-box support for languages such as Java, Node.js, PHP, Ruby, Python, Go and services like CICD, databases, etc. OpenShift 4 offers a rich catalog of over 100 services delivered through Operators, either by Red Hat or by our strong ecosystem of partners.
OpenShift 4 also adds a graphical UI (the developer console) dedicated to developers, allowing them to easily deploy applications to their namespaces from different sources (git code, external registries, Dockerfile…) and providing a visual representation of the application components to materialize how they interact together.
In addition, OpenShift provides Codeready sets of tools for developers, such as Codeready Workspaces, a fully containerized web-IDE that runs on top of OpenShift itself, providing an IDE-as-a-service experience. For developers who still want to run everything on their laptop, they can rely on Codeready Containers, which is an all-in-one OpenShift 4 running on the laptop.
The integrated webIDE-as-a-service allows you to efficiently develop on Kubernetes/OpenShift
OpenShift also offers out-of-the-box advanced CI/CD features; such as containerized Jenkins with a DSL to accelerate writing your pipelines, or Tekton (now in Tech preview) for a more Kubernetes-native CICD experience. Both solutions offer a native integration with the OpenShift console, allowing to trigger pipeline, view deployments, logs etc.
4- Application services
OpenShift allows you to deploy traditional stateful applications, alongside cutting-edge cloud-native applications, by supporting modern architectures such as microservices or serverless. In fact; OpenShift Service Mesh provides Istio, Kiali and Jaeger out-of-the-box to support your adoption of microservices. OpenShift Serverless includes Knative but also joint initiatives with Microsoft such as Keda to provide Azure functions on top of OpenShift.
The integrated OpenShift ServiceMesh (Istio, Kiali, Jaeger) helps you with microservices development
To reduce the gap between legacy applications and containers, OpenShift allows you now to even migrate your legacy virtual machines to OpenShift itself by using Container Native Virtualization (now in TechPreview), making hybrid applications a reality and easing portability across clouds, both private and public.
A Windows 2019 Virtual Machine running natively on OpenShift with Container Native Virtualization (currently in Tech preview)
5- Cluster Services
Every enterprise-grade platform requires supporting services like monitoring, centralized logs, security mechanisms, authentication and authorization, networking management, and these are all features that come out-of-the-box with OpenShift with out-of-the-box open source solutions like ElasticSearch, Prometheus, Grafana. All these solutions are packed with pre-built dashboards, metrics, alerts that come from Red Hat’s experience in monitoring clusters at scale, giving you right-away the most important information for your production.
OpenShift also adds essential enterprise services like authentication with a built-in oauth provider, integration to your identity providers such as LDAP, ActiveDirectory, OpenID Connect, and so on.
Out-of-the-Box Grafana dashboards allows you to monitor your OpenShift cluster
Out-of-the-Box Prometheus metrics and alerting rules (+150) allows you to monitor your OpenShift cluster
This rich set of features and the deep-expertise Red Hat has in the Kubernetes ecosystem are the reason why OpenShift has a significant head start against other solutions in the market, as we can see in the following figure (see this article for more information).
“So far, Red Hat stands out as the market leader with 44 percent market share.
The company is reaping the fruit of its hands-on sales strategy, where they consult and train enterprise developers first and then monetize once the enterprise deploys containers in production.”
I hope you enjoyed this first part of a series to come, where I will be discussing the benefits that OpenShift adds on top of Kubernetes in every one of these categories.