A Comparison of Kubernetes Distributions

Kubernetes is currently one of the most successful and fastest growing IT infrastructure projects. Kubernetes was introduced in 2014 as an open source version of the internal Google orchestrator Borg. 2017 saw an increase in Kubernetes adoption by enterprise and by 2018, it has become widely adopted across diverse businesses, from software developers to airline companies. One of the reasons why Kubernetes gained popularity so fast is its open source architecture and an incredible number of manuals, articles and support provided by its loyal community.

No wonder that just as for any successful open source project, several distributions can be found in the market (think Linux here), offering various extra features and targeting a specific category of users.

Why do we at all have all these distributions? The answer is obvious: every vendor wants to guarantee performance. Since Kubernetes is open source, companies offering their own Kubernetes distributions cannot sell it. However, they provide support and maintenance of K8s clusters (the so-called "managed Kubernetes"). And, naturally, they want to support the product that they are confident in, so even if they make no changes to the code, they test their distro to know their product inside out and see how it works in real conditions.

Now let’s imagine that your company is planning to adopt Kubernetes and is not eager to setup and maintain the cluster itself. How do you choose the vendor and who are these vendors in the first place? Let’s see how some of K8s distros available on the market compare to each other and to the “vanilla” Kubernetes distribution.

"Vanilla Kubernetes"

What Are the Features?

If you install Kubernetes from the official repositories, you get… Kubernetes! There’s no sense in listing all the features of Kubernetes here because they are aplenty. If you don’t know what Kubernetes is, I suggest referring to the official docs. In short, if you install the classic Kubernetes, you don’t lose anything in functionality. You get an open source product with all its benefits and drawbacks, i.e. updates, freedom to customize it, incredible support by community, and (oops — but that’s a common story with open source — don’t blame Kubernetes!) bugs that you have to deal with or ask peers for help.

Who Backs It?

Kubernetes is a project supported by the Cloud Native Computing Foundation (CNCF) and a diverse community of Kubernetes users. If you don’t know what CNCF is, you should check it out — it is a reputable organization dedicated to cloud technology. Kubernetes is the first project to graduate from CNCF with Prometheus announced to graduate in August 2018. If you have heard of such tools as Helm, containerd, CoreDNS (if not, it’s time to check them out!), you should know that it’s CNCF that stands for their development.

What Is the Licensing and Pricing Model?

Kubernetes is open source and 100% free. You can install, use and update it to new versions just like any open source product.

Is It Easy to Setup?

There are many ways to setup a Kubernetes cluster ranging from very easy to moderately difficult. You can use minikube to install Kubernetes locally for experimentation and testing or kubeadm to bootstrap a cluster in cloud. For troubleshooting or best practice examples, you can consult a wide range of resources or simply create an issue on the official GitHub repository.

What and Who Is It For?

Kubernetes out-of-the-box is great for anyone who is either already familiar with the technology or wants to experiment with it. What I mean by this is that Kubernetes is powerful, but definitely not easy. Running it in production not being ready — and experienced — to handle issues may result in critical downtimes, so it’s wise to spend some time learning Kubernetes in sandbox mode before adopting it for end-user services.

RedHat OpenShift

What Are the Features?

OpenShift started as a separate project before Kubernetes and used a completely different technology. However, RedHat realized that Kubernetes was getting more and more traction so they intelligently adopted it as the core for OpenShift in the version 3.

The main differences from classic Kubernetes are:

• Advanced and integrated user management
• Integrated Docker registry
• Integrated CI pipelines
• Integrated resources templates
• Usage of similar but slightly different concepts, like Routers instead of Ingresses, Projects instead of Namespaces, etc.

Who Backs It?

OpenShift is backed by RedHat, an open source software mastodon which doesn’t need any further introduction. RedHat is the third largest contributor to the Kubernetes project after Google and the community according to Stackalytics, so they are probably one of the most legitimate companies to maintain a Kubernetes distribution.

What Is the Licensing and Pricing Model?

OpenShift has three pricing models:

• OKD, a free distribution of Kubernetes by OpenShift.
• OpenShift Enterprise, which can be either hosted and managed by RedHat, or be deployed on-premises by the client. The hosted version starts at $48,000/year for 3 masters, 3 etcds, and 4 application nodes.
• OpenShift Online, which is the PaaS version available online. It costs approximately $50/month for each 2 Gb of RAM, which is a bit expensive in comparison to other Kubernetes-as-a-Service providers.

Is It Easy to Setup?

It is not very complicated but it needs a specific configuration. Installation uses Ansible, so some experience with this configuration management tool is recommended.

What and Who Is It For?

OpenShift is clearly an enterprise-grade distribution. Their objective is to focus on stability rather than features. That’s why the released version is always one step behind Kubernetes. So at the moment, while Kubernetes has already released 1.11, OpenShift has released the 3.10 version, which is based on Kubernetes 1.10.

OpenShift is aimed at enterprise clients that prefer stable releases over features.

Tectonic

What Are the Features?

Tectonic is a very popular Kubernetes distribution which is currently being integrated with RedHat. The features in comparison to a vanilla Kubernetes are the following:

• Easy setup
• User friendly web UI
• User management
• Native support for Operators

Who Backs It?

Tectonic has been created by CoreOS, a company dedicated to container technologies. Their portfolio includes very valuable and popular products like CoreOS Linux, Quay Docker registry, Etcd distributed key value store, and Flannel CNI. The company has been acquired by RedHat, so we will probably see a convergence of OpenShift and Tectonic as announced by RedHat in their integration roadmap in the next months.

What Is the Licensing and Pricing Model?

Tectonic has a commercial licensing model, free for up to 10 nodes, and for bigger clusters, the pricing is approximately $1,000/month per 10 nodes which includes support.

Is It Easy to Setup?

Yes. There is the possibility to install it either through an installer or through Terraform.

What and Who Is It For?

Tectonic is for enterprise clients. However, the future of Tectonic is uncertain. Most likely the distribution will completely disappear and will be integrated with OpenShift, so if you plan to deploy a Kubernetes distribution, Tectonic is probably not your best option for the long term.

Rancher

What Are the Features?

Rancher is a container management platform that includes Kubernetes, so we can consider it a Kubernetes distribution. The main features it adds to a vanilla Kubernetes are:

• Cross-provider cluster deployment
• User management
• Web UI
• Integrated CI/CD pipelines

Who Backs It?

Rancher Kubernetes is backed by Rancher Labs, a startup founded in 2014. Their star product is Rancher container platform, but they also created RancherOS, a container-focused Linux distribution.

What Is the Licensing and Pricing Model?

Rancher is 100% open source. Their business model is based on offering consulting and support services. Their pricing is not public.

Is It Easy to Setup?

Very easy and well-documented. It is possible to use their own Kubernetes installation tool called RKE.

What and Who Is It For?

It’s difficult to know the target customer of Rancher since they are not very clear about it. In terms of features, the distribution is adapted to any type of company.

Canonical Kubernetes

What Are the Features?

The distribution is basically a vanilla Kubernetes that is easily deployable on the major public cloud providers and on private cloud solutions like OpenStack, enabling you to easily setup and manage cross-provider and cross-location Kubernetes clusters. The UI is the official Kubernetes Dashboard.

Who Backs It?

This distribution is backed by Canonical, the company behind the popular Linux distribution Ubuntu.

What Is the Licensing and Pricing Model?

Canonical Kubernetes is completely free. However, there are several support packages starting from $200/year for each virtual node (a minimum of $2,500 applies) and a maintenance package starting from 14,600 per ten nodes.

Is It Easy to Setup?

The installation can be done using Conjure-up or Juju, the deployment tools developed by Canonical.

What and Who Is It For?

Canonical Kubernetes doesn't add much to Kubernetes, and it shares the same features with vanilla Kubernetes. However, it allows setting up cross-provider and cross-location Kubernetes clusters and offers enterprise-grade support. It can be recommended for those companies already having or planning to have strong commercial ties with Canonical.

Kubernetes Distribution by Containerum

What Are the Features?

Containerum has two different (but complementary) products:

• Kubernetes distribution called KDC — Kubernetes Distribution by Containerum. The KDC is a also vanilla version of Kubernetes, tested and supported by Containerum team.
• Open Source Containerum Platform that can be installed on top of Kubernetes as a UI with several extra features like user management, metrics monitoring, CI/CD pipelines, etc.

Who Backs It?

Containerum is backed by Exon LV, a Latvian startup dedicated to container technologies.

What Is the Licensing and Pricing Model?

Both KDC and Containerum Platform are 100% open source, so you can deploy their solutions freely. Their business model is based on two streams of income:

• Consulting on DevOps and Infrastructure topics
• Support for Kubernetes Distribution by Containerum. Installation and support prices start from $550 per 10 nodes/year, installation is included in the price.

Their prices are very competitive in comparison to the other distributions.

Is It Easy to Setup?

It is relatively easy to setup, very similar to vanilla Kubernetes installation. You can also install Containerum Platform on top of an existing Kubernetes cluster with a Helm chart.

What and Who Is It For?

KDC + Containerum Platform is probably the simplest Kubernetes distribution. It fits also the need of those who already have a Kubernetes cluster but need more features than a vanilla Kubernetes cluster.

Conclusion

The list of distributions presented here is not exhaustive — I tried to gather some well-established players as well as some lesser-known ones. The final choice depends, of course, on the available resources — some companies prefer to invest time in nurturing in-house expertise, while some find it easier to use third-party services. In case you use Kubernetes for small-scale projects or just for fun (Kubernetes IS fun), a better choice is to go with the standard distribution which is just fine if you don’t launch hundreds of microservices. However, for large scale projects and critical applications it is recommended to use Kubernetes distributions by third-party vendors to have instant access to support and troubleshooting. What do you think?