DIY Kubernetes-based platform building – part 2

In a series of 3 posts , we’ll take a look at how Kubernetes fits into the broader technology landscape, and how an enterprise container platform is crucial for digital transformation and the adoption of cloud-native.

Looking back at part 1

In the first part of this series, we talked about Enterprise Container Platforms, a complete suite for running, operating, and managing container-based applications at scale. Including computing hosts, container runtime, storage, networking, security, metrics, logging, tracing, security, testing, building, and CI/CD tools. Some would say that building a Kubernetes platform is just as complex as building a complete private IaaS infrastructure.

We also looked at the role of Kubernetes in an Enterprise Container Platform and concluded that it is critically important for running and scheduling modern applications, but we also see the limited functional role it plays in the entirety of an Enterprise Container Platform.

In this second part, we’ll look at what to expect from a Container Platform and try to point out that Kubernetes only scratches the surface of what a Container Platform needs.

What to expect from a Container Platform?

A container platform has to have many other functionalities to be successfully be adopted in the enterprise. We’ll list a few:

  1. Observability tools, to inspect and observe running applications in production to increase performance, reliability, and security, and reduce cost, outages, and errors. Observability includes tools for monitoring (metrics), logging, and (distributed) tracing, as well as dashboarding and alerting systems
  2. Stateful (block) storage to store application and configuration data, container images, including backup software
  3. State and configuration for package and artifact management, including Kubernetes configuration and GitOps-based workflows
  4. Security and compliance using Policy enforcement to provide audit logs, set and enforce compliance and security policies. This includes Single Sign-On and identity providers.
  5. CI/CD pipelines: to move code from the repository into production safely, quickly, and often
  6. Networking and service configuration to automate service topology, secure inter-service communication with mTLS, and ingress traffic, including load balancing and SSL termination
  7. Developer onboarding: Preferably automatically onboard development / DevOps team onto the platform, providing them with their own space and all required tools
  8. Developer Self Service: Next to running application workloads, a container platform should also be able to act as an Internal Developer Platform. Self-service makes developers more productive and lowers the pressure on Ops.

While this list may not be complete, we can quickly see that Kubernetes only scratches the surface of what an Enterprise Container Platform needs.

Kubernetes is hot, but does it have intrinsic business value?

And this is where the shoe pinches: Kubernetes is only one of a collection of complex technologies. On its own, Kubernetes is already a difficult technology to master, but many have succeeded, either with their internal Kubernetes deployments, or creating some kind of managed or hosted Kubernetes-as-a-Service: Kubernetes has become ubiquitous and commodity over the last year or two, and many, from public cloud providers, technology vendors, and service providers, have jumped on the bandwagon and released their own service or product. These offerings range from a managed ‘cluster-as-a-service’, a ‘distribution’ to easily install Kubernetes on your own bare-metal infrastructure or entire Platforms as a Service, which includes all the aforementioned ecosystem products for application development.

The reason we see so many managed or hosted Kubernetes services pop up, is because Kubernetes, inherently, is a complex piece of technology. Its inner workings are very visible and customizable, allowing for high levels of customizability to various use cases, but leading to an extremely high learning curve. All these offerings solve these complex problems, so you don’t have to. And these offerings exist for a reason. For the vast majority of use cases, DIY isn’t viable.

For many end-user organizations, it makes no sense to put in the effort of creating an internal service or product. They simply do not operate at the required scale or level of complexity to warrant a do-it-yourself approach. Unfortunately, many still do take this approach, like they DIY’ed on-prem, VM-based infrastructure in the past. The reasoning is often that DIY is required, because of the specific mix of products in the Enterprise Container Platform for things like storage, networking, security, monitoring (and more): you simply need to do it yourself because your company and its use of that specific mix of products is unique.

We’ve seen many organizations struggle with home-grown Kubernetes solutions, delivering below-par features, stability, self-serviceability, performance, and cost-to-value. Kubernetes’ complexity requires dedicated (and expensive) expertise, which is justified to find if you don’t operate at a high scale or level of complexity, and only have a single, internal, customer. We’ve seen the sunk cost fallacy effect happen for many that went down this path, only to discover DIY wasn’t the solution they needed.

The reality is, that the building blocks of cloud-native are just that: generic, common, ubiquitous building blocks. Even if your specific mix of products is unique. it doesn’t mean you should also DIY the building blocks themselves.

Kubernetes is merely one building block in a collection of building blocks needed to create digital services and products. It has no intrinsic business value in and of itself. Even an Enterprise Container Platform has no intrinsic business value. It’s the time and toil saved that has value: the fact that it allows software and cloud engineers to fully focus, with lower cognitive load, that they don’t need to switch context a lot, and can just focus on the task at hand makes them more efficient and productive. That’s the business value an Enterprise Container Platform enables.

So, an Enterprise Container Platform has value as a business enabler. But it only delivers business value if you make the most of the potential value: you have to enable the value, and make the most out of Kubernetes and the Enterprise Container Platform in order to achieve any kind of business value. In other words, you have to overcome the friction due to the complex nature of these technologies, before you start to see a return on investment. That means it makes sense to minimize the friction required to be successful with the implementation and operationalization of an Enterprise Container Platform, which usually means: do as little yourself as possible. If it’s not your core business, don’t do it yourself, but let the experts take care of it. These experts are, of course, the public cloud and technology vendors that do operate at the scale and level of complexity required to successfully ‘do’ Kubernetes. After all, it’s their core business (and not yours).

What to expect in the last part of this series?

In the 3rd part of this series, we’ll discuss Container Platform deployment models. We’ll take a closer look at the Do It Yourself (DIY) platform building approach, Managed Services, and using off-the-shelf products (COTS)

Latest Articles

Navigating the Evolution: Trends and Transformations in Kubernetes Platforms for 2024

Navigating the Evolution: Trends and Transformations in Kubernetes Platforms for 2024

As we look ahead to 2024, the excitement around building and managing container and Kubernetes platforms is shifting to a more realistic outlook. Companies are realizing that these tasks are more complex than originally thought. In the bigger picture, we can expect things to come together and simplify in the coming year. Let's break it down.

Read more
Mastering Dockerfile USER

Mastering Dockerfile USER

Mastering Dockerfile USER: The Key to seamless Kubernetes Deployment

Read more