Little's Law and Lots of Kubernetes

For several years now, platform evangelists have been making a case for PaaS tools being great for developer experience. PaaS tools typically accomplish this by effecting change in three areas of developer experience ― Throughput, Efficiency, and Productivity. In this article, a solid grounding for this claim is established by means of theory and the acumen of active practitioners.

Little’s Law

Little's Law states: 

Throughput = Work-In-Progress ÷ Cycle-Time

Little's Law provides a straightforward approach to assessing the efficiency of any queuing system. It helps determine the number (average) of items within a stationary queue and is dependent on the items already present in the queue plus additional ones arriving at its head.

What concerns us is the application of Little's Law to the work that has to be done by engineers on a software engineering team. Originally, this law was formulated from a combination of operations research and queuing theory; it has immense application in software engineering.

Consider any organization having several teams of software developers, each working on separate portions of the company's code. They follow the DevOps methodology and release to production frequently. Unfortunately, due to the nature of software, code contributed by engineers runs into reliability and latency issues when in production.  

Growing Queues, Applying Little’s Law

These issues, along with new features, now constitute the pipeline of developers in the organization. As more code is fed to production by developers, the quantum of software to be maintained by them monotonically increases. Add to this - infrastructure, monitoring, security, and other aspects due to the "left shift" culture being prevalent. Developers are now responsible for writing and maintaining a large volume of code unrelated to the application. This results in engineering resources being diverted away from where it is most valuable.

Enter Kubernetes!

The introduction of Kubernetes has really driven up the complexity, making developers more involved with its setup, maintenance, and teardown. This additional complexity has resulted in weaning them further away from tasks related to application development. Instead, it has developers working with infrastructure and Kubernetes configuration. This quote from the folks at Garden.io sums it up quite nicely ― On average, a cloud developer only spends 11% of their time actually writing code. Instead, about 3x as much time is spent working on boilerplate, plumbing, debugging pipelines, waiting, and writing "code about code" that has little to do with their business.

Daniel Bryant, writing for InfoQ, reports, Although Kubernetes does not provide a full platform-as-a-service (PaaS)-like an experience out of the box, the combination of a well-defined API, clear abstractions, and comprehensive extension points make this a perfect foundational component on which to build upon. These aspects have made the developer community very receptive toward Kubernetes. However, in reality ― access control, logging, monitoring, networking, ingress, storage, backups, and secret management are required in order to make Kubernetes clusters production-ready.

Technology for Technologies’ Sake?

Several people have echoed the sentiment about Kubernetes not being a comprehensive platform by itself. From the Platform.sh blog ― Kubernetes itself is not a platform; it’s a framework within which you can build one. It is well known that Kubernetes is a single piece of a much larger platform required to run services on production. The complexity of Kubernetes often comes across as a massive barrier to entry for what is a well-engineered piece of technology that has well-defined contracts between its components. Kubernetes’ success comes from being one of the most flexible container orchestration tools. Introduced at a time when software engineering teams were struggling with managing containers at scale, it is now one of the most popular open-source projects in the world. 

The sense of urgency is low for large-scale enterprise software engineering teams. They have sufficient resources and bandwidth to back up their efforts to build the right platform for their software teams. It isn’t for fast-moving startups, either. They have golden paths configured for production, which will serve them for the present. Instead, it is the hundreds of thousands of mid-tier companies that have the most pressing need. These teams have grown beyond the early phases of cobbling scripts together and haven’t reached the levels of investment of larger teams. 

Time Saved Is Money Earned

Developer time, or the man-month (or the more gender-appropriate variant person-month), is the most valuable resource of a software engineering team. It can also be expressed naively by simply multiplying the unit cost by the total number of hours worked by engineers. To formulate this in terms of calculus would be to integrate the cost of employing engineers as a function of time within a time range. These approaches break down when a deeper analysis is made of the varied nature of work that an engineer is required to do. For example - communication within the team is a vital part of an engineer’s work. This includes communication within the team, with external stakeholders, and communication on social media. There are also unplanned interruptions from past work, such as bug triage, technical debt, and other consultative work. 

The recent shift to paradigms, such as Infrastructure-as-code, and container management-as-code, was spearheaded by the “Shift Left” movement. This has caused developers to ration their time between writing (and maintaining) code that is responsible for various things in addition to contributing to the core application. This eats further into the critical resource; developer time. 

Kubernetes embraces all of these paradigms. By nature, Kubernetes espouses declarative syntax, which is heavily reliant on manually programmed states that are mimicked by running systems. The usage of automation along the cloud native CI/CD pipeline, from development stages to production, requires that all the parts are managed using code. This adds up to a significant overhead for developers, who have to write them and keep them updated. 

(*or the more gender-appropriate variant person-month)

For developers, this results in further reducing the time available for application development. This will bubble up the org in disastrous ways, namely slower product development cycles, which means a sluggish go-to-market strategy, which results in a loss of competitive advantage, ultimately bringing businesses to a grinding halt.

Is PaaS A Good Answer?

Technology also needs to solve these problems too. It does in the form of PaaS tools ― specifically opinionated, friction-free, golden paths to production. Opinionated frameworks have succeeded because they effectively abstract complexity surrounding the underlying layers resulting in minimizing the amount of time developers have to spend on tasks other than application development. The use of PaaS can typically boost developer velocity significantly, resulting in faster and more frequent releases ― which is the proven metric for productivity for software engineering teams. PaaS tools can also help alleviate engineering teams that have built script spaghetti around their deployment processes.

The downside of the PaaS approach is that the needs of a software engineering team will have to really resonate with the design of the PaaS. Failing to do so will result in a poor developer experience. 

A growing number of tools are creating an impact in the PaaS ecosystem around Kubernetes. Some of the oldest examples are: RedHat OpenShift, Cloud Foundry Korifi, and Google kf. More recently, a handful of entrants, such as D2IQ, KubeFirst, Qovery, Napptive, and Acorn, have emerged in the space. There are also a few tools that have embraced a git-first approach, calling themselves GitOps. A couple of examples are Weaveworks and Gimlet. 

If you’re working with Kubernetes yourself, I would encourage you to make use of any of the PaaS tools listed above. Working on Internal Developer Platforms is a good alternative too, but comes with associated costs that may be non-trivial. Fine-tuning PaaS tools to fit well into developer workflows can ensure the improvement of productivity metrics.