Microservices: Good for Developers' Mental Health

Let’s face it: developers’ work and responsibilities have increased dramatically in the last 15 years. We develop, run and maintain critical services used by millions of users around the globe 24/7, 365 days a year, and a single service disruption may cause severe business disruption. Remember the last time Gmail went down? Yeah, that. And as apps and APIs change even more frequently, we developers can feel overwhelmed, and can experience anxiety, burnout, stress and fatigue. The COVID new reality hasn't helped. 

So when I heard that we needed to transition our monolithic app to a microservices design, I feared that it meant that we would have even shorter deadlines to meet – with code that we had even less time to test. I did not look forward to breaking the news to my development team. Already, I could hear groans about the mountain of tech debt we would create in order to release a candidate by the deadline. Would it be resilient? Maintainable? Reliable? Observable. Sort of. This is a startup's way of saying, "Yes."

Although my team's journey to microservices got off to a rough start, it would not take long for us to realize the simple but very effective beauty of microservices. Releasing faster and more often did the opposite of increasing our stress: it made our lives much easier by helping us collaborate better and reduce risk in almost every area. Microservices helped us to improve the most important but difficult to measure metric on development teams: developer confidence. High confidence fuels good mental health among developers, which can have dramatic benefits for any business that depends on apps and APIs.

Critical to success in the early stages of our microservices journey was to forge ahead without checking off everything on a boilerplate list of best practices for microservices migrations. Next, I'll go into what we prioritized in the names of speed and sanity as we rushed to accelerate our transformation journey along parallel tracks: tools/automation and people/confidence.

1: Separation of Concerns: Business Continuity

Many developers already know that each microservice should have a single purpose. Keeping concerns separate allows developers to focus on very specific functional areas, reducing the risk of propagating errors that may disrupt a very measurable metric: loss of business continuity due to slow debugging, missed deadlines, and costly rollbacks. By focusing on single-purpose microservices, developers are much more likely to know exactly where to look in case of a functional or business logic bug. Increased confidence in releases leads to ongoing developer confidence.

Consider how more confidently developers may work with a proper separation of concerns in the example of in-app payments. Instead of stressing over whether deploying a bug fix in “Invoicing” will take down an entire system, a microservices developer can deploy the fix with the assurance that only “Invoicing” will be affected. Of course, you can accidentally kill “invoicing” for some time, but the rest of the system will remain safe. I want to be clear that the ritual of major coordinated deployments will stay arduous for changes that are not backward compatible. But those will be increasingly rare events.

2: Separation of Concerns: Speed

Historically, one way to reduce deployment risk has been to go live when the fewest users are online. But with modern apps and web services needing to change as frequently as multiple times per day for optimal user experience and product growth, developers have to find new ways to mitigate risk. When my team moved to microservices, we had to reimagine speed as a means to reduce risk. By deploying small increments in rapid iterations (within a proper separation of concerns), teams no longer need to experience the stress that comes from large-scale deployments with thousands of changes – where anything can go wrong. And if a bug arises in microservices, the impact will most likely be limited to specific areas of the application, avoiding the domino effect.

3: Reduction of Tech Debt

Laziness is a great virtue of software developers because it pushes us to do more with less code. The inevitable side effect is we may take shortcuts. Don’t get me wrong: some shortcuts are inevitable, but they are also often the driving force of tech debt. Let’s patch something together for now and we’ll deal with it later. Days, months, or years later?

But a proper microservices architecture eliminates a whole category of shortcuts. 

For example, using some functions from “user management” to quickly complete your development work for “invoicing” can be tempting, but the shortcut comes at the price of creating an unmanaged, unplanned dependency. If instead “user management” is a single-purpose microservice, it can be exposed via API to any other service that may need it via a reliably rigid contract that must be respected and maintained. Contract testing can be an additional confidence builder for any developer of microservices who wants to avoid tech debt in staging environments. 

Moreover, regardless of how tidy you are or how much contract testing you do, a smaller, compact codebase for a service is generally cleaner from the start as it has to achieve fewer goals. Even when the service is expanded, it does so by small, low-risk increments.

4: Scalability

By now, I would expect you’d never design a system that scales only vertically. But horizontal scalability comes in different flavors. Of course, you can take a big horizontally scalable monolith and replicate it, but you can’t say you have a good grip on how the resources are being used, and whether it’s an efficient setup or not.

If you segment your application in purpose-oriented microservices, you know exactly which services require higher parallelism over others, and you can apply more efficient controls on them. They will spin faster, and they will be easier to design in a way that only have short-lived transient states. 

Auto-scaling in microservices doesn’t look as scary anymore, and it becomes a great safety net, boosting developer confidence. Many “what-ifs” that teams kept in the back of their mind concerning scalability are replaced by a well-defined scaling strategy.

Another plus side is the fine art of handling “backpressure” becomes a fully manageable, measurable aspect of the application. How much should we run in parallel? How much should wait instead, and for how long? It’s a game, and a fun one.

5: Failing Fast, Failing Atomically

A system failure of any sort is always an unpleasant experience. Whether it stems from your code or an external factor such as a connectivity issue, failures happen. Monolithic platforms, sometimes with a heavy state, long-running operations, and massive bootstrap times are generally designed with recovery in mind. Except recovery is difficult, and even when adopting wise patterns such as the actor model, it may lead to uncertain, unknown, or intermediate states. 

Proper microservices are light, and have little to no state, resulting in fewer tasks to absolve and faster bootstrap times. Recovering from failure just isn’t worth it. Instead, the system can detect a failure as soon as possible, deem whether it’s critical, and if it is, gracefully kill the microservice and restart.

6: Testability - Coverage

Complexity is the archenemy of testability. When a function has dependencies and alters an object that cannot be easily created or mocked without a ton of context, well… shall we just say we’ll test it at the next refactoring?

Of course, we would all love to be able to sustain a software made up of just pure functions, but let’s face it: it’s not in every developer’s arsenal, nor possibility. Certainly not in mine, anyway.

Small, single-purpose microservices mitigate the number of situations in which a developer or QA engineer must surrender to poor testing coverage for the simple reason that microservices can’t have that much context. Imagine you need to mock a DAO layer to test your business logic. One thing is needing to mock 10 well-defined methods, all focused on a limited set of domain objects. It's quite another thing to mock thousands of methods.

Bottom line: Microservices make it now much easier to ensure sufficient coverage to keep our team on the right track when it comes to handling compatibility-breaking changes. 

7: Testability - Black Box Testing

Black box testing is about testing a service as a consumer (with a focus on input/output and expected outcomes or functions) without deep-diving into the internals of the system. The size of the black box plays a vital role here because if the black box is huge, then lots of functionalities and interactions will be there, lurking in the dark. The bigger the box, the harder it is for the QA engineer or tester to determine the coverage, and if something goes sideways, it gets super hard to determine what caused the issue.

Black box testing has been crucial to us in exploratory testing to simulate unexpected cases in real user scenarios. Powerful functional and end-to-end API testing is critical to our black box testing across all microservices.

8: Testability - Service Isolation

I often like to think of microservices as organs of a bigger organism. Unlike a real organ, though, a microservice can be extracted from its location and connected to a virtual infrastructure that behaves as if it was the real thing. The microservice should not notice the difference.

This technique, known as service isolation, is how you can deep test and debug a specific microservice. By creating a surrounding that is mostly mocked, you will invariably stabilize an environment, which then won’t cause the issue you’re trying to debug. This ensures that the issue can be diagnosed accurately in your testing subject. But if you can’t reproduce it, then the issue must certainly come from somewhere else.

Implementing a proper service isolation strategy for a whole system is a dynamic goal rather than a hard practice. So take your time, and parallelize your work to reach your goal. But take service isolation very seriously: it’s easy to ignore the opportunity to create a solid service isolation strategy when you are only dealing with a few microservices, but microservices and tech debt can grow very rapidly.

9: Replaceability

I’m leaving this topic for last not because it’s less important, but because it’s a big picture concern that you should still consider at the very beginning of your microservices journey. 

Sometimes, because you want to improve or wish to rely on “as-a-service” products, you may want to replace your current implementation of something with another. With a solid monolith, replaceability tends to be particularly complicated because it is difficult to isolate discrete parts of the stack for one functionality or another. Therefore, unless you’re super diligent, you will certainly find implementation-specific data objects sneaking in from their proper homes to other areas.

On the other hand, a microservices architecture poses physical barriers between functionalities. Certain specific objects will obviously not cross the transport medium. Therefore, when you reimplement and replace a service, your only concern will be maintaining the contract, and if you did a good job, then the rest of the system will be completely unaffected by what happened, even if you hot-swapped it. I bet more development teams moving to microservices will need to prioritize replaceability in order to leverage SaaS services to a greater extent.

10: Being Pragmatic

Building a proper microservices architecture is a lot of work–often more work than building a classic monolith. Frankly, it’s very hard to follow all the rules and principles that engineering textbooks throw at you, especially when you have a product manager breathing down your neck to meet tight deadlines. You will take shortcuts, despite the fact that you swore “not this time”. But it’s the nature of which shortcuts you allow that will make all the difference in the world. So let me share a quite pragmatic approach that served me well during the last year.

11: The Movie Set Approach

With your “microservices movie set,” build your grand design and commit to it. This doesn’t mean each microservice will do exactly what you planned it to do from the get-go. If it’s going to take too long, for a time-sensitive objective, to separate all concerns and flows, just create fatter microservices plus papier-mache microservices (facades) that sit where they should, and all they do is relay data. From a distance, everything will look exactly as planned, except the concerns will not be properly separated yet. This will make it easier to decommission areas of the fatter services later, and move the responsibilities where they belong (as long as the flow and the interconnections are as intended from the beginning).

And when you do, document everything. The problem with tech debt is it’s rarely part of a strategy. Sometimes, debt is a decision made by a lone wolf, other times it’s a collective decision. In both cases, debt can insidiously become just part of the tribal knowledge of “The Stuff We’ll Fix Sooner or Later.”

When you willingly take on tech debt, you already know how it should be done, but you can’t or don’t want to deal with a proper solution right now. The problem is as the memory of the nature of the debt fades, so fades your memory of the ideal design. So, whenever you willingly decide to take on debt, I recommend immediately creating a ticket that aims to solve it, describing how you’re implementing it, why it’s not great, and how it should be implemented.

Conclusion

You’ll never hear me praise a technology or a pattern because it’s trendy. My life is complex enough as it is to get myself through useless challenges, let alone investing in more problems.

So if I praise microservices this much, it really means that microservices really have had a significant impact on improving developers’ quality of work and life. While we only talked about a few key principles and benefits of microservices that we personally and directly experienced, there is much more to explore on the topic of microservices. Ultimately, this article is not about disassembling a perfectly working monolithic software into microservices because microservices are recommended by thought leaders. It’s about radically changing the way you and your development team approach your daily challenges with greater confidence, productivity, and peace of mind.