Create Versatile Microservices in Golang — Part 10 (Summary)

Wow, what a journey it has been! These posts took months to finish. The first post started in late 2017, we're halfway through 2018, and I'm just writing the final post.

It took a lot of effort and hard work to get here. It was a fairly daunting task, first because my own understanding was sketchy in parts. In fact, part of the reason I began writing these was to cement my own understanding and to get feedback on my approach from the community, and it couldn't have gone any better in that respect. I've learned so, so much and I've spoken to some awesome, super smart people throughout who have helped me so much. Second, the sheer amount of content I had to write to cover each stage of writing a microservice, to deploying it, covering all of those different technologies, databases, service providers, and best practices was one of the hardest things outside of work I've ever embarked upon.

But in spite of all that, I learned more through this process than I ever imagined, and I'd like to thank all the emails of support I got from people. I counted over 50 emails over the months, asking when the next part would be up, thanking me, and complimenting me. It really helped me to finish them, so thank you to everyone who emailed me, and I'm sorry it took ages for me to respond in some cases.

I'd like to thank Asim of Micro for all the time he has spent answering my dumb questions, and for that, I've gifted these tutorials to the official examples/tutorials repo for Micro in the hopes that they can help others adopt Micro.

I'd like to thank Ke Ding for his work in translating these articles into Chinese. As you may or may not know, the Go community is HUGE in China, and growing, so I really appreciate this.

If you're reading this and you speak another language, and you'd like to translate these posts into that language, just e-mail me and I'll link them in the original post. I had someone email me about translating them into Russian some months ago, but I can't find the email now, so if you're reading this, please hit me up! Russian would be awesome!

What Did I Learn and What Went Wrong?

I learned a lot about the technologies we covered, but I also learned a lot about technical writing and blogging. The biggest lesson I learned was to keep it simple, don't throw too many concepts into one article. If not for the readers' sakes, for your own, too. There were times throughout the series, where I felt as though I was juggling too much at once. I'd created several services, and occasionally, I'd make a mistake, which I'd have to correct or alter in several places. Which was tedious for me, and probably also tedious for the reader.

The biggest technical problem I faced, was not being able to find a sane approach to having a mono-repo. I had a few good suggestions from the community, but it felt like I'd be spending a lot of time making that work, so I had to make a call half-way through to split out into separate repos. I felt as though I lost some readership at that point, it's a rift in the series I don't like. It was an obvious admissions of ignorance, and at some point, I'd like to re-address that, either by making the first articles single repos, or figuring out mono-repos once and for all maybe.

I also lost the branches for each part in the series at this stage, which means people can't checkout the branch at the later stages and see the progression. I found Go's vendoring annoying to reason about when it came to having various branches, and it became too much of a cognitive strain to maintain. But it still irks me that I lost that useful feature for those following these tutorials.

Another problem was that as my understanding grew, I had to go back and alter some of the first code segments, and things drifted slightly as new posts were made, using my newly cemented understanding. Luckily, I had people from the community, via email, etc, point out some of those mistakes and I think we're now largely in-line. But, if you do spot any inconsistencies, please let me know.

A final problem is that I feel as though I rushed over some subjects slightly. Some posts were very short and snappy and covered the bare minimum. Unfortunately, because I was covering so much ground, if I went into great detail on each technology, I was worried I'd never finish. But, please, if you do want anything elaborated on, drop me an email, follow me on Patreon, and I'll try to accommodate if there's enough demand for it.

How Did They Do?

According to Google Analytics, the posts got just shy of 50,000 views (as of April 12, 2018), which I'm pretty happy with. You'll also have noticed the Patreon spam I started adding on every post. I have one subscriber currently, to whom I'm very thankful. If you value these posts, then please do show some love; it'll help me to prioritize this work over other work if it has some backing.

Final Thoughts on Microservices

Although I hugely see the benefit of microservices, I'm also highly cautious of adopting something wholesale just because it's a nice idea. As with most things, there's a time and a place. The start-up I helped build from the ground-up uses microservices; it went from being a monolith application to a set of microservices, shaped entirely by the processes and patterns described in this series.

A part of my reasoning for writing this series was to share what I'd learn in that process of splitting apart a start-up into microservices and to get a better understanding from the community, and to sanity check my approach.

Did we optimize for microservices too soon? Quite possibly — our monolith was doing fine, but it had gone through a complete pivot, was littered with code that was no longer used, or was written for the previous iteration of our company, but survived the transition due to some inherent usefulness. My point was that this monolith had gotten unwieldy, and I found it difficult to reason about. Our start-up was also very much in an exploratory phase; we were looking into features and business cases we weren't entirely sure would make the cut or not yet.

I felt that we were adding more and more to our monolith, and only some of it was getting used, and it was becoming a hefty maintenance job. So, although it was possibly premature for us to adopt microservices, I felt like it would allow us to prototype better, to spin up services and features, and just drop them again if they didn't get used. So although it was a huge upfront effort to move to this pattern, it did allow us to scale our ideas and move very quickly, and the code we were producing was much cleaner and easier to reason about, too. We also found we were better with writing tests — it wasn't as tricky as when everything was in a monolith. So although it was possibly too soon as a start-up to adopt them, I'm very glad we did, in the end.

Many people will tell you that if you're a start-up, you should steer clear of microservices, as it'll take too much time that could be spent on creating features. Well, I think that's only partially true. It's better to get things correct early on so you have better foundations to build on, and I think it's easier to write new features in a microservice world. The premise to that argument is that it's a lot of effort and hard work to set up microservices. If that was completely true, then I'd 100% agree with that argument, but I think the tooling and the concepts are becoming simpler and better understood, and that it shouldn't really take a huge amount of time and effort creating and maintaining microservices in the world we now live in.

Kubernetes comes out of the box, pre-configured and set-up in most cloud providers nowadays. The information is all there, and the friction in this approach is becoming smoothed over. AWS even offers tools such as Fargate, which handles most of the hard work for you.

Don't be put off if your start-up has multiple features, multiple avenues of exploration — do consider the benefits of this approach. Don't be spooked away by naysayers — it doesn't have to be difficult these days. But also, if you're happy with your well-built monolith, then don't feel the pressure to adopt a new approach.

Ultimately, if it ain't broke, then don't fix it!

So What's Next?

First of all, I'll be rounding up some of the feedback I've not yet actioned, and improving the earlier parts in the series, I won't just be forgetting about these now they're done. I'll be routinely going back through them, updating parts that have changed, or improving on parts where my own understanding has become more rigorous.

I'm feeling pretty good about completing this series, so I want to ride that wave and immediately embark on the next series. I've been thinking for months about what might be a good topic to cover and I've decided on serverless. Again, Golang, of course. But this time on AWS Lambda. I'm also going to focus more on creating a complete product, rather than just arbitrary code examples.

I'll be creating a fake, but functional start-up using AWS serverless technologies.

Beyond this, I've been considering moving this content away from my personal blog and onto some sort of dedicated learning Go site, which is tricky as it would require its own branding and marketing, but something for me to consider. Again, any feedback on these ideas greatly welcome.

Thanks again for all your support and feedback!

Final List of Articles

1. Getting started, gRPC

2. Docker compose and datastores

3. Authentication and JWT

4. Event brokering

5. Web Clients

6. Terraform

7. Kubernetes

8. CircleCI

Sponsor me on Patreon to support more content like this.