Monorepo’s for Microservices Architecture

As enterprises modify and optimize their architecture for the customer-centric digital commerce boom, many enterprises lack the institutional knowledge necessary to quickly make the transformation. Here at commercetools, we have built a cloud-first API only commerce architecture that most companies wish to emulate within their own ecosystem. Thus, our customers frequently ask us to help guide them in their transformation.

This leaves us in a difficult position. Developers are always trying to find the balance between extremely opinionated systems and being forced to reinvent the wheel. commercetools strives to allow developers total flexibility when leveraging its API, from programming language, to SDK options, to choosing REST or GraphQL. The unopinionated nature of commercetools leaves developers with the responsibility of choosing frameworks, cloud architecture, and configuring all the tooling necessary in between.

In order to simplify the development process, commercetools has released its Blueprint Architecture. It gives companies the ability to launch a ready made, proven architecture and provide developers a modern streamlined workflow that drastically improves the development experience.

While those of us who have been in this industry for decades may remember programming in a single language, on a single platform, and releasing the changes ourselves manually to a physical server, those days are long gone. There is now a long list of items and decisions required for any large project:

1. Source Control – Git is the clear leader, the real decision is where to host the remote repository, GitHub, GitLab, Azure, AWS CodeCommit, etc.
2. CI/CD Pipeline – Will you choose Jenkins, Travis, CircleCI, GitLab CI, or a cloud-specific option. More importantly, who will code this pipeline as it will act as the glue for your development from ensuring code quality, building your application, to deploying to your test, stage, and production environments.
3. Cloud Architecture – Cloud Functions, Elastic Beanstalk, Virtual Servers, or the growing Kubernetes. You will need to decide where to host your code and configure the CI/CD pipeline to for that choice.
4. Programming Language – Java has a plethora of developers, but would Node.js or Kotlin make for a better experience?
5. Frameworks and Libraries – Taking Node.js as an example, will you use the popular Express, Koa, or a compact framework like Micro?

The blueprint itself evaluates and makes the decisions providing an opinionated groundwork to build custom logic and code. It can accelerate the start of a project cutting two months off the initial timeline. If you want to see the solution you are welcome to download the whitepaper (here); rather than repeat that content here, I’d like to call out an often unconsidered option that can make a huge impact.

Monorepo

Traditionally, each software project is given its own repository. This maintains a clear separation of concerns, issues, and versions. When building monolithic applications this is typically the preferred approach. Amusingly for monoliths you don’t want a monorepo.

If you choose to adopt microservices, a monorepo becomes an incredible tool with many advantages.

• Ease of code reuse – Similar functionality can be abstracted into shared libraries and directly included by projects.

• Atomic commits – With separate repositories, releases need to sync versions to ensure compatibility, this becomes increasing difficult as projects grow, quickly leading to dependency hell. Monorepos allow developers to change multiple projects atomically, completely removing this issue.

• Code Refactoring – Developers will have access to the entire project, meaning refactors can extend throughout the code.

• Code Quality – Coding standards can be configured and enforced for all the code within the entire project. Linter configurations can be shared and tools like Jenkins can validate all the changes before they are merged.

It’s not surprising that Google, Facebook, Microsoft, Uber, Twitter, and many other enterprise companies choose to use very large monorepos to manage their code. Definitely check out the awesome-monorepo repository over at GitHub.

We’ve chosen Lerna to manage the blueprint monorepo. From Lerna’s homepage – Lerna is a tool that optimizes the workflow around managing multi-package repositories with git and npm.

If you want to leverage Lerna with your own monorepo the steps are very straight forward. Start by installing Lerna, creating a new folder, and initializing the project:

sudo npm install --global lerna

mkdir new-monorepo

cd new-monorepo

git init

lerna init

If you do not want to install Lerna globally, you can use npm's new npx command.

mkdir new-monorepo

cd new-monorepo

git init

npx lerna init

This will give you the following:

Now that you have a Lerna monorepo project, you can start to create packages. With a microservices architecture, each package will represent a single service to be deployed. For each package, you will create a new directory inside of packages and run npm init for each.

cd packages

mkdir default-service

cd default-service

npm init -y

cd ../

We’ll make a couple more…

mkdir product-service

cd product-service

npm init -y

cd ../

mkdir cart-service

cd cart-service

npm init -y

cd ../

At this point the folder structure is:

Say we wanted all three projects to depend on the npm package micro, we can simplify the process by running:

lerna add micro 

This will give each package it’s own version of micro, allowing it to be removed or updated on a per package basis. You could also use the --hoistoption to install micro at the root and simply reference it from each package:

lerna add micro --hoist 

One of the benefits of monorepos is code reuse. Let’s say we want cart-service to depend on default-service, we can simply run:

lerna add default-service –-scope=cart-service 

As you can see, Lerna makes managing your services and their dependencies much easier. Additional commands, such as lerna bootstrap for installing dependencies, lerna publish for creating releases, and lerna diff, will speed up your workflow.

Finally, you can leverage the lerna run command as part of your CI/CD pipeline to execute npm scripts in each of your individual packages. Adding these to your base package.json makes it easy to test, lint, build, push, and deploy your entire project.

{

"name": "root",

"private": true,

"devDependencies": {

	"lerna": "^3.10.7"

	},

"scripts": {

	"test": "lerna run test",

	"helm-lint-all": "lerna run ci-helm-lint",

	"build-all": "lerna run ci-package",

	"push-all": "lerna run ci-push",

	"deploy-all": "lerna run ci-deploy"

	}

}

Leveraging the CLI allows you complete control of each step on a package-by-package basis. You can leverage maven for your java services:

"test": "mvn -B -q clean test -Dorg.mockito.lock.singleton=true" 

Or the Vue CLI for an SPA fronte-nd:

"test:unit": "vue-cli-service test:unit",

"test:e2e": "vue-cli-service test:e2e"

Summary

Monorepos with Lerna are just one tool leveraged in a modern development workflow. It’s important to make the best choices when completing each step of a modern architecture.

While this article only discussed a single portion, the blueprint explains a complete approach to creating a performant architecture and changing the entire way a business operates. It provides a framework to get up and running fast and can guide anyone who requires a custom solution. Taking the points discussed you’ll see four major changes:

• Constant Deployment – no longer do you wait for change on semi-annual deploy schedules. Automated testing and automated, zero downtime deployments are how new functionality as well as fixes are pushed to production.
• Fully Elastic Cloud Environments – when capacity is needed, it is ready. No planning for peak or best-guessing how many servers are needed at the busiest minute of the busiest day. The environment responds automatically to the need for more capacity and shrinks when that capacity is no longer needed.
• Ecosystem of APIs and Events – The same API-first architecture that drives Amazon, Facebook, Spotify, and every other modern customer-centric platform is there to drive your business forward.
• Rapid, Low-Cost Development – New applications can be spun up in a matter of days and weeks instead of the heavy deployments of the past.

Businesses wanting to transform into an omnichannel, consumer centric enterprise need an architecture which can support it. The commercetools Blueprint Architecture allows the enterprise to rapidly transform into the modern digital enterprise that is winning in today’s digital economy.

For more information about the solutions commercetools offers and who we work with, please visit our Resources page.