DIY DevOps, CI, and CD with GitHub, Docker and a VPS

I clearly remember my first Azure invoice. It was for a simple web app and a tiny MySQL server. Not particularly scaled either, just simple stuff. In fact, not only do I remember the invoice, I still have PTSS because of it, and if I my memory still serves me well, I think I said something roughly resembling the following as I got my invoice ...

$%^£#€@*&$^%£*I£@&$^%*^(%$&@£^@ ...!!!!!!!!!

It was a €250 invoice, and it was for one month of usage. I suspect my screams as I opened my email can still be heard on clear days where I live, echoing between the mountain ranges, scaring off tourists with urban legends of Sasquatch and BigFoot ...

Today I work for a small startup as their CTO, and I suspect my CEO would murder me if I told him we needed monthly recurring bills in the above range for serving simple web apps, intended to be used by less than 3 people on a daily basis. Especially considering we'll be needing 5 of these apps over the next 6 months, implying 5 web apps and 5 MySQL databases. In case you don't understand why, multiple the above €250 by 5, at which point you end up with a monthly cost of €1,250, and realise I'm currently running the whole shebang on a €24 per month droplet from DigitalOcean.

Azure or AWS would cost us 52x as much as our current VPS costs!

However, this creates a problem for me, since AWS and Azure gives me all this really great stuff I now have to figure out some sort of DIY process to solve. Since I've already slaughtered Scrum to economise our little company, and I'm now spending way too much time doing (manual) deployments, I figured I'll just have to solve the CI and CD problem, allowing me to automatically create new deployments as I merge/push to my master branch for my Angular frontend. And since I'm in general terms a nice guy, I figured I might as well share my experiences in these regards with the good folks at DZone.

Now, before you continue reading, realise that our main infrastructure is based upon a Magic VPS. You can check out how to deploy such a thing by reading this recipe. However, I suspect the process would be similar enough for most of you to change it in accordance to your own needs, regardless of exactly what these are. Also realise I've (obviously) changed company names, secrets, paths, and URLs, which you'll have to change to match your personal requirements if you're to copy and paste my stuff into your own CI and CD deployment scenario - But if you have some basic Docker knowledge, this shouldn't be too difficult ...

GitHub Actions

First of all, we're using GitHub as our source control repository, and as our backlog/whiteboard. However, to trigger automatic CI and CD builds upon commits, I'll need a GitHub Action/Workflow file, which triggers on all commits towards my master branch. Below is my GitHub Actions Workflow file.

on:
  push:
    branches:
      - master

jobs:

  build_and_push:
    name: Build, push, and deploy Docker image to DO Container Registry
    runs-on: ubuntu-latest
    steps:

    - name: Checkout master
      uses: actions/checkout@master

    - name: Build container image latest version
      run: docker build -t registry.digitalocean.com/acme/app:latest .

    - name: Install doctl
      uses: digitalocean/action-doctl@v2
      with:
        token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}

    - name: Log in to DigitalOcean Container Registry with short-lived credentials
      run: doctl registry login --expiry-seconds 600

    - name: Push image to DigitalOcean Container Registry
      run: docker push registry.digitalocean.com/acme/app:latest

  deploy:
    runs-on: ubuntu-latest
    needs: build_and_push

    steps:
      - name: Deploy to Digital Ocean droplet via SSH action
        uses: appleboy/ssh-action@v0.1.3
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          password: ${{ secrets.PASSWORD }}
          port: 22
          script: |
            cd acme.app
            docker-compose stop
            docker-compose rm -f
            docker-compose pull
            docker-compose up -d

There are basically two jobs in the above GitHub action file, which executes sequentially and does the following.

• build_and_push - Builds my Dockerfile and push it to our DigitalOcean private Docker container registry
• deploy - Uses SSH to tear down and restart a "docker-compose.yml" file I've got on my droplet

FYI; You can get a private Docker container registry from DigitalOcean allowing you to store 50MB at for free if you've got an account with them.

Since I built the whole app entirely using Magic, all I need to do now, is to build my Angular frontend upon commits and merges towards my master branch, and the actual "Dockerfile" referenced above is the default Dockerfile that comes out of the box whenever you scaffold a new Angular frontend using Magic. You can see my entire Dockerfile below.

FROM tiangolo/node-frontend:10 as build-stage
WORKDIR /app
COPY package*.json /app/
RUN npm install
COPY ./ /app/
RUN npm run build -- --output-path=./dist/out

FROM nginx:1.19.6
COPY --from=build-stage /app/dist/out/ /usr/share/nginx/html

COPY --from=build-stage /nginx.conf /etc/nginx/conf.d/default.conf

The above basically builds any Angular project though, however what your Docker file builds or creates, should be irrelevant, as long as it's a valid Docker file, producing a Docker image of some sort. The last parts to this process, is a "docker-compose.yml" file I need to manually copy and paste into my droplet, inside a folder named "acme.app", referenced in the above deploy parts in the GitHub action file. My "docker-compose.yml" file looks like the following.

version: '2'

services:
  app:
    image: registry.digitalocean.com/acme/app:latest
    restart: always
    environment:
      - VIRTUAL_HOST=app.acme.com
      - LETSENCRYPT_HOST=app.acme.com
      - LETSENCRYPT_EMAIL=cto@acme.com

    networks:
      - proxy

networks:
  proxy:
    external:
      name: nginx-proxy

This "docker-compose.yml" file again, needs to be manually copied to your VPS server, and put into a folder called "acme.app".

The reasons why the above file works, is because I have created a shared proxy network with Docker, following this recipe (my recipe for deploying Magic). This ensures installation of cryptography certificates, creates a reverse proxy in nGinx, etc, etc, etc.

Basically, I am running both my Magic backend, Magic frontend, and the (modified) scaffolded Angular app frontend on the same €24 per month VPS Droplet from DigitalOcean. Hence, already at this point, I've reduces my costs from at least €400 per month to €24 per month (2 Angular apps, one .Net 5 app, and one MySQL database). Of course, as time proceeds, this price reduction will only grow further, as we start adding more apps to the mix, since the same VPS can probably easily run 10 to 20 such frontends, and one Magic backend can serve all of our micro services at the moment - However, already I've saved €376 per month, obviously making my CEO smile I would assume. As time proceeds and we start adding more apps to the mix, I suspect we'll be saving more than €1,500 per month, and be left with a monthly invoice of around 25-50 EUROs, instead of an invoice of €1,500 and more per month.

Then I'll need to create two secrets in DigitalOcean under their "API" menu item as follows.

The first secret is for allowing me to authenticate towards my private container repository, and becomes both my username and my password in the next step, which implies I'll need to login to my VPS and execute the following terminal command once.

docker login registry.digitalocean.com

At which point I'm asked for my username and password, where I provide the value of my above "docker_login" secret as both my username and password. This will persist my credentials to my private Docker container repository on my VPS. This ensures that as my "docker-compose.yml" file tries to pull our (private) image, it's authenticated to do so.

The "DIGITALOCEAN_ACCESS_TOKEN" secret above is used in my GitHub Actions file, to allow my GitHub Action to automatically push my container as it's being built, as the last step of my GitHub workflow file.

At this point there is only one thing remaining, which is to provide 3 additional secrets in your GitHub project's "Secrets" settings. This resembles the next screenshot. The reasons for this is because my GitHub Action needs to execute scripts on my VPS server over SSH, at which point it'll need the root username and password, in addition to the IP address. The HOST secret is basically your droplet's IP address, and the username/password parts your root username and password on the droplet. If you want to do this slightly more advanced, you can mess around with a KEY instead of the root user's password, however assuming GitHub secrets are stored securely, this should not really matter that much. If you're super paranoid though, you might want to use an asymmetric KEY instead of your root user's password here.

And voila! I've got CI and CD towards my production environment every time I push or merge anything into my master branch. And since GitHub provides me with an infinite amount of free private repositories, fully fledged with Issues and the whole shebang, I'me ready to rock!

If we are to break down the above process into sequential steps occurring as I commit to my GitHub's master branch, here is what happens in order.

1. My GitHub action triggers
2. The action pulls my code from my master branch
3. My action file executes my Dockerfile, which builds my Angular frontend
4. Then my action file pushes the newly built Docker Angular image to my private DigitalOcean container repository
5. Then my action file executes a script, on my VPS, which tears down and updates my docker-image through a "docker-compose.yml" file I had to manually copy to my VPS

As I explained further up, this is a CI/CD DevOps recipe for Magic, and specifically an Angular frontend, scaffolded using Magic - However, it shouldn't be too difficult to modify it to deploy really anything you want to a droplet, or another VPS too for that matter, if you've got some basic understanding of Docker. Below I am demonstrating the app in its entirety in case you haven't seen it before ...