Using A Windows Gaming PC as a (Linux) Docker Host

Docker Desktop is a perfectly serviceable way to use Docker on either MacOS or Windows, but for non-trivial use cases, it leaves much to be desired.

I recently happened upon one such use case that you might think would be relatively common: I develop on MacOS, but since my MacBook Pro only has 16GB of memory, I’d like to use another host – in this case, my personal Windows gaming computer, which has a whopping 32GB of memory – as a remote Docker host. How many of us hackers, nerds, and geeks out there work with our (usually) MacBook laptops during the day but then flip over to our custom-built Windows boxes after work to blast some aliens in the face?

You’d think this would be pretty well-traveled territory by now and, thus, relatively easy to achieve. But you’d be dead wrong. It turns out that getting Docker working as a local network host on Windows is simple; in fact, it requires a lot of kludgy hodgepodge of hacks to work.

Disclaimer: I am NOT a Windows admin by any stretch of the imagination. I am fully aware that you can do pretty much anything with Windows these days that you can with *nix operating systems in terms of configuration, services, etc., but I fully admit that I do not know the current best practices to do so. 

Also, don’t use this setup in any mission-critical or production situation.

tl;dr Summary

1. Install WSL 2, then install Docker as a daemon inside that.
2. Configure said daemon to listen on 0.0.0.0:2375.
3. Realize that WSL will force it to bind to localhost, not 0.0.0.0, no
   matter what you do.
4. Wire up a TCP proxy running in Windows userspace to bind to 0.0.0.0:2375
   and use that to shuttle traffic to localhost:2375.
5. Test connectivity from an external host to verify it works.

Docker Desktop: What NOT to Do

Docker Desktop includes a simple way to make your Windows machine a Docker host (or so it would seem): a simple check box in the configuration. You’d think this is all you need, but there’s a “gotcha” here: it will only bind Docker to localhost:2375, NOT to 0.0.0.0:2375, meaning that using this option will only make Docker available over the local machine. It’ll be unreachable from anywhere else on your network using this option.

Side note: Docker Desktop includes this functionality to make current versions of the daemon available to legacy docker CLI clients. Still, for our purposes, it’s pretty much useless.

At first, I didn’t realize this. I used the aforementioned checkbox and tried to reach Docker from another machine on my local network, to no avail. Wondering what I’d done wrong, I looked at the Windows Resource Monitor. I found that the Docker Desktop application was listening on port 2375, but it was bound to localhost only, so it couldn’t receive traffic from anywhere else on my network.

Next, I did what any sane person would do: I asked “The Google”™. This article on Docker configuration this article on Docker configuration that implied I could control this behavior by modifying the JSON configuration exposed in the Docker Desktop app. So naturally, I added the following to the Docker configuration:

{  "hosts": ["tcp://0.0.0.0:2375"]
}

Then, I disabled my firewall in both Windows and my antivirus software just to rule it out as a possibility in case that didn’t work. When I restarted Docker, I checked the Network tab of the Resource Monitor again and looked for what was bound to port 2375. Yep, it's Docker Desktop, all right, but it's still bound ONLY to localhost! Even with the right configuration, according to the documentation, Docker Desktop on Windows just refuses to bind to all network interfaces, making Docker Desktop essentially useless for all but the most trivial of use cases.

Using Windows Subsystem for Linux

After a few more trial-and-error attempts to get the above working failed, out of frustration, I decided to nuke the entire Docker installation from within Windows and re-install just the daemon inside WSL. Once I had that installed and configured, I reasoned, I could expose the daemon from within WSL to the network and be able to get some work done finally.

How wrong, how hopelessly wrong I was.

For the uninitiated, “WSL” stands for “Windows Subsystem for Linux” and is possibly the most amazing thing Microsoft has ever built into their flagship product. The first version of WSL added hooks/shims to the Windows kernel directly so that when a complied Linux binary made syscalls to what it thought was the Linux kernel, the Windows kernel accepted those syscalls and redirected or re-implemented the logic that was needed in “The Windows Way” such that any Linux binary running had no idea it was running on Windows. Think virtualization without needing a hypervisor, or another OS installed.

As with anything, this was less than perfect. Over time, issues that Microsoft eventually decided to address with WSL version 2 began implementing an extremely lightweight, totally transparent virtual machine within Windows that allowed for the same features, just implemented differently. This has some tradeoffs vs. the previous version, but it is, for the most part, the recommended way to go these days.

You’ll need Windows 10, preferably the most recent build version, to use WSL. You may also require the Pro edition or better (I’m not sure if the Home edition is allowed to run WSL or not). You can read more about how to install it here.

Installing the daemon under Linux is straightforward enough. Docker provides excellent documentation on the install process and a great post-install guide, both of which I followed for an Ubuntu 20.04 WSL 2 install. That was simple enough; however, because WSL starts up a little differently than a normal Linux distro, you don’t have a working systemd, so you’ll need to figure out another way to auto-start the Docker daemon.

Once everything’s installed and configured, you can launch the docker daemon manually by running dockerd as root.

However, if you review the Network tab of the Windows Resource Monitor, you won’t see dockerd anywhere. Instead, that’ll be under the process wslhost, which will be bound only to local loopback. No matter what you do, WSL will not allow you to expose processes inside of it to the network directly. We’re still stuck with local loopback only.

So, more magic will be needed to get things working. How about a TCP proxy?

Sticking a Proxy in Front: An Exercise in Madness

At this point, I realized there was no way I would ever get Docker exposed to the entire network directly, so I resolved to put a TCP proxy in front of it. “As long as it’s just blindly shuttling traffic back and forth,” I reasoned, “the docker CLI won’t even know the difference.”

So, what proxy should we put in place? I had experience with HAproxy, but it looks like they don’t have a build for Windows. I needed something in the Windows user space, so that was out.

The other proxy I was familiar with was Envoy, so I looked at their website to see if they had a Windows build available. It turns out that the “documentation” (such as it is) for running Envoy on Windows tells you to compile it from source (at the time this happened; hopefully, this changes in the future), and I had neither the time nor patience, to install a massive Windows development environment just to build Envoy once. That seemed kind of insane.

After some more looking around, I found their FAQ, linked to this document, as the place to download binaries. Only one problem: they’re all Docker images! So I’d need Docker to run Docker?!

That made zero sense, so I looked at the project’s GitHub releases — no luck. So, I started looking at the project’s “issues” to see if anyone else had asked for some help in getting a pre-compiled Windows release, hoping maybe they got a useful answer.

Unfortunately, the most useful response on the issue of providing pre-built Windows binaries for Envoy was this response from one of the maintainers:

See envoyproxy. We don’t provide binaries, but you can copy them out of the docker container.

The thread does go on, and there are links provided to documentation and a tool for downloading Envoy on Linux machines. Still, any mention of Windows is entirely missing from any of the documentation that’s been linked (as of the time of this writing), making the responses in that thread entirely pointless.

And with that, I was off to go rip apart a Docker image.

Doing Ungodly Things With Docker Images

In my opinion, using Windows for development and/or terminal-based work/workflows is still a pain. It’s just not a smooth experience, not to mention I’d have to re-install about a bajillion different things and configure a ton more just to get a decent workflow set up.

So, I proceeded to grab the official Envoy Docker Image for Windows from Docker Hub while on my Mac (using Docker desktop, which was eating over 80% of my memory at the time just to pull images):

$ docker pull envoyproxy/envoy-windows
Using default tag: latest
Error response from daemon: manifest for envoyproxy/envoy-windows:latest not found: manifest unknown: manifest unknown

All right, fine, Docker, be difficult. Let’s just force the most recently published version as the tag:

$ docker pull envoyproxy/envoy-windows:v1.18.2
v1.18.2: Pulling from envoyproxy/envoy-windows
4612f6d0b889: Downloading
5ff1512f88ec: Downloading
ecaafce5c67e: Pulling fs layer
d507bef22a1e: Waiting
9bdb36426121: Waiting
e5b3f220cc68: Waiting
e5410c30e7a9: Waiting
4b47829da3b1: Waiting
35eb096aecb4: Waiting
68a2a4ec685d: Waiting
9c1c360fc8b3: Waiting
a08a596feca8: Waiting
image operating system "windows" cannot be used on this platform

…aaaaaand, nope. Docker won’t even let you pull the image if there’s a mismatch between the host and container operating systems.

After cursing the gods, I decided I wouldn’t give up. I remembered that Docker images are just big tarballs, so I should be able to download the flat tarball image from Docker Hub and, in theory, unpack it somewhere to browse the filesystem of what would otherwise be a Windows image.

As it turns out, this isn’t so easy. Many custom headers and tokens are involved in retrieving a Docker image, and it’s not easy to do manually. Luckily, I stumbled on this script from the Moby project that did exactly what I needed! I retrieved the script and ran it to fetch the Windows Envoy image:

$ ./script.sh win envoyproxy/envoy-windows:v1.18.2
Downloading 'envoyproxy/envoy-windows:v1.18.2@v1.18.2' (12 layers)...
<snip>
Download of images into 'win' complete.
Use something like the following to load the result into a Docker daemon:
  tar -cC 'win' . | docker load

Note: If you get an error message complaining that ‘mapfile’ is not found, you may need to upgrade your version of Bash. MacOS ships with an old version that’s two major releases behind. Alternatively, you could run this on your WSL machine, which may already have an updated version of Bash available.

Now, we can browse the win/ directory that this script created for us:

$ ls win
Permissions Size User Date Modified Name
drwxr-xr-x     - jah   4 May 15:42  1a5606bb70dc20fb56456b2583d295bdd2e847c0617da3da68d152bdd6a10b78
drwxr-xr-x     - jah   4 May 15:42  4e6cb5497aca4d83d2b91ef129fa823c225b0c76cefd88f5a96dd6c0fccdd6c7
drwxr-xr-x     - jah   4 May 15:42  6bfb8784732bcc28ef5c20996dbe6f15d3a004bf241ba59575b8af65de0a0aaf
drwxr-xr-x     - jah   4 May 15:42  3712aa599c08d0fb31285318af13e44d988391806d2460167643340c4f3a7123
drwxr-xr-x     - jah   4 May 15:42  698765937dc05ffcc458d8c2653563450bc169a724c62ed6a2c58f23c054b0ff
drwxr-xr-x     - jah   4 May 15:42  a4c3f3e7cef6cd7492338a26b7b307c0cd26e29379655f681d402c1eeaf595b6
drwxr-xr-x     - jah   4 May 15:42  b93d56fb00e644574bb7c2df769bb383d7fa351730393d46239078026bbc8efc
.rw-r--r--  3.7k jah   4 May 15:42  b775d72f61762e116864ab49adc8de32045e001efd1565c7ed3afe984d6e07f0.json
drwxr-xr-x     - jah   4 May 15:42  c42480d1b057b159309c4e55553ba75d84c21dc6c870f7ed77b0744c72e755f5
.rw-r--r--  3.7k jah   4 May 15:40  d00ba7ba582355550f5e42f453d99450754df890dec22fc86adb2520f3f81da2.json
drwxr-xr-x     - jah   4 May 15:42  d59df72a9d52b10ca049b2b6b1ce5b94f6ebb8a100ec71cea71ec7d8c0369383
drwxr-xr-x     - jah   4 May 15:43  d8067d34f431844ea7a3068d31cdb9254f1fcb93bcaf1c182ceebdec17c8d1fc
drwxr-xr-x     - jah   4 May 15:42  ea8955ac8603cc8dbb34e70e0922b59271522839db7d626e0f79f45b954c0d12
drwxr-xr-x     - jah   4 May 15:42  ec233e633fbbcbaf9d6f7ba3496ebc676f9b70ac4b95ba1127c466723976f55a
.rw-r--r--  1.2k jah   4 May 15:43  manifest.json
.rw-r--r--    51 jah   4 May 15:43  repositories

After poking around each directory one by one and decompressing layer.tar that was found in each, I eventually found what I was looking for:

$ cd 3712aa599c08d0fb31285318af13e44d988391806d2460167643340c4f3a7123
$ tar xf layer.tar && chmod -R 777 * && tree .
tree .
.
├── Files
│   ├── Documents\ and\ Settings -> [Error\ reading\ symbolic\ link\ information]
│   └── Program\ Files
│       └── envoy
│           └── envoy.exe
├── Hives
│   ├── DefaultUser_Delta
│   ├── Sam_Delta
│   ├── Security_Delta
│   ├── Software_Delta
│   └── System_Delta
├── VERSION
├── json
└── layer.tar
4 directories, 10 files

Finally! We found envoy.exe!

After copying envoy.exe out of the container tarball, I could now write a configuration file to proxy TCP traffic from 0.0.0.0:2375 to local loopback on the same port. That config file wound up looking about like this:

static_resources:
  listeners:
    - name: docker-proxy
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 2375
          protocol: TCP
      filter_chains:
        filters:
          - name: envoy.filters.network.tcp_proxy
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
              cluster: docker-actual
              stat_prefix: docker-proxy
  clusters:
    - name: docker-actual
      connect_timeout: 1s
      type: STATIC
      load_assignment:
        cluster_name: docker-actual
        endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: 172.25.65.236
                    port_value: 2375
                    protocol: TCP
admin:
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 9901

Important note: Every time WSL launches, it assigns a different IPv4 address to itself. One thing I haven’t figured out just yet is how to get it to always use the same IP address. You’ll need to get your WSL IP address by running ifconfig eth0 from within a WSL shell and substitute your IP for the one in my example configuration. You’ll need to do this every time you reboot and/or every time you stop and start WSL.

Start It Up

With Envoy finally ready to go, we can start everything up and test it. On my Windows machine, I launched a shell into WSL and ran the Docker daemon manually:

$ sudo dockerd --tls=false &

Then, in another terminal window, I launched a Powershell session and ran Envoy:

PS J:\envoy> .\envoy.exe -c .\envoy.yaml

If all is working as it should, you should be able to see that wslhost is bound to the local loopback on port 2375 and that envoy.exe is bound to “IPv4 unspecified” on 2375 as well:

Now I was finally able to use a computer with some significant hardware as my Docker host. All that remained was to export the environment variable and use it!

$ export DOCKER_HOST=<ip of windows box>
$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Success!

Closing Thoughts

In case it wasn’t obvious, what I did here should NEVER be done in a production situation. This was purely for my own use on my own network for development and testing purposes only.

The setup described herein relies quite a bit on manual work. Every time you reboot the host, you’ll need to dig up the internal IPv4 address for WSL, change that entry in the Envoy configuration, start up dockerd within WSL, and launch envoy.exe again to connect everything. Right now, I’m doing this by hand since I don’t have to reboot that Windows machine all too often. That said, I plan on finding ways to automate the launch of that stuff every time I boot the computer up. I believe that can be accomplished with a BAT file that you call through a Windows “Scheduled Task” (see Task Scheduler app) every time you start the computer prior to login.

This process was way harder than it needed to be. From the Windows Docker Desktop application refusing to honor the host's configuration array to the Envoy project claiming they have a Windows release of their proxy but making it so incredibly hard to retrieve to the way Microsoft refuses to expose processes bound to ports in WSL to the network by default, this was a major pain in the posterior. It’s fair to criticize all three companies involved here in how hard they’ve made it just to do what should otherwise be a very simple thing. It’s also fair to say that none of this would be possible without the fine job the people at these aforementioned places have done in building excellent tools that allow people like myself to accomplish such feats of “Mad Science” as this.

However, I will admit that removing the Docker Desktop application from Windows wasn't necessary
since the daemon we installed within WSL wound up being bound only to local loopback anyway. In theory, at least, you should be able to do all of this without using WSL at all (minus retrieving the Envoy Docker image, you need a recent-ish version of bash for that). The important words above are “in theoare “in theory” – I haven’t tested this with the Docker Desktop app for Windows and have no idea what additional headaches one might run into should they try to use it.

None of this would have been necessary, however, if either:

• Apple shipped their default MacBook Pro laptops with more than 16GB of RAM or;
• Docker didn’t require an entire VM with dedicated resources to “fake” the notion of containers on MacOS.

But we don’t live in a perfect world.