Docker, on macOS, DIY-Style

I've always been the kind of person to do-it-myself. I designed my own computer games in grade school, built my own servers in high school, and I'm even in the process of writing my own programming language.

So when it came time to start doing some serious day-to-day work with Docker, on my Macbook, I started looking into the nuts and bolts of running a Linux container platform on the decidedly non-Linux Darwin kernel.

The simple fact is, you can't. You cannot run Linux containers on Darwin. That's not how this works. You can only run Linux containers on Linux kernels.

The easiest way to get up and running is to use something like Docker for Mac. It takes care of everything for you, spinning up a custom Linux virtual machine using macOS's lightweight Hyperkit / xhy.ve virtualization framework. Docker for Mac handles all of the wiring for you, so that when you execute docker images in your macOS terminal, your client contacts the docker daemon executing inside the Linux VM.

Neat.

Waving aside some of the many reported performance issues with Docker for Mac + Hyperkit, there's the simple fact that I already run a Linux Vagrant instance which I use for other things. I'd really rather not incur the cost of running two of them.

As it turns out, the wiring to get a macOS user-space docker client hooked up to a Linux kernel-space docker daemon is pretty straightforward.

1. Bind Docker On All Interfaces

My Vagrant box is Ubuntu-based (Xenial, 16.04 LTS), and Docker is configured out of the box to only bind the UNIX domain socket, for zero-conf communication with local docker clients only. Luckily, we can change that.

Xenial also means systemd, and these steps should apply nicely to Bionic (18.04 LTS). First, we need to edit the docker systemd .service file:

$ sudo vim /lib/systemd/system/docker.service

The line we're interested in is ExecStart=...; my service file looks like this:

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
Type=notify
ExecStart=/usr/bin/docker daemon -H fd:// -H tcp://0.0.0.0:9098
harbor.escalon.cf-app.com
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity

[Install]
WantedBy=multi-user.target

The additional -H flag will cause the docker daemon to bind another interface. Here, I've chosen TCP port 9098, on all interfaces.

Once the .service file is all fixed up, we need to inform systemd, via a daemon-reload (to pick up on the fact that the file changed) and a restart (to actually make the changes real):

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

To verify, you can either grep the process table:

$ ps -ef | grep docker
root 1935 1 0 13:05 ? 00:00:02 /usr/bin/docker daemon -H fd:// -H tcp://0.0.0.0:9098

... or check netstat:

$ sudo netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address  Foreign Address  State   PID/Program name
tcp        0      0 0.0.0.0:22     0.0.0.0:*        LISTEN  1937/sshd
tcp6       0      0 :::9098        :::*             LISTEN  1935/docker
tcp6       0      0 :::22          :::*             LISTEN  1937/sshd

Good to go!

2. Port-forward Like There's No Tomorrow

Next up, we need to be able to access the TCP port that docker is listening on inside the Linux VM, from outside the VM.

If you're running Vagrant like I am, you can just add this to your Vagrantfile:

Vagrant.configure('2') do |config|
  config.vm.box = 'jhunt/vagabond'
  config.vm.box_version = '1.0.2'
  config.vm.synced_folder ".", "/vagrant"

  for i in 9000...9099
    config.vm.network :forwarded_port, guest: i, host: i
  end
end

For these changes to take effect, you will have to restart your Vagrant instance. I find that port-forwarding a whole block (the 90xx block) makes my life easier later on down the road.

3. Set Your .${SHELL}rc

The last piece of the puzzle is to direct the docker client on the Mac to use the forwarded port, instead of a local UNIX socket. The DOCKER_HOST environment variable governs that, so I added this to my ~/.bashrc:

export DOCKER_HOST=tcp://127.0.0.1:9098

Now (after sourcing ~/.bashrc, or opening a new terminal), I can use docker on my mac!

$ uname -s
Darwin

$ docker images
REPOSITORY  TAG     IMAGE ID      CREATED       SIZE
nginx       latest  e2c463314119  2 weeks ago   0B
ubuntu      latest  c6f8c325f4ca  8 weeks ago   0B
alpine      latest  189d5ae0f1aa  4 months ago  0B

A Parting Note

I've been using this setup for a week or so now, and it works really well. Most of the time, I forget I'm even bouncing through the Linux VM, it's such a seamless experience.

One thing I have noticed is that docker images (as shown above) does not report the image sizes appropriately. I'm not sure why this is, but other than being an oddity, it hasn't caused much grief.

Happy Hacking!

James (@iamjameshunt) works on the Internet, spends his weekends developing new and interesting bits of software and his nights trying to make sense of research papers.

Currently working on Bolo.