Deploying With Docker

Glyph wrote about an interesting way to deploy Python applications to Docker. I wanted to try it out, and exercise ncolony a little. The result is NaNoAuto, which is still very preliminary. It was mostly written as a way to test out deploying to Docker and exercise ncolony. Several notes that I learned the hard way:

• On Ubuntu, all docker client commands need to run as root to access the docker container’s UNIX domain socket. I think there is a way to run it on a host/port and ask for a username/password, but I have not figured that out.
• Glyph’s scripts need a recent version of Docker. The built-in version for Ubuntu is too old — make sure to install docker from Docker’s apt repository.
• The entry point you give Docker must not exit — and so must not daemonize. Glyph’s app used an explicit reactor running it (well, even worse — Klein’s implicit reactor). Civilized people, of course, use ‘twistd’. When building things like a civilized person, please remember to pass the “–nodaemonize” flag (AKA “-n”) to twistd.
• On my personal TODO list is to add a hash checker to the runner, so that the wheels installed from the internet can have their hashes matched. It is possible that “peep” could play a role here.
• The scripts are divided into “build-base” and “do-build”. “build-base” is the long-running but fairly stable part — it only changes if you change the basic platform (OS, Python interpreter, needed non-Pythonic parts). The “do-build” part is fairly fast, and needs to be run for every change to a Python file.

Note that this still means a little bit of overhead on top of building a new wheel for each change to a Python file. It should be possible to write something on top of “docker exec” that will copy Python files directly from source to container so as to facilitate even quicker edit/debug cycles.