Is there a “multi-user” Docker mode, e.g. for scientific clusters?

Question

I want to use Docker for isolating scientific applications for the use in a HPC Unix cluster. Scientific software often has exotic dependencies so isolating them with Docker appears to be a good idea. The programs are to be run as jobs and not as services.

I want to have multiple users use Docker and the users should be isolated from each other. Is this possible?

I performed a local Docker installation and had two users in the docker group. The call to docker images showed the same results for both users.

Further, the jobs should be run under the calling users's UID and not as root.

Is such a setup feasible? Has it been done before? Is this documented anywhere?

Answer 1

Yes there is! It's called Singularity and it was designed with scientific applications and multi user HPCs. More at http://singularity.lbl.gov/

Answer 2

OK, I think there will be more and more solutions pop up for this. I'll try to update the following list in the future:

• udocker for executing Docker containers as users
• Singularity (Kudos to Filo) is another Linux container based solution

Answer 3

Don't forget about DinD (Docker in Docker): jpetazzo/dind

You could dedicate one Docker per user, and within one of those docker containers, the user could launch a job in a docker container.

Answer 4

I'm also interested in this possibility with Docker, for similar reasons. There are a few of problems I can think of:

1. The Docker Daemon runs as root, providing anyone in the docker group with effective host root permissions (e.g. leak permissions by mounting host / dir as root).
2. Multi user Isolation as mentioned
3. Not sure how well this will play with any existing load balancers?

I came across Shifter which may be worth a look an partly solves #1: http://www.nersc.gov/research-and-development/user-defined-images/

Also I know there is discussion to use kernel user namespaces to provide mapping container:root --> host:non-privileged user but I'm not sure if this is happening or not.

Answer 5

There is an officially supported Docker image that allows one to run Docker in Docker (dind), available here: https://hub.docker.com/_/docker/. This way, each user can have their own Docker daemon. First, start the daemon instance:

docker run --privileged --name some-docker -d docker:stable-dins

Note that the --privileged flag is required. Next, connect to that instance from a second container:

docker run --rm --link some-docker:docker docker:edge version