Stop a running Docker container by sending SIGTERM

Question

I have a very very simple Go app listening on port 8080

http.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(200)
    w.He         


        

Answer 1

A SIGTERM is propagated by the docker run command to the Docker daemon by default but it will not take effect unless the signal is specifically handled in main process being run by Docker.

The first process you run in a container will have PID 1 in that containers context. This is treated as a special process by the linux kernel. It will not be sent a signal unless the process has a handler installed for that signal. It is also PID 1's job to forward signals onto other child processes.

docker run and other commands are API clients for the Remote API hosted by the docker daemon. The docker daemon runs as a seperate process and is the parent for the commands you run inside a container context. This means that there is no direct sending of signals between run and the daemon, in the standard unix manner.

The docker run and docker attach command have a --sig-proxy flag that defaults signal proxying to true. You can turn this off if you want.

docker exec does not proxy signals.

In a Dockerfile, be careful to use the "exec form" when specifying CMD and ENTRYPOINT defaults if you don't want sh to become the PID 1 process (Kevin Burke):

CMD ["executable", "param1", "param2"]

Signal Handling Go Example

Using the sample Go code here: https://gobyexample.com/signals

Run both a regular process that doesn't handle signals and the Go daemon that traps signals and put them in the background. I'm using sleep as it's easy and doesn't handle "daemon" signals.

$ docker run busybox sleep 6000 &
$ docker run gosignal &

With a ps tool that has a "tree" view, you can see the two distinct process trees. One for the docker run process under sshd. The other for the actual container processes, under docker daemon.

$ pstree -p
init(1)-+-VBoxService(1287)
        |-docker(1356)---docker-containe(1369)-+-docker-containe(1511)---gitlab-ci-multi(1520)
        |                                      |-docker-containe(4069)---sleep(4078)
        |                                      `-docker-containe(4638)---main(4649)
        `-sshd(1307)---sshd(1565)---sshd(1567)---sh(1568)-+-docker(4060)
                                                          |-docker(4632)
                                                          `-pstree(4671)

The details of docker hosts processes:

$ ps -ef | grep "docker r\|sleep\|main"
docker    4060  1568  0 02:57 pts/0    00:00:00 docker run busybox sleep 6000
root      4078  4069  0 02:58 ?        00:00:00 sleep 6000
docker    4632  1568  0 03:10 pts/0    00:00:00 docker run gosignal
root      4649  4638  0 03:10 ?        00:00:00 /main

Killing

I can't kill the docker run busybox sleep command:

$ kill 4060
$ ps -ef | grep 4060
docker    4060  1568  0 02:57 pts/0    00:00:00 docker run busybox sleep 6000

I can kill the docker run gosignal command that has the trap handler:

$ kill 4632
$ 
terminated
exiting

[2]+  Done                       docker run gosignal

Signals via docker exec

If I docker exec a new sleep process in the already running sleep container, I can send an ctrl-c and interrupt the docker exec itself, but that doesn't forward to the actual process:

$ docker exec 30b6652cfc04 sleep 600
^C
$ docker exec 30b6652cfc04 ps -ef
PID   USER     TIME   COMMAND
    1 root       0:00 sleep 6000   <- original
   97 root       0:00 sleep 600    <- execed still running
  102 root       0:00 ps -ef

TL;DR

Any process you run with docker must handle signals itself.

or use the --init flag to run tini as PID 1

Answer 2

A very comprehensive description of this problem and the solutions can be found here: https://vsupalov.com/docker-compose-stop-slow

In my case, my app expects to receive SIGTERM signal for graceful shutdown didn't receive it because the process started by a bash script which called from a dockerfile in this form: ENTRYPOINT ["/path/to/script.sh"]

so the script didn't propagate the SIGTERM to the app. The solution was to use exec from the script run the command that starts the app: e.g. exec java -jar ...

Answer 3

So there are two factors at play here:

1) If you specify a string for an entrypoint, like this:

ENTRYPOINT /go/bin/myapp

Docker runs the script with /bin/sh -c 'command'. This intermediate script gets the SIGTERM, but doesn't send it to the running server app.

To avoid the intermediate layer, specify your entrypoint as an array of strings.

ENTRYPOINT ["/go/bin/myapp"]

2) I built the app I was trying to run with the following string:

docker build -t first-app .

This tagged the container with the name first-app. Unfortunately when I tried to rebuild/rerun the container I ran:

docker build .

Which didn't overwrite the tag, so my changes weren't being applied.

Once I did both of those things, I was able to kill the process with ctrl+c, and bring down the running container.