fig docker monitoring broken container

Question

I have a fig configuration for launch N dockers containers (app, redis, mongo, postgre, etc...)

When I run fig up everything is ok.

      Name      Command               State            Ports           
--------------------------------------------------------------------------
my_mongodb_1   /usr/local/bin/run       Up      28017/tcp, 27017/tcp     
my_redis_1     /usr/local/bin/run       Up      6379/tcp                 
my_pg_1        /usr/local/bin/run       Up      5432/tcp                 
my_app_1       ...                      Up      443->443/tcp, 80->80/tcp 

but for one not important reason one of this containers could be turned off.

      Name      Command               State            Ports           
--------------------------------------------------------------------------
my_mongodb_1   /usr/local/bin/run       Up      28017/tcp, 27017/tcp     
my_redis_1     /usr/local/bin/run       Exit    6379/tcp                 
my_pg_1        /usr/local/bin/run       Up      5432/tcp                 
my_app_1       ...                      Up      443->443/tcp, 80->80/tcp 

Is possible to configurate supervisord for monitoring all containers and start the container which has been turned off

Answer 1

You have to configure each program (container) in different files and them must be into /etc/supervisor/conf.d/ folder, in where the supervisor should look for the programs. In your case I propose:

#It is the /redis.conf
[program:redis]
command= /bin/bash -c "fig up redis"
        "fig logs redis"
directory=/path/of/fig_file
autostart=true
autorestart=true
stdout_logfile=/path/to/log/redis.log
redirect_stderr=true

And for pg:

#It is the /pg.conf
[program:pg]
command= /bin/bash -c "fig up pg"
        "fig logs pg"
directory=/path/of/fig_file
autostart=true
autorestart=true
stdout_logfile=/path/to/log/pg.log
redirect_stderr=true

And the same configuration (mongo.conf and app.conf) for the others program (mongo and app).

When you boot your machine or restart it, each program must be up.

The example above you run the container and you can keep it alive because you fallow the logs of it.

You can check the state of each programs with:

sudo supervisorctl 

And see:

app                          RUNNING    pid 17036, uptime 0:22:28
mongodb                      RUNNING    pid 17018, uptime 0:22:29
pg                           RUNNING    pid 17030, uptime 0:22:28
redis                        RUNNING    pid 17019, uptime 0:22:29

good luck!!

Answer 2

I would use Upstart which I've had pretty good experience from. Supervisord is best when processes don't daemonize.