Running cron python jobs within docker

Question

I would like to run a python cron job inside of a docker container in detached mode. My set-up is below:

My python script is test.py

#!/usr/bin/env pyt         


        

Answer 1

Single Container Method

You may run crond within the same container that is doing something closely related using a base image that handles PID 0 well, like phusion/baseimage.

Specialized Container Method

May be cleaner would be to have another Container linked to it that just runs crond. For example:

Dockerfile

 FROM busybox
 ADD crontab /var/spool/cron/crontabs/www-data
 CMD crond -f

crontab

 * * * * * echo $USER

Then run:

 $ docker build -t cron .
 $ docker run --rm --link something cron

Note: In this case it'll run the job as www-data. Cannot just mount the crontab file as volume because it needs to be owned by root with only write access for root, else crond will run nothing. Also you'll have to run crond as root.

Answer 2

Adding crontab fragments in /etc/cron.d/ instead of using root's crontab might be preferable.

This would:

• Let you add additional cron jobs by adding them to that folder.
• Save you a few layers.
• Emulate how Debian distros do it for their own packages.

Observe that the format of those files is a bit different from a crontab entry. Here's a sample from the Debian php package:

# /etc/cron.d/php5: crontab fragment for php5
#  This purges session files older than X, where X is defined in seconds
#  as the largest value of session.gc_maxlifetime from all your php.ini
#  files, or 24 minutes if not defined.  See /usr/lib/php5/maxlifetime

# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime)

Overall, from experience, running cron in a container does work very well (besides cron logging leaving a lot to be desired).

Answer 3

Here is my checklist for debugging cron python scripts in docker:

1. Make sure you run cron command somewhere. Cron doesn't start automatically. You can run it from a Dockerfile using RUN or CMD or add it to a startup script for the container. In case you use CMD you may consider using cron -f flag which keeps cron in the foreground and won't let container die. However, I prefer using tail -f on logfiles.
2. Store environment variables in /etc/envoronment. Run this from a bash startscript: printenv > /etc/environment. This is an absolute must if you use environment variables inside of python scripts. Cron doesn't know anything about the environment variables by default. By it can read them from /etc/environment.
3. Test Cron by using the following config:

* * * * * echo "Cron works" >>/home/code/test.log
* * * * * bash -c "/usr/local/bin/python3 /home/code/test.py >>/home/code/test.log 2>/home/code/test.log"

The python test file should contain some print statements or something else that displays that the script is running. 2>/home/code/test.log will also log errors. Otherwise, you won't see errors at all and will continue guessing.

Once done, go to the container, using docker exec -it <container_name> bash and check:

1. That crontab config is in place using crontab -l
2. Monitor logs using tail -f /home/code/test.log

I have spent hours and days figuring out all of those problems. I hope this helps someone to avoid this.

Answer 4

Here's an alternative solution.

in Dockerfile

ADD docker/cron/my-cron /etc/cron.d/my-cron
RUN chmod 0644 /etc/cron.d/my-cron

ADD docker/cron/entrypoint.sh /etc/entrypoint.sh

ENTRYPOINT ["/bin/sh", "/etc/entrypoint.sh"]

in entrypoint.sh

 #!/usr/bin/env bash
  printenv | cat - /etc/cron.d/my-cron > ~/my-cron.tmp \
    && mv ~/my-cron.tmp /etc/cron.d/my-cron

cron -f

Answer 5

We are using below solution. It supports both docker logs functionality and ability to hang the cron process in the container on PID 1 (if you use tail -f workarounds provided above - if cron crashes, docker will not follow restart policy):

cron.sh:

#!/usr/bin/env bash

printenv | cat - /etc/cron.d/cron-jobs > ~/crontab.tmp \
    && mv ~/crontab.tmp /etc/cron.d/cron-jobs

chmod 644 /etc/cron.d/cron-jobs

tail -f /var/log/cron.log &

cron -f

Dockerfile:

RUN apt-get install --no-install-recommends -y -q cron 

ADD cron.sh /usr/bin/cron.sh
RUN chmod +x /usr/bin/cron.sh

ADD ./crontab /etc/cron.d/cron-jobs
RUN chmod 0644 /etc/cron.d/cron-jobs

RUN touch /var/log/cron.log

ENTRYPOINT ["/bin/sh", "/usr/bin/cron.sh"]

crontab:

* * * * * root <cmd> >> /var/log/cron.log 2>&1

And please don't forget to add the creepy new line in your crontab

Answer 6

Here is a complement on rosksw answer.

There is no need to do some string replacement in the crontab file in order to pass environment variables to the cron jobs.

It is simpler to store the environment variables in a file when running the contrainer, then load them from this file at each cron execution. I found the tip here.

In the dockerfile:

CMD mkdir -p /data/log && env > /root/env.txt && crond -n

In the crontab file:

* * * * * root env - `cat /root/env.txt` my-script.sh