Why is crond failing to run a non-root crontab on alpine linux?

Question

I am having a nasty time running a non-root crontab file on Alpine Linux.

I've been through two other cron related posts and I don't have an answer:

https://askubuntu.com/questions/23009/why-crontab-scripts-are-not-working

https://serverfault.com/questions/449651/why-is-my-crontab-not-working-and-how-can-i-troubleshoot-it

Here is the setup.

My crontab looks like this:

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/bin
SHELL=/bin/bash

* * * * * /opt/monitor/monitor.sh >> /var/log/monitor.log 2>&1
0 3 * * * /opt/monitor/monitor-log-clean.sh >> /var/log/monitor.log 2>&1

My Dockerfile is a little messy now, but only because I have been desperately trying to resolve this. It looks something like this. In short, I add SUID for crontab -e to work as other users, I create my user, I import my crontab file, and then I provide permissions to everything I can think of.

FROM alpine:3.5

# DEPENDENCY TO ALLOW USERS TO RUN crontab -e
RUN apk add --update busybox-suid

# I LIKE BASH
RUN apk --no-cache add bash bash-doc
RUN apk --no-cache add util-linux pciutils usbutils coreutils binutils findutils grep

#... lots of custom stuff ...    

# CREATE USER
RUN adduser -S robuser && \
    mkdir -p /home/robuser

# ADD ENTRY POINT
ADD src/entrypoint.sh /home/robuser/entrypoint.sh

# GIVE MY USER ACCESS
RUN mkdir /etc/cron.d
RUN echo "robuser" > /etc/cron.allow
RUN echo "" >> /etc/cron.allow
RUN chmod -R 644 /etc/cron.d

# ADD MY CRONTAB
RUN mkdir -p /var/spool/cron/crontabs
ADD ./src/crontab.conf /tmp/cloudwatch/crontab.conf
RUN crontab -u robuser /tmp/cloudwatch/crontab.conf

# DEBUG... GIVE MY USER ACCESS TO EVERYTHING
RUN chown -R robuser /etc/cron.d
RUN chmod -R 755 /etc/cron.d
RUN chown -R robuser /var/spool/cron
RUN chmod -R 744 /var/spool/cron
RUN chown robuser /var/spool/cron/crontabs
RUN chmod 744 /var/spool/cron/crontabs
RUN chown -R robuser /etc/crontabs
RUN chmod -R 744 /etc/crontabs
RUN chown robuser /etc/crontabs/robuser
RUN chmod -R 744 /etc/crontabs/robuser
RUN chmod 600 /var/spool/cron/crontabs/robuser

# ADD MY MONITORING PROGRAM
RUN mkdir -p /opt/monitor
ADD src/monitor /opt/monitor
RUN mkdir -p /opt/monitor/.tmp && \
    chown -R robuser /opt/monitor && \
    chmod -R 700 /opt/monitor

RUN touch /var/log/entrypoint.log && \
    touch /var/log/monitor.log && \
    touch /var/log/cron.log && \
    touch /var/log/awslogs.log && \
    chown -R robuser /var/log

USER robuser

ENTRYPOINT /home/robuser/entrypoint.sh

meanwhile, my entrypoint.sh has this somewhere in it. I start the cron daemon as a background service and log to cron.log verbosely. I've also tried specifying -d 0 to get even more debug, but the didn't really add anything to the output.

#!/bin/bash

crond -b -l 0 -L /var/log/cron.log

#... lots of other startup stuff ...

An important point: If I don't switch to robuser, everything works okay as root.

If I check the cron.log, its pretty empty:

crond: crond (busybox 1.25.1) started, log level 0
crond: wakeup dt=45
crond: wakeup dt=60
crond: wakeup dt=60

Meanwhile, /var/log/monitor.log is completely empty (see crontab at the beginning of the post).

So crond is not printing any errors.

I've tried everything i can think of to debug this. There's no error message. It simply runs and never prints. A good suggestion was to simply my crontab.. but this also did not work:

PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/bin
SHELL=/bin/bash

* * * * * touch /tmp/test.txt

I've tried searching for other alpine containers who use non-root cron, but most people don't go through the trouble of getting their alpine containers to run non-root.

Does anyone have any further suggestions to help debug this?

Answer 1

cron itself should run as root, regardless of which user you want to use to run the jobs.

Indeed, when you run:

RUN crontab -u robuser /tmp/cloudwatch/crontab.conf

This will install a crontab for user robuser. When cron executes jobs from this particular crontab, it will automatically switch users to robuser. However, cron can't switch users like that if it's not running as root, which is why you need to be running cron as root.

So, to make cron work here, you'll need to remove this directive from your Dockerfile:

USER robuser

---

Note that you probably won't be out of the woods once you fix this issue: if you're using environment variables to pass AWS credentials to your monitoring scripts (it seems you're using AWS here), this won't work, because cron will remove those prior to switching users. This is largely a security feature in cron to avoid env-variable leakage to unprivileged users.

As an aside: I wrote an open-source crontab runner, Supercronic, specifically designed for container use cases, which fixes that (and you can run it as an unprivileged user just fine). If you get frustrated with regular cron, you could always give a shot.

Answer 2

coded the fix to run crond as non-root, basically crond was implemented in the busybox code base, and it called the function 'change_identity' which was invoking the syscall setgroups (the linux CAP_SETGID capability required commonly), to switch the job privilege into the normal user / group privilege, same as the job of the user, so crond process must be running as root, instead I didn't get any lucks on the docker option --cap-add setgid

I pushed the patched alpine onto docker hub:

geekidea/alpine-cron:3.7
geekidea/alpine-cron:3.8
geekidea/alpine-cron:3.9

A sample dockerfile:

FROM geekidea/alpine-cron:3.9
RUN mkdir /tmp/crontabs \
    && echo 'SHELL=/bin/sh' > /tmp/crontabs/nobody \
    && echo '* * * * * /tmp/nobody.sh' >> /tmp/crontabs/nobody \
    && echo 'echo "$(date) blahblahblah nobody" >> /tmp/nb-cron.log' > /tmp/nobody.sh \
    && chmod 0755 /tmp/nobody.sh \
    && chown -R nobody.nobody /tmp/crontabs/nobody

USER nobody
CMD ["crond", "-c", "/tmp/crontabs", "-l", "0", "-d", "0", "-f"]

see more: https://github.com/inter169/systs/blob/master/alpine/crond/README.md