Running python script with cron only if not running

Question

I need to run a python script (job.py) every minute. This script must not be started if it is already running. Its execution time can be between 10 seconds and several hours.

So I put into my crontab:

* * * * * root cd /home/lorenzo/cron && python -u job.py 1>> /var/log/job/log 2>> /var/log/job/err

To avoid starting the script when it is already running, I use flock().

This is the script (job.py):

import fcntl
import time
import sys

def doIncrediblyImportantThings ():
    for i in range (100):
        sys.stdout.write ('[%s] %d.\n' % (time.strftime ('%c'), i) )
        time.sleep (1)

if __name__ == '__main__':
    f = open ('lock', 'w')
    try: fcntl.lockf (f, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except:
        sys.stderr.write ('[%s] Script already running.\n' % time.strftime ('%c') )
        sys.exit (-1)
    doIncrediblyImportantThings ()

This approach seems to work.

Is there anything I am missing? Are there any troubles I can run into using this approach?

Are there more advised or "proper" ways of achieving this behaviour?

I thank you for any suggestion.

Answer 1

The only suggestion I would make is to make your exception handling a little more specific. You don't want to accidentally delete the fcntl import one day and hide the NameError that results. Always try to catch the most specific exception you want to handle. In this case, I suggest something like:

import errno

try:
    fcntl.lock(...)
except IOError, e:
    if e.errno == errno.EAGAIN:
        sys.stderr.write(...)
        sys.exit(-1)
    raise

This way, any other cause of the lock being unobtainable shows up (probably in your email since you're using cron) and you can decide if it's something for an administrator to look at, another case for the program to handle, or something else.

Answer 2

You're in trouble when the machine reboots or freezes with the script running (and thus an active lock). Simple way to counter this is to use the @reboot cron timestamp to run rm /path/to/lock.

Answer 3

I ran into this exact problem last week, and although I did find some good solutions, I decided to make a very simple and clean python package and uploaded it to PyPI.

Install with: pip install quicklock

Using it is extremely simple:

[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> # Let's create a lock so that only one instance of a script will run
...
>>> singleton('hello world')
>>>
>>> # Let's try to do that again, this should fail
...
>>> singleton('hello world')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nate/live/gallery/env/lib/python2.7/site-packages/quicklock/quicklock.py", line 47, in singleton
    raise RuntimeError('Resource <{}> is currently locked by <Process {}: "{}">'.format(resource, other_process.pid, other_process.name()))
RuntimeError: Resource <hello world> is currently locked by <Process 24801: "python">
>>>
>>> # But if we quit this process, we release the lock automatically
...
>>> ^D
[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> singleton('hello world')
>>>
>>> # No exception was thrown, we own 'hello world'!

Take a look: https://pypi.python.org/pypi/quicklock

Answer 4

You can use The Fat Controller which is a daemon that will restart a script x seconds after the previous instance finished, so you can never have overlapping instances of the same script.

You can even tune it to start an instance immediately afterwards if a certain condition is met.

(I'm afraid the website is a bit basic, but the project is stable and is running on at last several websites that I know of. I will make a nice, pretty website once I get v0.0.3 out the door!)