right way to run some code with timeout in Python

Question

I looked online and found some SO discussing and ActiveState recipes for running some code with a timeout. It looks there are some common approaches:

• Use threa

Answer 1

I found this with eventlet library:

http://eventlet.net/doc/modules/timeout.html

from eventlet.timeout import Timeout

timeout = Timeout(seconds, exception)
try:
    ... # execution here is limited by timeout
finally:
    timeout.cancel()

Answer 2

What you might be looking for is the multiprocessing module. If subprocess is too heavy, then this may not suit your needs either.

import time
import multiprocessing

def do_this_other_thing_that_may_take_too_long(duration):
    time.sleep(duration)
    return 'done after sleeping {0} seconds.'.format(duration)

pool = multiprocessing.Pool(1)
print 'starting....'
res = pool.apply_async(do_this_other_thing_that_may_take_too_long, [8])
for timeout in range(1, 10):
    try:
        print '{0}: {1}'.format(duration, res.get(timeout))
    except multiprocessing.TimeoutError:
        print '{0}: timed out'.format(duration) 

print 'end'

Answer 3

If it's network related you could try:

import socket
socket.setdefaulttimeout(number)

Answer 4

If you're running code that you expect to die after a set time, then you should write it properly so that there aren't any negative effects on shutdown, no matter if its a thread or a subprocess. A command pattern with undo would be useful here.

So, it really depends on what the thread is doing when you kill it. If its just crunching numbers who cares if you kill it. If its interacting with the filesystem and you kill it , then maybe you should really rethink your strategy.

What is supported in Python when it comes to threads? Daemon threads and joins. Why does python let the main thread exit if you've joined a daemon while its still active? Because its understood that someone using daemon threads will (hopefully) write the code in a way that it wont matter when that thread dies. Giving a timeout to a join and then letting main die, and thus taking any daemon threads with it, is perfectly acceptable in this context.

Answer 5

solving with the 'with' construct and merging solution from -

• Timeout function if it takes too long to finish

• this thread which work better.

  import threading, time
  
  class Exception_TIMEOUT(Exception):
      pass
  
  class linwintimeout:
  
      def __init__(self, f, seconds=1.0, error_message='Timeout'):
          self.seconds = seconds
          self.thread = threading.Thread(target=f)
          self.thread.daemon = True
          self.error_message = error_message
  
      def handle_timeout(self):
          raise Exception_TIMEOUT(self.error_message)
  
      def __enter__(self):
          try:
              self.thread.start()
              self.thread.join(self.seconds)
          except Exception, te:
              raise te
  
      def __exit__(self, type, value, traceback):
          if self.thread.is_alive():
              return self.handle_timeout()
  
  def function():
      while True:
          print "keep printing ...", time.sleep(1)
  
  try:
      with linwintimeout(function, seconds=5.0, error_message='exceeded timeout of %s seconds' % 5.0):
          pass
  except Exception_TIMEOUT, e:
      print "  attention !! execeeded timeout, giving up ... %s " % e
  

Answer 6

A completely general solution to this really, honestly does not exist. You have to use the right solution for a given domain.

• If you want timeouts for code you fully control, you have to write it to cooperate. Such code has to be able to break up into little chunks in some way, as in an event-driven system. You can also do this by threading if you can ensure nothing will hold a lock too long, but handling locks right is actually pretty hard.

• If you want timeouts because you're afraid code is out of control (for example, if you're afraid the user will ask your calculator to compute 9**(9**9)), you need to run it in another process. This is the only easy way to sufficiently isolate it. Running it in your event system or even a different thread will not be enough. It is also possible to break things up into little chunks similar to the other solution, but requires very careful handling and usually isn't worth it; in any event, that doesn't allow you to do the same exact thing as just running the Python code.