python-multithreading

I am using ThreadPoolExecutor class from the concurrent.futures package def some_func(arg): # does some heavy lifting # outputs some results from concurrent.futures import ThreadPoolExecutor with ThreadPoolExecutor(max_workers=1) as executor: for arg in range(10000000): future = executor.submit(some_func, arg) but I need to limit the queue size somehow, as I don't want millions of futures to be created at once, is there a simple way to do it or should I stick to queue.Queue and threading package to accomplish this? Python's ThreadPoolExecutor doesn't have the feature you're looking for, but

I would try to post a minimal working example, but unfortunately this problem just requires a lot of pieces so I have stripped it down best I can. First of all, I'm using a simple script that simulates pressing keys through a function call. This is tweaked from here . import ctypes SendInput = ctypes.windll.user32.SendInput PUL = ctypes.POINTER(ctypes.c_ulong) class KeyBdInput(ctypes.Structure): _fields_ = [("wVk", ctypes.c_ushort), ("wScan", ctypes.c_ushort), ("dwFlags", ctypes.c_ulong), ("time", ctypes.c_ulong), ("dwExtraInfo", PUL)] class HardwareInput(ctypes.Structure): _fields_ = [("uMsg"

I need to program the execution of a give method every x minutes. I found two ways to do it: the first is using the sched module, and the second is using Threading.Timer . First method : import sched, time s = sched.scheduler(time.time, time.sleep) def do_something(sc): print "Doing stuff..." # do your stuff sc.enter(60, 1, do_something, (sc,)) s.enter(60, 1, do_something, (s,)) s.run() The second : import threading def do_something(sc): print "Doing stuff..." # do your stuff t = threading.Timer(0.5,do_something).start() do_something(sc) What's the difference and if there is one better than

It is unclear how to properly timeout workers of joblib's Parallel in python. Others have had similar questions here , here , here and here . In my example I am utilizing a pool of 50 joblib workers with threading backend. Parallel Call (threading): output = Parallel(n_jobs=50, backend = 'threading') (delayed(get_output)(INPUT) for INPUT in list) Here, Parallel hangs without errors as soon as len(list) <= n_jobs but only when n_jobs => -1 . In order to circumvent this issue, people give instructions on how to create a timeout decorator to the Parallel function ( get_output(INPUT) ) in the

I have been trying to create a tkinter top level window that streams video form webcam and do a QR scan. I got this QR scan code from SO and another code that just updates images from webcam instead of streaming the video on a tkinter label. and i tried to combine these both so that a toplevel window with a label updating image from webcam and a close button to close the toplevel window. And while it streams the images, it can scan for QR code and if a scan is successful, the webcam and the toplevel window gets closed. here is what i tried. import cv2 import cv2.cv as cv import numpy import

When calling a linux binary which takes a relatively long time through Python's subprocess module, does this release the GIL? I want to parallelise some code which calls a binary program from the command line. Is it better to use threads (through threading and a multiprocessing.pool.ThreadPool ) or multiprocessing ? My assumption is that if subprocess releases the GIL then choosing the threading option is better. When calling a linux binary which takes a relatively long time through Python's subprocess module, does this release the GIL? Yes, it releases the Global Interpreter Lock (GIL) in the