python-multithreading

The right way to limit maximum number of threads running at once?

孤街浪徒 提交于 2019-11-26 07:26:52
问题 I\'d like to create a program that runs multiple light threads, but limits itself to a constant, predefined number of concurrent running tasks, like this (but with no risk of race condition): import threading def f(arg): global running running += 1 print(\"Spawned a thread. running=%s, arg=%s\" % (running, arg)) for i in range(100000): pass running -= 1 print(\"Done\") running = 0 while True: if running < 8: arg = get_task() threading.Thread(target=f, args=[arg]).start() What\'s the safest

what is the use of join() in python threading

蹲街弑〆低调 提交于 2019-11-26 02:40:00
问题 I was studying the python threading and came across join(). The author told that if thread is in daemon mode then i need to use join() so that thread can finish itself before main thread terminates. but I have also seen him using t.join() even though t was not daemon example code is this import threading import time import logging logging.basicConfig(level=logging.DEBUG, format=\'(%(threadName)-10s) %(message)s\', ) def daemon(): logging.debug(\'Starting\') time.sleep(2) logging.debug(\

thread starts running before calling Thread.start

孤街浪徒 提交于 2019-11-26 02:15:01
问题 t1=threading.Thread(target=self.read()) print \"something\" t2=threading.Thread(target=self.runChecks(), args=(self)) self.read runs indefinitely, so the program won\'t ever reach the print line. How is this possible without calling t1.start() ? (Even if I call that, it shold start running and go on to the next line, shouldn\'t it?) 回答1: You're passing the result of self.read to the target argument of Thread. Thread expects to be passed a function to call, so just remove the parentheses and

Are lists thread-safe?

别等时光非礼了梦想. 提交于 2019-11-26 01:35:45
问题 I notice that it is often suggested to use queues with multiple threads, instead of lists and .pop() . Is this because lists are not thread-safe, or for some other reason? 回答1: Lists themselves are thread-safe. In CPython the GIL protects against concurrent accesses to them, and other implementations take care to use a fine-grained lock or a synchronized datatype for their list implementations. However, while lists themselves can't go corrupt by attempts to concurrently access, the lists's

Daemon Threads Explanation

*爱你&永不变心* 提交于 2019-11-26 00:45:26
问题 In the Python documentation it says: A thread can be flagged as a \"daemon thread\". The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. Does anyone have a clearer explanation of what that means or a practical example showing where you would set threads as daemonic ? Clarify it for me: so the only situation you wouldn\'t set threads as daemonic , is when you want them to continue

Python: execute cat subprocess in parallel

妖精的绣舞 提交于 2019-11-26 00:29:27
问题 I am running several cat | zgrep commands on a remote server and gathering their output individually for further processing: class MainProcessor(mp.Process): def __init__(self, peaks_array): super(MainProcessor, self).__init__() self.peaks_array = peaks_array def run(self): for peak_arr in self.peaks_array: peak_processor = PeakProcessor(peak_arr) peak_processor.start() class PeakProcessor(mp.Process): def __init__(self, peak_arr): super(PeakProcessor, self).__init__() self.peak_arr = peak

Python 3 Timed Input [duplicate]

假如想象 提交于 2019-11-25 23:37:53
问题 This question already has answers here : Keyboard input with timeout? (11 answers) Closed 3 months ago . What I would like to be able to do is ask a user a question using input. For example: print(\'some scenario\') prompt = input(\"You have 10 seconds to choose the correct answer...\\n\") and then if the time elapses print something like print(\'Sorry, times up.\') Any help pointing me in the right direction would be greatly appreciated. 回答1: Interesting problem, this seems to work: import

Timeout on a function call

◇◆丶佛笑我妖孽 提交于 2019-11-25 22:16:27
问题 I\'m calling a function in Python which I know may stall and force me to restart the script. How do I call the function or what do I wrap it in so that if it takes longer than 5 seconds the script cancels it and does something else? 回答1: You may use the signal package if you are running on UNIX: In [1]: import signal # Register an handler for the timeout In [2]: def handler(signum, frame): ...: print "Forever is over!" ...: raise Exception("end of time") ...: # This function *may* run for an