python-multithreading | 易学教程

The right way to limit maximum number of threads running at once?

阅读更多关于 The right way to limit maximum number of threads running at once?

I'd like to create a program that runs multiple light threads, but limits itself to a constant, predefined number of concurrent running tasks, like this (but with no risk of race condition): import threading def f(arg): global running running += 1 print("Spawned a thread. running=%s, arg=%s" % (running, arg)) for i in range(100000): pass running -= 1 print("Done") running = 0 while True: if running < 8: arg = get_task() threading.Thread(target=f, args=[arg]).start() What's the safest/fastest way to implement this? It sounds like you want to implement the producer/consumer pattern with eight

A very simple multithreading parallel URL fetching (without queue)

阅读更多关于 A very simple multithreading parallel URL fetching (without queue)

I spent a whole day looking for the simplest possible multithreaded URL fetcher in Python, but most scripts I found are using queues or multiprocessing or complex libraries. Finally I wrote one myself, which I am reporting as an answer. Please feel free to suggest any improvement. I guess other people might have been looking for something similar. Simplifying your original version as far as possible: import threading import urllib2 import time start = time.time() urls = ["http://www.google.com", "http://www.apple.com", "http://www.microsoft.com", "http://www.amazon.com", "http://www.facebook

cython shared memory in cython.parallel.prange - block

阅读更多关于 cython shared memory in cython.parallel.prange - block

问题 I have a function foo that takes a pointer to memory as argument and both writes and reads to that memory: cdef void foo (double *data): data[some_index_int] = some_value_double do_something_dependent_on (data) I'm allocating to data like so: cdef int N = some_int cdef double *data = <double*> malloc (N * sizeof (double)) cdef int i for i in cython.parallel.prange (N, nogil=True): foo (data) readout (data) My question is now: how do the different threads treat this? My guess is that the

Python - appending to same file from multiple threads

阅读更多关于 Python - appending to same file from multiple threads

问题 I'm writing an app that appends lines to the same file from multiple threads. I have a problem in which some lines are appended without a new line. Any solution for this? class PathThread(threading.Thread): def __init__(self, queue): threading.Thread.__init__(self) self.queue = queue def printfiles(self, p): for path, dirs, files in os.walk(p): for f in files: print(f, file=output) def run(self): while True: path = self.queue.get() self.printfiles(path) self.queue.task_done() pathqueue =

How to Multi-thread an Operation Within a Loop in Python

阅读更多关于 How to Multi-thread an Operation Within a Loop in Python

问题 Say I have a very large list and I'm performing an operation like so: for item in items: try: api.my_operation(item) except: print 'error with item' My issue is two fold: There are a lot of items api.my_operation takes forever to return I'd like to use multi-threading to spin up a bunch of api.my_operations at once so I can process maybe 5 or 10 or even 100 items at once. If my_operation() returns an exception (because maybe I already processed that item) - that's OK. It won't break anything.

what is the use of join() in python threading

阅读更多关于 what is the use of join() in python threading

I was studying the python threading and came across join() . The author told that if thread is in daemon mode then i need to use join() so that thread can finish itself before main thread terminates. but I have also seen him using t.join() even though t was not daemon example code is this import threading import time import logging logging.basicConfig(level=logging.DEBUG, format='(%(threadName)-10s) %(message)s', ) def daemon(): logging.debug('Starting') time.sleep(2) logging.debug('Exiting') d = threading.Thread(name='daemon', target=daemon) d.setDaemon(True) def non_daemon(): logging.debug(

Thread vs. Threading

阅读更多关于 Thread vs. Threading

问题 What\'s the difference between the threading and thread modules in Python? 回答1: In Python 3, thread has been renamed to _thread . It is infrastructure code that is used to implement threading , and normal Python code shouldn't be going anywhere near it. _thread exposes a fairly raw view of the underlying OS level processes. This is almost never what you want, hence the rename in Py3k to indicate that it is really just an implementation detail. threading adds some additional automatic

thread starts running before calling Thread.start

阅读更多关于 thread starts running before calling Thread.start

t1=threading.Thread(target=self.read()) print "something" t2=threading.Thread(target=self.runChecks(), args=(self)) self.read runs indefinitely, so the program won't ever reach the print line. How is this possible without calling t1.start() ? (Even if I call that, it shold start running and go on to the next line, shouldn't it?) user634175 You're passing the result of self.read to the target argument of Thread. Thread expects to be passed a function to call, so just remove the parentheses and remember to start the Thread: t1=threading.Thread(target=self.read) t1.start() print "something" For

How to terminate a thread when main program ends?

阅读更多关于 How to terminate a thread when main program ends?

问题 If I have a thread in an infinite loop, is there a way to terminate it when the main program ends (for example, when I press Ctrl + C )? 回答1: Check this question. The correct answer has great explanation on how to terminate threads the right way: Is there any way to kill a Thread in Python? To make the thread stop on Keyboard Interrupt signal (ctrl+c) you can catch the exception "KeyboardInterrupt" and cleanup before exiting. Like this: try: start_thread() except (KeyboardInterrupt,

Freezing/Hanging tkinter Gui in waiting for the thread to complete

阅读更多关于 Freezing/Hanging tkinter Gui in waiting for the thread to complete

问题 My interface is freezing on pressing the button. I am using threading but I am not sure why is still hanging. Any help will be appreciated. Thanks in advance class magic: def __init__(self): self.mainQueue=queue.Queue() def addItem(self,q): self.mainQueue.put(q) def startConverting(self,funcName): if(funcName==\"test\"): while not self.mainQueue.empty(): t = Thread(target = self.threaded_function) t.start() t.join() def threaded_function(self): time.sleep(5) print(self.mainQueue.get()) m