queue

Send socket object to forked running process (multiprocessing.Queue)

橙三吉。 提交于 2019-12-30 19:01:31
问题 I am learning to use HTML5 WebSockets and as part of that I am writing a server in Python so I can know the nitty gritty of how they work. I created one the other day that worked pretty well, but I wanted to expand it so it would support multiple endpoints with each endpoint being a different "service" which can handle websocket clients. At the moment, my implementation works with spawning processes and such (I am using multiprocessing instead of threading since I read that threading isn't

Threadsafe FIFO Queue/Buffer

别说谁变了你拦得住时间么 提交于 2019-12-30 06:18:49
问题 I need to implement a sort of task buffer. Basic requirements are: Process tasks in a single background thread Receive tasks from multiple threads Process ALL received tasks i.e. make sure buffer is drained of buffered tasks after a stop signal is received Order of tasks received per thread must be maintained I was thinking of implementing it using a Queue like below. Would appreciate feedback on the implementation. Are there any other brighter ideas to implement such a thing? public class

How to Build a PHP Queue System

与世无争的帅哥 提交于 2019-12-29 11:43:06
问题 I had to build a PHP Queue System, and found this brilliant article http://squirrelshaterobots.com/programming/php/building-a-queue-server-in-php-part-1-understanding-the-project and I used it to create a PHP queue system, its very easy to set-up and use. Below is the code for queue.php, run from shell (puTTy or somesuch). <?PHP //. set this constant to false if we ever need to debug //. the application in a terminal. define('QUEUESERVER_FORK', true); //////// fork into a background process /

random unique number in java

家住魔仙堡 提交于 2019-12-29 09:07:21
问题 I'm trying to get a a list of random number and put it in queue without any repetition of the random number. int number = 40; for (int j = 0; j<number; j++) { int pick = random.nextInt(number); myQueue.add(Integer.toString(pick)); } System.out.println("the queue size: "+myQueue.size()); Iterator it = myQueue.iterator(); while(it.hasNext()){ String iteratorValue = (String)it.next(); System.out.println("queue next value: "+iteratorValue); } with the above code, I got some repetition of the

Multithreaded file copy is far slower than a single thread on a multicore CPU

自作多情 提交于 2019-12-29 08:59:09
问题 I am trying to write a multithreaded program in Python to accelerate the copying of (under 1000) .csv files. The multithreaded code runs even slower than the sequential approach. I timed the code with profile.py . I am sure I must be doing something wrong but I'm not sure what. The Environment: Quad core CPU. 2 hard drives, one containing source files. The other is the destination. 1000 csv files ranging in size from several KB to 10 MB. The Approach: I put all the file paths in a Queue, and

In C# would it be better to use Queue.Synchronized or lock() for thread safety?

杀马特。学长 韩版系。学妹 提交于 2019-12-28 11:43:07
问题 I have a Queue object that I need to ensure is thread-safe. Would it be better to use a lock object like this: lock(myLockObject) { //do stuff with the queue } Or is it recommended to use Queue.Synchronized like this: Queue.Synchronized(myQueue).whatever_i_want_to_do(); From reading the MSDN docs it says I should use Queue.Synchronized to make it thread-safe, but then it gives an example using a lock object. From the MSDN article: To guarantee the thread safety of the Queue, all operations

How to use multiprocessing.Queue.get method?

六眼飞鱼酱① 提交于 2019-12-28 06:53:08
问题 The code below places three numbers in a queue. Then it attempts to get the numbers back from the queue. But it never does. How to get the data from the queue? import multiprocessing queue = multiprocessing.Queue() for i in range(3): queue.put(i) while not queue.empty(): print queue.get() 回答1: I originally deleted this answer after I read @Martijn Pieters', since he decribed the "why this doesn't work" in more detail and earlier. Then I realized, that the use case in OP's example doesn't

Concurrent Set Queue

Deadly 提交于 2019-12-28 05:56:21
问题 Maybe this is a silly question, but I cannot seem to find an obvious answer. I need a concurrent FIFO queue that contains only unique values. Attempting to add a value that already exists in the queue simply ignores that value. Which, if not for the thread safety would be trivial. Is there a data structure in Java or maybe a code snipit on the interwebs that exhibits this behavior? 回答1: If you want better concurrency than full synchronization, there is one way I know of to do it, using a

Which concurrent Queue implementation should I use in Java?

我们两清 提交于 2019-12-27 12:07:59
问题 From the JavaDocs: A ConcurrentLinkedQueue is an appropriate choice when many threads will share access to a common collection. This queue does not permit null elements. ArrayBlockingQueue is a classic "bounded buffer", in which a fixed-sized array holds elements inserted by producers and extracted by consumers. This class supports an optional fairness policy for ordering waiting producer and consumer threads LinkedBlockingQueue typically have higher throughput than array-based queues but

Using many consumers in SQS Queue

ぃ、小莉子 提交于 2019-12-27 11:51:14
问题 I know that it is possible to consume a SQS queue using multiple threads. I would like to guarantee that each message will be consumed once. I know that it is possible to change the visibility timeout of a message, e.g., equal to my processing time. If my process spend more time than the visibility timeout (e.g. a slow connection) other thread can consume the same message. What is the best approach to guarantee that a message will be processed once? 回答1: What is the best approach to guarantee