queue

How should I read multiple queues, pro rata by user and priority?

限于喜欢 提交于 2019-12-08 10:42:48
问题 I am currently trying to think of ways to replace a MySQL + Cron queuing system with a message queue system (AWS SQS/Beanstalkd/Iron MQ/Redis). Let's say I have 100 users. These users are able to make API requests to me. Each API request is an SMS which I must send via a single modem which I operate. Each SMS can have a priority of 1-3. The problem that I am facing, is that the single modem is a bottleneck, so I can't simply process the queue in a FIFO order, because if one user sends 10,000

Bash queue - atomic wait & lock

情到浓时终转凉″ 提交于 2019-12-08 10:23:26
问题 I want to use a queue in Bash, which one process (eg. one Bash script) will be writing to and several other processes will be reading from. I want to do this: writer: # block if another process is reading or writing, # then lock the queue using an exclusive lock lock -ex queue echo "$message" >> queue unlock queue reader: # wait until there is an item in the queue # and then lock the queue using a shared lock wait_while_empty_and_lock -sh queue read -r message < <(tail -n+1 queue) (do stuff..

Multiprocessing Queue need Manager?

☆樱花仙子☆ 提交于 2019-12-08 10:23:01
问题 Why program A deadlock but program B runs fine? Both uses a queue to pass result between main process and sub-processes. Both wait for all sub-processes to end before retrieving the results from the queue. In the actual program, I need a queue to pass streams of results between steps. This is for working on results when the results are generated and before one of the step completes. Changing the mp.queue in program A to a queue of a mp.Manager would fix the deadlock. But using manager seems

LinkedList Memory Leak

喜夏-厌秋 提交于 2019-12-08 09:18:09
问题 I am attempting to debug an application that is hanging. The program uses a queue implemented with a LinkedList , and during stress testing I've discovered that the program stops responding due to running out of heap memory. I analyzed the heap dump and found that memory seems to be leaking from the LinkedList . The relevant part of the heap dump: ▶java.net.Vectior @ 0xff5eacd0 ▶▶java.util.Vector @ 0xff629f30 ▶▶▶java.lang.Object[1280] @ 0xff629f50 ▶▶▶▶class com.itnade.vsm.staticobject

C# Using A Queue to Store multiple 2D Arrays

荒凉一梦 提交于 2019-12-08 09:01:50
问题 I am currently making a board game in visual studio console where I would like to replay the entire game at the end using a queue. To do this I have made sure to enqueue every time I draw my board so the instance of that board is added to the queue. How I instantiate the queue: Queue replay = new Queue(); The code used every time I update the board: draw.UpdateBoard(board); replay.Enqueue(board); The code used to display all the instances of the board at the end: foreach (int[,] q in replay)

Threading / Queue in Python

余生颓废 提交于 2019-12-08 09:01:43
问题 I intend using threads/queues with python 2.5.2 But it seems that python becomes freezed at the queue.join()-command. The output of the followong code is only: BEFORE import Queue import threading queue = Queue.Queue() class ThreadUrl(threading.Thread): def __init__(self, queue): threading.Thread.__init__(self) self.queue = queue def run(self): while True: i = self.queue.get() print i self.queue.task_done() def main(): for i in range(5): t = ThreadUrl(queue) t.setDaemon(True) t.start() for i

check a list constantly and do something if list has items

大城市里の小女人 提交于 2019-12-08 08:48:58
问题 I have a global list where items are added constantly (from network clients): mylist = [] def additem(uuid,work): mylist.append(uuid,work) And a function which should check the list and if there are items proceed them: def proceeditems(): while True: itemdone = [] if len(mylist) > 0: for item in mylist: try: #This can go wrong and than need to be done again result = gevent.spawn(somework(item)) #result returns the uuid itemdone.append(result.value) except: pass for item in itemdone: mylist[:]

Producer consumer code with wait/notify doesn't work on second produce

戏子无情 提交于 2019-12-08 08:17:38
问题 This is a follow-up question from my previous question asked here. I am using a PriorityBlockingQueue now. I changed my producer to the following: synchronized(Manager.queue) { Manager.queue.add(new Job()); Manager.queue.notify(); } And changed Consumer to the following. Full code skeleton is here: //my consumer thread run() public void run() { synchronized(Manager.queue) { while (Manager.queue.peek() == null) { System.out.println("111111111111111"); try { Manager.queue.wait(); } catch

How does the dispatcher work when mixing sync/async with serial/concurrent queue?

流过昼夜 提交于 2019-12-08 08:09:54
问题 In Grand Central Dispatch, how does the dispatcher work with different queues ( serial and concurrent ) when using the dispatch_sync function and the dispatch_async function? 回答1: First of all we need two type of queue : one serial and one concurrent : dispatch_queue_t serialQueue = dispatch_queue_create("com.matteogobbi.dispex.serial_queue", DISPATCH_QUEUE_SERIAL); dispatch_queue_t concurrentQueue = dispatch_queue_create("com.matteogobbi.dispex.concurrent_queue", DISPATCH_QUEUE_CONCURRENT);

Service to process queued long-running processes - where to start?

ⅰ亾dé卋堺 提交于 2019-12-08 07:40:39
问题 What's the best way to build a service to handle queued long-running processes? For example, this is what we're trying to do User uploads 401k data to web Data goes into processing queue (a database table) 401k is processed (could take a couple minutes per client) User is informed via e-mail that 401k was accepted We could certainly write a service to handle this, but what if the service fails? How would someone be notified? What if there is an exception? Also, should this service handle