Is there a way to use asyncio.Queue in multiple threads?

前端 未结 4 1409
渐次进展
渐次进展 2020-12-05 11:36

Let\'s assume I have the following code:

import asyncio
import threading

queue = asyncio.Queue()

def threaded():
    import time
    while True:
        ti         


        
4条回答
  •  鱼传尺愫
    2020-12-05 11:52

    What about just using threading.Lock with asyncio.Queue?

    class ThreadSafeAsyncFuture(asyncio.Future):
        """ asyncio.Future is not thread-safe
        https://stackoverflow.com/questions/33000200/asyncio-wait-for-event-from-other-thread
        """
        def set_result(self, result):
            func = super().set_result
            call = lambda: func(result)
            self._loop.call_soon_threadsafe(call)  # Warning: self._loop is undocumented
    
    
    class ThreadSafeAsyncQueue(queue.Queue):
        """ asyncio.Queue is not thread-safe, threading.Queue is not awaitable
        works only with one putter to unlimited-size queue and with several getters
        TODO: add maxsize limits
        TODO: make put corouitine
        """
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
            self.lock = threading.Lock()
            self.loop = asyncio.get_event_loop()
            self.waiters = []
    
        def put(self, item):
            with self.lock:
                if self.waiters:
                    self.waiters.pop(0).set_result(item)
                else:
                    super().put(item)
    
        async def get(self):
            with self.lock:
                if not self.empty():
                    return super().get()
                else:
                    fut = ThreadSafeAsyncFuture()
                    self.waiters.append(fut)
            result = await fut
            return result
    

    See also - asyncio: Wait for event from other thread

提交回复
热议问题