Is there a way to use asyncio.Queue in multiple threads?

Question

Let\'s assume I have the following code:

import asyncio
import threading

queue = asyncio.Queue()

def threaded():
    import time
    while True:
        ti         


        

Answer 1

If you do not want to use another library you can schedule a coroutine from the thread. Replacing the queue.put_nowait with the following works fine.

asyncio.run_coroutine_threadsafe(queue.put(time.time()), loop)

The variable loop represents the event loop in the main thread.

EDIT:

The reason why your async coroutine is not doing anything is that the event loop never gives it a chance to do so. The queue object is not threadsafe and if you dig through the cpython code you find that this means that put_nowait wakes up consumers of the queue through the use of a future with the call_soon method of the event loop. If we could make it use call_soon_threadsafe it should work. The major difference between call_soon and call_soon_threadsafe, however, is that call_soon_threadsafe wakes up the event loop by calling loop._write_to_self() . So let's call it ourselves:

import asyncio
import threading

queue = asyncio.Queue()

def threaded():
    import time
    while True:
        time.sleep(2)
        queue.put_nowait(time.time())
        queue._loop._write_to_self()
        print(queue.qsize())

@asyncio.coroutine
def async():
    while True:
        time = yield from queue.get()
        print(time)

loop = asyncio.get_event_loop()
asyncio.Task(async())
threading.Thread(target=threaded).start()
loop.run_forever()

Then, everything works as expected.

As for the threadsafe aspect of accessing shared objects,asyncio.queue uses under the hood collections.deque which has threadsafe append and popleft. Maybe checking for queue not empty and popleft is not atomic, but if you consume the queue only in one thread (the one of the event loop) it could be fine.

The other proposed solutions, loop.call_soon_threadsafe from Huazuo Gao's answer and my asyncio.run_coroutine_threadsafe are just doing this, waking up the event loop.