How to iterate Queue.Queue items in Python?

Question

Does anyone know a pythonic way of iterating over the elements of a Queue.Queue without removing them from the Queue. I have a producer/consumer-type p

Answer 1

You can loop over a copy of the underlying data store:

for elem in list(q.queue)

Eventhough this bypasses the locks for Queue objects, the list copy is an atomic operation and it should work out fine.

If you want to keep the locks, why not pull all the tasks out of the queue, make your list copy, and then put them back.

mycopy = []
while True:
     try:
         elem = q.get(block=False)
     except Empty:
         break
     else:
         mycopy.append(elem)
for elem in mycopy:
    q.put(elem)
for elem in mycopy:
    # do something with the elements

Answer 2

Listing queue elements without consuming them:

>>> from Queue import Queue
>>> q = Queue()
>>> q.put(1)
>>> q.put(2)
>>> q.put(3)
>>> print list(q.queue)
[1, 2, 3]

After operation, you can still process them:

>>> q.get()
1
>>> print list(q.queue)
[2, 3]

Answer 3

You can subclass queue.Queue to achieve this in a thread-safe way:

import queue


class ImprovedQueue(queue.Queue):
    def to_list(self):
        """
        Returns a copy of all items in the queue without removing them.
        """

        with self.mutex:
            return list(self.queue)