auto-repeat flag in a pack format string

Question

In php, unpack() has the \"*\" flag which means \"repeat this format until the end of input\". For example, this prints 97, 98, 99

$str = \"abc\";
$b = unpac         


        

Answer 1

In Python 3.4 and later, you can use the new function struct.iter_unpack.

struct.iter_unpack(fmt, buffer)

Iteratively unpack from the buffer buffer according to the format string fmt. This function returns an iterator which will read equally-sized chunks from the buffer until all its contents have been consumed. The buffer’s size in bytes must be a multiple of the size required by the format, as reflected by calcsize().

Each iteration yields a tuple as specified by the format string.

Let's say we want to unpack the array b'\x01\x02\x03'*3 with the repeating format string '<2sc' (2 characters followed by a single character, repeat until done).

With iter_unpack, you can do the following:

>>> import struct
>>> some_bytes = b'\x01\x02\x03'*3
>>> fmt = '<2sc'
>>> 
>>> tuple(struct.iter_unpack(fmt, some_bytes))
((b'\x01\x02', b'\x03'), (b'\x01\x02', b'\x03'), (b'\x01\x02', b'\x03'))

If you want to un-nest this result, you can do so with itertools.chain.from_iterable.

>>> from itertools import chain
>>> tuple(chain.from_iterable(struct.iter_unpack(fmt, some_bytes)))
(b'\x01\x02', b'\x03', b'\x01\x02', b'\x03', b'\x01\x02', b'\x03')

Of course, you could just employ a nested comprehension to do the same thing.

>>> tuple(x for subtuple in struct.iter_unpack(fmt, some_bytes) for x in subtuple)
(b'\x01\x02', b'\x03', b'\x01\x02', b'\x03', b'\x01\x02', b'\x03')