Reading a binary file into a struct

Question

I have a binary file with a known format/structure.

How do I read all the binary data in to an array of the structure?

Something like (in pseudo code)

<

Answer 1

Use the struct module; you need to define the types in a string format documented with that library:

struct.unpack('=HHf255s', bytes)

The above example expects native byte-order, two unsigned shorts, a float and a string of 255 characters.

To loop over an already fully read bytes string, I'd use itertools; there is a handy grouper recipe that I've adapter here:

from itertools import izip_longest, imap
from struct import unpack, calcsize

fmt_s = '=5i'
fmt_spec = '=256i'
size_s = calcsize(fmt_s)
size = size_s + calcsize(fmt_spec)

def chunked(iterable, n, fillvalue=''):
    args = [iter(iterable)] * n
    return imap(''.join, izip_longest(*args, fillvalue=fillvalue))

data = [unpack(fmt_s, section[:size_s]) + (unpack(fmt_spec, section[size_s:]),)
    for section in chunked(bytes, size)]

This produces tuples rather than lists, but it's easy enough to adjust if you have to:

data = [list(unpack(fmt_s, section[:size_s])) + [list(unpack(fmt_spec, section[size_s:]))]
    for section in chunked(bytes, size)]