What to put in a binary data file's header

Question

I have a simulation that reads large binary data files that we create (10s to 100s of GB). We use binary for speed reasons. These files are system dependent, converted from te

Answer 1

@rstevens said 'an identifier for the type of file'...sound advice. Conventionally, that's called a magic number and, in a file, isn't a term of abuse (unlike in code, where it is a term of abuse). Basically, it is some number - typically at least 4 bytes, and I usually ensure that at least one of those bytes is not ASCII - that you can use to validate that the file is of the type you expect with a low probability of being confused. You can also write a rule in /etc/magic (or local equivalent) to report that files containing your magic number are your special file type.

You should include a file format version number. However, I would recommend not using the SVN number of the code. Your code may change when the file format does not.