How to store dynamic data (unknown number of fields) to a file?

Question

I need to store some data in a file. FOR THE MOMENT each record (data set) consists in:

• a string (variable length),
• an array of integers (variable len

Answer 1

I don't think you need a database for this. If you use a database I don't see how it solves the problem of your data structure changing.

I personally would store to YAML format which is very easily extensible. That requires quite a bit of work linking to some LIBYAML so a very lightweight alternative would be to store to INI files. These are easily extensible whilst maintaining compatibility with old files.

You can quite easily roll your own binary format that is extensible. What you do is you write each record to a block. Each block has a short header which includes its length.

When you read the data you read up to the end of the block and then if you are expecting more data you simply stop reading and use default values for the data. If you have read all the data you know about but are not at the end of the block, the file must have come from a later version of your program and you just skip to the end of the block. Perhaps you warn that the file contained data which you didn't know about.

Extensibility is achieved by always writing data out in the same order as previous versions. Any new data goes at the end of each block.