Delphi: storing data in classes vs records, memory usage reduction

Question

I have quite a lot of data to store, read and modify in memory while the application works. The data could be compared to a tree, where each node is described by limited num

Answer 1

If you can set limits on strings as Kornel says, it really matters. ansistring has some internal overhead, and the additional overhead too. However shortstring is always allocated, even when not used.

If you are really tight on memory, doing your own allocation for the strings is more sensible, specially if the data is relatively immutable. Then simply allocate a big block, and put all strings in there, prefixed with a 16-bit length or so.

Less lowlevel tricks, like simply deduping (some of the) strings saves a lot of storage space too.

Note that the record vs class discussion of Rob only goes if you manage to statically instantiate the class in memory you allocate very cheaply, which you probably don't do. This because you can use array of record. Otherwise the fact that it is always a reference type causes heapoverhead and -slack (fastmm, 16 byte granularity)

I would recommend against using tstringlist/tlist/tobjectlist, because insertion deletion in very large lists (millions) can be painfull because the deletion/insertion is O(n), and inserting in the middle means shifting half the data. This gets painful somewhere between 20-100k and 1M elements, depending on how your access pattern is.

Using a tlist of tlists, and not letting every tlist get too big is already a good workaround.

When I did this (for an OLAP cluster when 2GB server memory was still $2000), I at one point even used the alignment bits in pointers to store the size-class of allocations. I wouldn't recommend that though :-)

Of course going 64-bit with FPC is also an option. I've got the core server part of above 32-bit solution working in 64-bit in under an hour.