Delphi: Store data in somekind of structure

Question

For a simulation program I\'m working in Delphi 2010. The simulation isn\'t a problem but I need to use large collection of data which gives a problem. The data is available

Answer 1

What you need is the so-called "serialization" mechanism.

1. The standard way

1.1 SaveToStream

In Delphi, we usually implement a SaveToStream method, which will save the content of each object in a destination TStream (either a TFileStream or a TMemoryStream).

You'll have to write the serialization by hand.

1.2 DFM-like streaming

See TWriter / TReader classes.

If you define your data in published properties, you are able to serialize them using those standard Delphi classes.

For some methods able to serialize any TCollection to and from JSON content, see this blog article.

2. The RTTI

See for instance this SO question.

In particular, the new enhanced RTTI (available since Delphi 2010) opens new opportunities to serialization.

3. Use records instead of classes

If each item does not store a lot of content (some integer/boolean), it may make sense to use records instead of objects. For speed and memory consumption/fragmentation, it may be worth it.

Here is some wrapper able to serialize any dynamic array, even containing nested records or dynamic arrays.

4. Use a database engine

Perhaps the better approach is not to have your data stuck in a non-evolving binary form, proprietary to your application. If you want to add a property, you'll have to manage it by hand. Or if you want to access your data from other applications, it may be difficult.

There are a lot of database solutions around - instead of using an external database (like MS SQL, FireBird or Oracle), it could be a good idea to embed the database inside your application (much easier to install). Worth mentioning SQLite which has a lot of wrappers, including our version (which will allow you to change to any other database if you want to use MS SQL or Oracle instead).

You have other solutions around - see this SO question - and if you need performance, take a look at our Big Table library.