How to manage multiple versions of the same record

Question

I am doing short-term contract work for a company that is trying to implement a check-in/check-out type of workflow for their database records.

Here's how it should work...

1. A user creates a new entity within the application. There are about 20 related tables that will be populated in addition to the main entity table.
2. Once the entity is created the user will mark it as the master.
3. Another user can make changes to the master only by "checking out" the entity. Multiple users can checkout the entity at the same time.
4. Once the user has made all the necessary changes to the entity, they put it in a "needs approval" status.
5. After an authorized user reviews the entity, they can promote it to master which will put the original record in a tombstoned status.

The way they are currently accomplishing the "check out" is by duplicating the entity records in all the tables. The primary keys include EntityID + EntityDate, so they duplicate the entity records in all related tables with the same EntityID and an updated EntityDate and give it a status of "checked out". When the record is put into the next state (needs approval), the duplication occurs again. Eventually it will be promoted to master at which time the final record is marked as master and the original master is marked as dead.

This design seems hideous to me, but I understand why they've done it. When someone looks up an entity from within the application, they need to see all current versions of that entity. This was a very straightforward way for making that happen. But the fact that they are representing the same entity multiple times within the same table(s) doesn't sit well with me, nor does the fact that they are duplicating EVERY piece of data rather than only storing deltas.

I would be interested in hearing your reaction to the design, whether positive or negative.

I would also be grateful for any resoures you can point me to that might be useful for seeing how someone else has implemented such a mechanism.

Thanks!
Darvis

Answer 1

I've worked on a system like this which supported the static data for trading at a very large bank. The static data in this case is things like the details of counterparties, standard settlement instructions, currencies (not FX rates) etc. Every entity in the database was versioned, and changing an entity involved creating a new version, changing that version and getting the version approved. They did not however let multiple people create versions at the same time.

This lead to a horribly complex database, with every join having to take version and approval state into account. In fact the software I wrote for them was middleware that abstracted this complex, versioned data into something that end-user applications could actually use.

The only thing that could have made it any worse was to store deltas instead of complete versioned objects. So the point of this answer is - don't try to implement deltas!

Answer 2

You are describing a homebrew Content Management System which was probably hacked together over time, is - for the reasons you state - redundant and inefficient, and given the nature of such systems in firms is unlikely to be displaced without massive organizational effort.

Answer 3

This looks like an example of a temporal database schema -- Often, in cases like that, there is a distinction made between an entity's key (EntityID, in your case) and the row primary key in the database (in your case, {EntityID, date}, but often a simple integer). You have to accept that the same entity is represented multiple times in the database, at different points in its history. Every database row still has a unique ID; it's just that your database is tracking versions, rather than entities.

You can manage data like that, and it can be very good at tracking changes to data, and providing accountability, if that is required, but it makes all of your queries quite a bit more complex.

You can read about the rationale behind, and design of temporal databases on Wikipedia