Very Large Mnesia Tables in Production

Question

We are using Mnesia as a primary Database for a very large system. Mnesia Fragmented Tables have behaved so well over the testing period. System has got about 15 tables, eac

Answer 1

The hint of having a power of two number of fragments is simply related to the fact the default fragmentation module mnesia_frag uses linear hashing so using 2^n fragments assures that records are equally distributed (more or less, obviously) between fragments.

Regarding the hardware at disposal, it's more a matter of performance testing. The factors that can reduce performance are many and configuring a database like Mnesia is just one single part of the general problem. I simply advice you to stress test one server and then test the algorithm on both servers to understand if it scales correctly.

Talking about Mnesia fragments number scaling remember that by using disc_only_copies most of the time is spent in two operations:

• decide which fragment holds which record

• retrieve the record from corresponding dets table (Mnesia backend)

The first one is not really dependent from the number of fragments considered that by default Mnesia uses linear hashing. The second one is more related to hard disk latency than to other factors.

In the end a good solution could be to have more fragments and less records per fragment but trying at the same time to find the middle ground and not lose the advantages of some hard disk performance boosts like buffers and caches.