I am writing a C# application that needs to read about 130,000 (String, Int32) pairs at startup to a Dictionary. The pairs are stored in a .txt file, and are thus easily mod
Is it safe enough to use BinaryFormatter instead of storing the contents directly in the text file? Obviously not. Because others can easily "destroy" the file by opening it by notepad and add something, even though he can see strange characters only. It's better if you store it in a database. But if you insist your solution, you can easily improve the performance a lot, by using Parallel Programming in C#4.0 (you can easily get a lot of useful examples by googling it). Something looks like this:
//just an example
Dictionary source = GetTheDict();
var grouped = source.GroupBy(x =>
{
if (x.Key.First() >= 'a' && x.Key.First() <= 'z') return "File1";
else if (x.Key.First() >= 'A' && x.Key.First() <= 'Z') return "File2";
return "File3";
});
Parallel.ForEach(grouped, g =>
{
ThreeStreamsToWriteToThreeFilesParallelly(g);
});
Another alternative solution of Parallel is creating several threads, reading from/writing to different files will be faster.