DataTable.Select vs DataTable.rows.Find vs foreach vs Find(Predicate)/Lambda

Question

I have a DataTable/collection that is cached in memory, I want to use this as a source to generate results for an auto complete textbox (using AJAX of course). I am evaluati

Answer 1

On my autocomplete, i tried first the linq/lambda approach, the performance is a little slow. DataTable.Select is faster than linq, so I use this. I haven't yet compared the performance between datatable.Select and datatable.Find

Answer 2

What about a DataView? You could apply your filter condition AND sort by the the priority, and easily iterate through the results to add to your results.

Answer 3

The charts aren't posted on my blog entry; more details can be found at http://msdn.microsoft.com/en-us/library/dd364983.aspx

One other thing that I've since discovered is that, for large data sets, using a chained generic dictionary performs incredibly well. It also helps alleviate many of the issues caused by the sort operations required for aggregation operations such as min and max (either with DataTable.Compute or LINQ).

By "chained generic dictionary," I mean a Dictionary(Of String, Dictionary(Of String, Dictionary(Of Integer, List(Of DataRow)))) or similar technique, where the key for each dictionary is a search term.

Granted, this won't be useful in all circumstances, but I have at least one scenario where implementing this approach lead to a 500x performance improvement.

In your case, I'd consider using a simple dictionary with the first 1-5 characters, then a List(Of String). You'd have to build up this dictionary once, adding the words to the lists with the first 1-5 characters, but after that you'll be able to get blazingly fast results.

I generally wrap things like this in a class that allows me to do things like add words easily. You may also want to use a SortedList(Of String), to get the results sorted automatically. This way, you can quickly look up the list of words that match the first N characters that have been typed.

Answer 4

as per the following blog

http://blog.dotnetspeech.net/archive/2008/08/26/performance----datatable.select-vs-dictionary.aspx

DataTable.Rows.Find is much, much faster than DataTable.Select.

Answer 5

We could speculate about it all day, but since this is not a huge piece of code, why not write each one and benchmark them against each other?

public delegate void TestProcedure();

public TimeSpan Benchmark(TestProcedure tp)
{
    int testBatchSize = 5;
    List<TimeSpan> results = new List<TimeSpan>();
    for(int i = 0; i<testBatchSize; i++)
    {
        DateTime start = DateTime.Now;
        tp();
        results.Add(DateTime.Now - start);
    }
    return results.Min();
}