Should LINQ be avoided because it's slow? [closed]

Answer 1

Maybe I've missed something, but I'm pretty sure your benchmarks are off.

I tested with the following methods:

• The Any extension method ("LINQ")
• A simple foreach loop (your "optimized" method)
• Using the ICollection.Contains method
• The Any extension method using an optimized data structure (HashSet<T>)

Here is the test code:

class Program
{
    static void Main(string[] args)
    {
        var names = Enumerable.Range(1, 10000).Select(i => i.ToString()).ToList();
        var namesHash = new HashSet<string>(names);
        string testName = "9999";
        for (int i = 0; i < 10; i++)
        {
            Profiler.ReportRunningTimes(new Dictionary<string, Action>() 
            {
                { "Enumerable.Any", () => ExecuteContains(names, testName, ContainsAny) },
                { "ICollection.Contains", () => ExecuteContains(names, testName, ContainsCollection) },
                { "Foreach Loop", () => ExecuteContains(names, testName, ContainsLoop) },
                { "HashSet", () => ExecuteContains(namesHash, testName, ContainsCollection) }
            },
            (s, ts) => Console.WriteLine("{0, 20}: {1}", s, ts), 10000);
            Console.WriteLine();
        }
        Console.ReadLine();
    }

    static bool ContainsAny(ICollection<string> names, string name)
    {
        return names.Any(s => s == name);
    }

    static bool ContainsCollection(ICollection<string> names, string name)
    {
        return names.Contains(name);
    }

    static bool ContainsLoop(ICollection<string> names, string name)
    {
        foreach (var currentName in names)
        {
            if (currentName == name)
                return true;
        }
        return false;
    }

    static void ExecuteContains(ICollection<string> names, string name,
        Func<ICollection<string>, string, bool> containsFunc)
    {
        if (containsFunc(names, name))
            Trace.WriteLine("Found element in list.");
    }
}

Don't worry about the internals of the Profiler class. It just runs the Action in a loop and uses a Stopwatch to time it. It also makes sure to call GC.Collect() before each test to eliminate as much noise as possible.

Here were the results:

      Enumerable.Any: 00:00:03.4228475
ICollection.Contains: 00:00:01.5884240
        Foreach Loop: 00:00:03.0360391
             HashSet: 00:00:00.0016518

      Enumerable.Any: 00:00:03.4037930
ICollection.Contains: 00:00:01.5918984
        Foreach Loop: 00:00:03.0306881
             HashSet: 00:00:00.0010133

      Enumerable.Any: 00:00:03.4148203
ICollection.Contains: 00:00:01.5855388
        Foreach Loop: 00:00:03.0279685
             HashSet: 00:00:00.0010481

      Enumerable.Any: 00:00:03.4101247
ICollection.Contains: 00:00:01.5842384
        Foreach Loop: 00:00:03.0234608
             HashSet: 00:00:00.0010258

      Enumerable.Any: 00:00:03.4018359
ICollection.Contains: 00:00:01.5902487
        Foreach Loop: 00:00:03.0312421
             HashSet: 00:00:00.0010222

The data is very consistent and tells the following story:

• Naïvely using the Any extension method is about 9% slower than naïvely using a foreach loop.

• Using the most appropriate method (ICollection<string>.Contains) with an unoptimized data structure (List<string>) is approximately 50% faster than naïvely using a foreach loop.

• Using an optimized data structure (HashSet<string>) completely blows any of the other methods out of the water in performance terms.

I have no idea where you got 243% from. My guess is it has something to do with all that casting. If you're using an ArrayList then not only are you using an unoptimized data structure, you're using a largely obsolete data structure.

I can predict what comes next. "Yeah, I know you can optimize it better, but this was just an example to compare the performance of LINQ vs. non-LINQ."

Yeah, but if you couldn't even be thorough in your example, how can you possibly expect to be this thorough in production code?

The bottom line is this:

How you architect and design your software is exponentially more important than what specific tools you use and when.

If you run into performance bottlenecks - which is every bit as likely to happen with LINQ vs. without - then solve them. Eric's suggestion of automated performance tests is an excellent one; that will help you to identify the problems early so that you can solve them properly - not by shunning an amazing tool that makes you 80% more productive but happens to incur a < 10% performance penalty, but by actually investigating the issue and coming up with a real solution that can boost your performance by a factor of 2, or 10, or 100 or more.

Creating high-performance applications is not about using the right libraries. It's about profiling, making good design choices, and writing good code.

Answer 2

Maybe linq is slow, but with linq i can parallelize my code very simple.

Like this:

lst1.Cast<MyLinqTestClass1>().AsParallel().Any(item => item.Name == "9999");

How you would parallelize cycle?

Answer 3

To me, this sounds like you're working on a contract and the employer either doesn't understand LINQ, or doesn't understand the performance bottlenecks of the system. If you're writing an applicaiton with a GUI, the minor performance impact of using LINQ is negligible. In a typical GUI/Web app, in-memory calls make up less than 1% of all wait time. You, or rather your employer, is trying to optimize that 1%. Is that really beneficial?

However, if you are writing an application that is scientific or heavily math oriented, with very little disk or database access, then I'd agree that LINQ is not the way to go.

BTW, the cast is not needed. The following is functionally equivalent to your first test:

       for (int i = 0; i < 10000; i++)
            isInGroup = lst1.Any(item => item.Name == "9999");

When I ran this using a test list containing 10,000 MyLinqTestClass1 objects, the original ran in 2.79 seconds, the revised in 3.43 seconds. Saving 30% on operations that likely take up less than 1% percent of CPU time is not a good use of your time.

Answer 4

Your test case is a little skewed. The ANY operater will begin enumerating through your results and return true on the first instance if finds and quit. Try this with simple lists of strings to see the result. To answer your question about avoiding LINQ, you should really transition toward the use of LINQ. It makes code easier to read when doing complex queries in addition to compile time checking. Also you don't need to use the Cast operator in your example.

string compareMe = "Success";
string notEqual = "Not Success";

List<string> headOfList = new List<string>();
List<string> midOfList = new List<string>();
List<string> endOfList = new List<string>();

//Create a list of 999,999 items
List<string> masterList = new List<string>();
masterList.AddRange(Enumerable.Repeat(notEqual, 999999));

//put the true case at the head of the list
headOfList.Add(compareMe);
headOfList.AddRange(masterList);

//insert the true case in the middle of the list
midOfList.AddRange(masterList);
midOfList.Insert(masterList.Count/2, compareMe);

//insert the true case at the tail of the list
endOfList.AddRange(masterList);
endOfList.Add(compareMe);


Stopwatch stopWatch = new Stopwatch();

stopWatch.Start();
headOfList.Any(p=>p == compareMe);
stopWatch.ElapsedMilliseconds.Dump();
stopWatch.Reset();

stopWatch.Start();
midOfList.Any(p=>p == compareMe);
stopWatch.ElapsedMilliseconds.Dump();
stopWatch.Reset();

stopWatch.Start();
endOfList.Any(p=>p == compareMe);
stopWatch.ElapsedMilliseconds.Dump();
stopWatch.Stop();

Answer 5

There are a thousand times better reasons to avoid Linq.

Following quote from a discussion on Linq names a few of them:

QUOTE1

"For instance this works:

var a = new { x = 1, y = 2 };
a = new { x = 1, y = 3 };

But this does not work:

var a = new { x = 1, y = 2 };
a = new { x = 1, y = 2147483649 };

It returns : Error 1 Cannot implicitly convert type 'AnonymousType#1' to 'AnonymousType#2'

But this works:

var a = new { x = 1, y = 2147483648 };
a = new { x = 1, y = 2147483649 };

When you compile:

var a = new { x = 1, y = 2 };

The type of the x and y components is arbitrarily declared as a 32 bit signed integer, and it is one of the many integer types the platform has, without anything special.

But there is more. For instance this works:

double x = 1.0;
x = 1;

But this does not work:

var a = new { x = 1.0, y = 0 }; 
a = new { x = 1, y = 0 };

The numeric conversion rules are not applicable to this kind of types. As you can see, elegance is in every detail."

QUOTE2

"It appears, then, that 'AnonymousType#1' and 'AnonymousType#2' are not synonymous--they name distinct types. And as { x = 1, y = 2 } and { y = 2, x = 1 } are expressions of those two types, respectively, not only do they denote distinct values, but also values of distinct types.

So, I was right to be paranoid. Now my paranoia extends even further and I have to ask what LinQ makes of the following comparison:

new { x = 1, y = 2 } == new { x = 1, y = 2 }

The result is false because this is a pointer comparison.

But the result of:

(new { x = 1, y = 2 }).Equals(new { x = 1, y = 2 })

Is true.

And the result of:

(new { x = 1, y = 2 }).Equals(new { y = 2, x = 1 })

and

(new { x = 1, y = 2 }).Equals(new { a = 1, b = 2 })

Is false."

QUOTE3

"updates are record oriented :-O

This, I agree, is problematic, and derives from LINQ's sequence-oriented nature.

This is a show stopper for me. If I have to use SQL for my updates anyway why to bother about LinQ?

the optimization in LinQ to objects is unexistent.

There is not any algebraic optimization nor automatic expression rewrite. Many people don't want to use LinQ to Objects because they lose a lot of performance. Queries are executed in the same way as you write them."

Answer 6

While premature pessimization is (imho) as bad as premature optimization, you shouldn't rule out an entire technology based on absolute speed without taking usage context into consideration. Yes, if you're doing some really heavy number-crunching and this is a bottleneck, LINQ could be problematic - profile it.

An argument you could use in favour of LINQ is that, while you can probably outperform it with handwritten code, the LINQ version could likely be clearer and easier to maintain - plus, there's the advantage of PLINQ compared to complex manual parallelization.