Explanation why IEnumerable is more efficient than a List

Question

I keep hearing that in .net 3.5 you should use IEnumerable over a List, but I can’t find any reference materials or articles that explain why it’s so much more proficient. Does

Answer 1

Enumerables have several very nice properties that you lose when converting them to a list. Namely they:

• Use Deferred/lazy Execution
• Are Composable
• Are Unbounded

First I'll look at deferred execution. Pop quiz: how many times will the following code iterate over the lines in the input file?

IEnumerable ReadLines(string fileName)
{
    using (var rdr = new StreamReader(fileName) )
    {
       string line;
       while ( (line = rdr.ReadLine()) != null) yield return line;
    }
}


var SearchIDs = new int[] {1234,4321, 9802};

var lines = ReadLines("SomeFile.txt")
              .Where(l => l.Length > 10 && l.StartsWith("ID: "));
              .Select(l => int.Parse(l.Substring(4).Trim()));
              .Intersect(SearchIDs);

The answer is exactly one zero. It doesn't actually do any work until you iterate over the results. You need to add this code before it even opens the file:

foreach (string line in lines) Console.WriteLine(line);

Even after the code runs, it still only ever loops over the lines once. Compare that to how many times you need to iterate over the lines in this code:

var SearchIDs = new int[] {1234,4321, 9802};
var lines = File.ReadAllLines("SomeFile.txt"); //creates a list
lines = lines.Where(l => l.Length > 10 && l.StartsWith("ID: ")).ToList();
var ids = lines.Select(l => int.Parse(l.Substring(4).Trim())).ToList();
ids = ids.Intersect(SearchIDs).ToList();

foreach (string line in lines) Console.WriteLine(line);

Even if you ignore the File.ReadAllLines() call and use the same iterator block from the first sample, the first sample would still be faster. Of course, you could write it to be just as fast using lists, but to do requires tying the code that reads the file to the code the code that parses it. And so you lose another important feature: composability.

To demonstrate composability, I'll add one the final feature — unbounded series. Consider the folllowing:

IEnumerable Fibonacci()
{
   int n1 = 1, n2 = 0, n;
   yield return 1;
   while (true)
   {
        n = n1 + n2;
        yield return n;
        n2 = n1;
        n1 = n;
   }
}

This looks like it would go forever, but you can use the composability properties of IEnumerable to build something that safely gives, say, the first 50 values, or every value that's less than a given number:

  foreach (int f in Fibonacci().Take(50)) { /* ... */ }
  foreach (int f in Fibonacci().TakeWhile(i => i < 1000000) { /* ... */ }

Finally, IEnumerable is just more flexible. Unless you absolutely need the ability to append to the list or access items by index, you're almost always better writing functions to accept IEnumerables as arguments instead of Lists. Why? Because you can still pass the list to the function if you want — A list is an IEnumerable. For that matter, so is an array and many other collection types are well. So by using an IEnumerable here, you take the exact same function and make it more powerful, because it can act on more different kinds of data.