Yes, you're right. It's easy to write slow code in LINQ. The others are right, too: it's easy to write slow code in C# without LINQ.
I wrote the same loop as you in C and it ran some number of milliseconds faster. The conclusion I draw from this is that C# itself is slow.
As with your LINQ->loop expansion, in C it will take more than 5 times as many lines of code to do the same thing, making it slower to write, harder to read, more likely to have bugs, and tougher to find and fix them, but if saving a few milliseconds for every billion iterations is important, that's often what it takes.