While running benchmark tests this morning, my colleagues and I discovered some strange things concerning performance of C# code vs. VB.NET code.
The difference is in the loop; your C# code is computing the square root on every iteration. Changing that one line from:
for (Int32 i = 2; i <= Math.Sqrt(suspectPrime); i++) {
to:
var lim = Math.Sqrt(suspectPrime);
for (Int32 i = 2; i <= lim; i++) {
dropped the time on my machine from 26 seconds to 7.something.