The example below throws an InvalidOperationException, \"Collection was modified; enumeration operation may not execute.\" when executing the code.
var urls
I would create two lists add into the second and then update the reference like this:
var urls = new List();
var destUrls = new List(urls);
urls.Add("http://www.google.com");
foreach (string url in urls)
{
// Get all links from the url
List newUrls = GetLinks(url);
destUrls.AddRange(newUrls);
}
urls = destUrls;