I am getting memory exception “System.IO.out of exception” error

Question

For small directory size code is working fine ,it gives this error message when size of directory files are big.

My code :

IEnumerable text         


        

Answer 1

try to use next code, it uses ReadLines, which doesn't load entire file into memory, but read file line by line. It also uses HashSet to store unique results from matching a regular expression.

Regex regex = new Regex(@"User:\s*(?[^\s]+)");
IEnumerable textLines = 
      Directory.GetFiles(@"C:\Users\karansha\Desktop\watson_query\", "*.*")
               .Select(filePath => File.ReadLines(filePath))
               .SelectMany(line => line)
               .Where(line => !line.Contains("appGUID: null"));

HashSet users = new HashSet(
    textLines.SelectMany(line => regex.Matches(line).Cast())
             .Select(match => match.Groups["username"].Value)
);

int numberOfUsers = users.Count(name => name.Length <= 10);
Console.WriteLine("Unique_Users_Express=" + numberOfUsers);