How can I speed up line by line reading of an ASCII file? (C++)

Question

Here\'s a bit of code that is a considerable bottleneck after doing some measuring:

//-----------------------------------------------------------------------         


        

Answer 1

Quick profiling on my system (linux-2.6.37, gcc-4.5.2, compiled with -O3) shows that I/O is not the bottleneck. Whether using fscanf into a char array followed by dict.insert() or operator>> as in your exact code, it takes about the same time (155 - 160 ms to read a 240k word file).

Replacing gcc's std::unordered_set with std::vector in your code drops the execution time to 45 ms (fscanf) - 55 ms (operator>>) for me. Try to profile IO and set insertion separately.