Why is “slurping” a file not a good practice?

Question

Why is \"slurping\" a file not a good practice for normal text-file I/O, and when is it useful?

For example, why shouldn\'t I use these?

File.read(\'         


        

Answer 1

Why is "slurping" a file not a good practice for normal text-file I/O

The Tin Man hits it right. I'd also like to add:

• In many cases, reading the entire file into memory is not tractable (because either the file is too big, or the string manipulations have exponential O() space)

• Often times, you cannot anticipate the file size (special case of above)

• You should always try to be cognizant of memory usage, and reading all the file in at once (even in trivial situations) is not good practice if an alternative option exists (eg, line-by-line). I know from experience that VBS is horrible in this sense and one is forced into manipulating files through the command line.

This concept applies not just for files, but for any other process where your memory size grows quickly and you have to handle each iteration (or line) at a time. Generator functions help you out by handling the process, or line read, one by one so as to not work with all the data in memory.

As an aside/extra, Python is very smart in reading files in and its open() method is designed to read line-by-line by default. See "Improve Your Python: 'yield' and Generators Explained" which explains a good use case example for generator functions.