How can I set the file-read buffer size in Perl to optimize it for large files?

Question

I understand that both Java and Perl try quite hard to find a one-size-fits all default buffer size when reading in files, but I find their choices to be increasingly antiqu

Answer 1

You can affect the buffering if you're running on an OS that supports setvbuf; see the documentation for IO::Handle.

If you're using perl v5.10 or later then there is no need to explicitly create an IO::Handle object as described in the documentation, as all file handles are implicitly blessed into IO::Handle objects since that release.

use 5.010;
use strict;
use warnings;

use autodie;

use IO::Handle '_IOLBF';

open my $handle, '<:utf8', 'foo';

my $buffer;
$handle->setvbuf($buffer, _IOLBF, 0x10000);

while ( my $line = <$handle> ) {
    ...
}