Reading large files from end

Question

Can I read a file in PHP from my end, for example if I want to read last 10-20 lines?

And, as I read, if the size of the file is more than 10mbs I start getting erro

Answer 1

If your code is not working and reporting an error you should include the error in your posts!

The reason you are getting an error is because you are trying to store the entire contents of the file in PHP's memory space.

The most effiicent way to solve the problem would be as Greenisha suggests and seek to the end of the file then go back a bit. But Greenisha's mecanism for going back a bit is not very efficient.

Consider instead the method for getting the last few lines from a stream (i.e. where you can't seek):

while (($buffer = fgets($handle, 4096)) !== false) {
    $i1++;
    $content[$i1]=$buffer;
    unset($content[$i1-$lines_to_keep]);
}

So if you know that your max line length is 4096, then you would:

if (4096*lines_to_keep<filesize($input_file)) {
   fseek($fp, -4096*$lines_to_keep, SEEK_END);
}

Then apply the loop I described previously.

Since C has some more efficient methods for dealing with byte streams, the fastest solution (on a POSIX/Unix/Linux/BSD) system would be simply:

$last_lines=system("last -" . $lines_to_keep . " filename");

Answer 2

If you know about how long the lines are, you can avoid a lot of the black magic and just grab a chunk of the end of the file.

I needed the last 15 lines from a very large log file, and altogether they were about 3000 characters. So I just grab the last 8000 bytes to be safe, then read the file as normal and take what I need from the end.

    $fh = fopen($file, "r");
    fseek($fh, -8192, SEEK_END);
    $lines = array();
    while($lines[] = fgets($fh)) {}

This is possibly even more efficient than the highest rated answer, which reads the file character by character, compares each character, and splits based on newline characters.

Answer 3

It's not pure PHP, but the common solution is to use the tac command which is the revert of cat and loads the file in reverse. Use exec() or passthru() to run it on the server and then read the results. Example usage:

<?php
$myfile = 'myfile.txt';
$command = "tac $myfile > /tmp/myfilereversed.txt";
exec($command);
$currentRow = 0;
$numRows = 20;  // stops after this number of rows
$handle = fopen("/tmp/myfilereversed.txt", "r");
while (!feof($handle) && $currentRow <= $numRows) {
   $currentRow++;
   $buffer = fgets($handle, 4096);
   echo $buffer."<br>";
}
fclose($handle);
?>

Answer 4

It depends how you interpret "can".

If you wonder whether you can do this directly (with PHP function) without reading the all the preceding lines, then the answer is: No, you cannot.

A line ending is an interpretation of the data and you can only know where they are, if you actually read the data.

If it is a really big file, I'd not do that though. It would be better if you were to scan the file starting from the end, and gradually read blocks from the end to the file.

Update

Here's a PHP-only way to read the last n lines of a file without reading through all of it:

function last_lines($path, $line_count, $block_size = 512){
    $lines = array();

    // we will always have a fragment of a non-complete line
    // keep this in here till we have our next entire line.
    $leftover = "";

    $fh = fopen($path, 'r');
    // go to the end of the file
    fseek($fh, 0, SEEK_END);
    do{
        // need to know whether we can actually go back
        // $block_size bytes
        $can_read = $block_size;
        if(ftell($fh) < $block_size){
            $can_read = ftell($fh);
        }

        // go back as many bytes as we can
        // read them to $data and then move the file pointer
        // back to where we were.
        fseek($fh, -$can_read, SEEK_CUR);
        $data = fread($fh, $can_read);
        $data .= $leftover;
        fseek($fh, -$can_read, SEEK_CUR);

        // split lines by \n. Then reverse them,
        // now the last line is most likely not a complete
        // line which is why we do not directly add it, but
        // append it to the data read the next time.
        $split_data = array_reverse(explode("\n", $data));
        $new_lines = array_slice($split_data, 0, -1);
        $lines = array_merge($lines, $new_lines);
        $leftover = $split_data[count($split_data) - 1];
    }
    while(count($lines) < $line_count && ftell($fh) != 0);
    if(ftell($fh) == 0){
        $lines[] = $leftover;
    }
    fclose($fh);
    // Usually, we will read too many lines, correct that here.
    return array_slice($lines, 0, $line_count);
}

Answer 5

For Linux you can do

$linesToRead = 10;
exec("tail -n{$linesToRead} {$myFileName}" , $content); 

You will get an array of lines in $content variable

Pure PHP solution

$f = fopen($myFileName, 'r');

    $maxLineLength = 1000;  // Real maximum length of your records
    $linesToRead = 10;
    fseek($f, -$maxLineLength*$linesToRead, SEEK_END);  // Moves cursor back from the end of file
    $res = array();
    while (($buffer = fgets($f, $maxLineLength)) !== false) {
        $res[] = $buffer;
    }

    $content = array_slice($res, -$linesToRead);

Answer 6

Here is another solution. It doesn't have line length control in fgets(), you can add it.

/* Read file from end line by line */
$fp = fopen( dirname(__FILE__) . '\\some_file.txt', 'r');
$lines_read = 0;
$lines_to_read = 1000;
fseek($fp, 0, SEEK_END); //goto EOF
$eol_size = 2; // for windows is 2, rest is 1
$eol_char = "\r\n"; // mac=\r, unix=\n
while ($lines_read < $lines_to_read) {
    if (ftell($fp)==0) break; //break on BOF (beginning...)
    do {
            fseek($fp, -1, SEEK_CUR); //seek 1 by 1 char from EOF
        $eol = fgetc($fp) . fgetc($fp); //search for EOL (remove 1 fgetc if needed)
        fseek($fp, -$eol_size, SEEK_CUR); //go back for EOL
    } while ($eol != $eol_char && ftell($fp)>0 ); //check EOL and BOF

    $position = ftell($fp); //save current position
    if ($position != 0) fseek($fp, $eol_size, SEEK_CUR); //move for EOL
    echo fgets($fp); //read LINE or do whatever is needed
    fseek($fp, $position, SEEK_SET); //set current position
    $lines_read++;
}
fclose($fp);