PHP flock() alternative

Question

PHP\'s documentation page for flock() indicates that it\'s not safe to use under IIS. If I can\'t rely on flock under all circumstances, is there another way I

Answer 1

You can implement a filelock - unlock pattern around your read/write operations based on mkdir, since that is atomic and pretty fast. I've stress tested this and unlike mgutt did not find a bottleneck. You have to take care of deadlock situations though, which is probably what mgutt experienced. A dead lock is when two lock attempts keep waiting on each other. It can be remedied by a random interval on the lock attempts. Like so:

// call this always before reading or writing to your filepath in concurrent situations
function lockFile($filepath){
   clearstatcache();
   $lockname=$filepath.".lock";
   // if the lock already exists, get its age:
   $life=@filectime($lockname);
   // attempt to lock, this is the really important atomic action:
   while (!@mkdir($lockname)){
         if ($life)
            if ((time()-$life)>120){
               //release old locks
               rmdir($lockname);
               $life=false;
         }
         usleep(rand(50000,200000));//wait random time before trying again
   }
}

Then work on your file in filepath and when you're done, call:

function unlockFile($filepath){
   $unlockname= $filepath.".lock";   
   return @rmdir($unlockname);
}

I've chosen to remove old locks, well after the maximum PHP execution time in case a script exits before it has unlocked. A better way would be to remove locks always when the script fails. There is a neat way for this, but I have forgotten.

Answer 2

I appreciate this question is a few years old but I kinda felt that a working example/replacement for flock might be worth building. I've based this on the other answers but for someone who is purely looking to replace the flock functionality (rather than write a file at the same time (although this does reflect the PHP manual flock example)) I believe the following will suffice

function my_flock ($path,$release = false){
    if ($release){
        @rmdir($path);
    } else {
        return !file_exists($path) && @mkdir($path);
    }
}

Answer 3

Based on mkdir:

// call this always before reading or writing to your filepath in concurrent situations
function lockFile($filepath){
   clearstatcache();
   $lockname=$filepath.".lock";
   // if the lock already exists, get its age:
   $life=@filectime($lockname);
   // attempt to lock, this is the really important atomic action:
   while (!@mkdir($lockname)){
     if ($life)
        if ((time()-$life)>120){
           //release old locks
           rmdir($lockname);
     }else $life=@filectime($lockname);
     usleep(rand(50000,200000));//wait random time before trying again
   }
}

To avoid deadlock when one script in case a script exits before it has unlocked and one (or more scripts) at the same time have no result on $life=@filectime($lockname); because all scripts starts at the same time and then directory isn't created yet. To unlock then call:

function unlockFile($filepath){
   $unlockname= $filepath.".lock";   
  return @rmdir($unlockname);
}

Answer 4

None of these methods are fully atomic.

I have made some tests, confirming this.

The code for T7, using 7 files named by their size in kB:

clearstatcache();
$_DEBUG_ = false;

echo "Lock and flush tester.".time()."<br>";
$time_constant = 1570787996;
die; // Remove this line when you set time_constant 

while ( time()<$time_constant )
 {
 usleep(500);
 }


function test($n, $p, $_DEBUG_){
//  $delay_multiplier = $n*2.5;
  $sname = "$n";    // source
  $tname = "$n.txt";// target
  echo "<h4>$n at ".time()."</h4>";
  for ($i = 0; $i<50; $i++ ){
    $start = microtime(true);
    clearstatcache(); // needed for filesize and touch    
    $st = stat("$sname");
    $original_size = $st['size'];
    if ( $_DEBUG_ )
      echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
    $fsize = filesize($sname);
    if ( $original_size <> $fsize )
      die("; fsize total FAILTURE; ");
    if ($fsize === 0)
     echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";    
    else
      {
      // READ OPERATION AND LOCK FOR SHARE
       $locked = false;     
       for ($c = 0; !$locked; $c++):      
         if ( $c > 400)
           break;
         $fp = fopen($sname, "r");
         $locked = flock($fp, LOCK_SH);
         if ($locked)
           break;
         else
           {
           echo "failed to get LOCK_SH;<br>";
           usleep(5000);
           }
       endfor;
       $s = fread($fp, $fsize );
       $success = flock($fp, LOCK_UN);
       if ( $success === false  )
         die("; r flock release failed; ");
       $success = fclose($fp);
       if ( $success === false  )
         die("; fclose failed; ");
       // 10 - loaded data , $p - broser
       if ( $success )
         { 
         $result = touch("$sname",strlen($s),$p);
         if ( $_DEBUG_ )
            echo "; TOUCH: $result;";
         }
       else
         die("fclose FAIL.");
       if ( strlen($s)<60 ) 
          echo "*$s LENGTH:".strlen($s)."<br>";
      }
    clearstatcache();
    $st = stat("$tname");                               
    if ( $_DEBUG_ )
      echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";

    // WRITE OPERATION WITH LOC_EX
    $fp = fopen($tname, "w");
    $locked = false; 
    /*
    // TOTO NEMÁ VLIV NA ZAMKNUTÍ
    for ($c = 0; !$locked; $c++ ):
      $c++;
      if ( $c > 400)
        break;
      $locked = flock($fp, LOCK_EX);
      if ($locked)
        break;
      else
        {
        echo "failed to get LOCK_EX;<br>";
        usleep(5000);
        }
    endfor;
    */
    $locked = flock($fp, LOCK_EX);
    if ( $locked ) {  // acquire an exclusive lock
        $success = fwrite($fp, $s);
        if ( $success === false)
          echo "; w FAILED;";
        else
          if ( $_DEBUG_ )
                echo " $success B written; ";
        $success = fflush($fp);// flush output before releasing the lock
        if ( $success === false ) 
          echo "; flush FAILED; ";
        $success = flock($fp, LOCK_UN);    // release the lock
        if ( $success === false ) 
          echo "; release FAILED; ";
        $success = fclose($fp);
        if ( $success === false ) 
          echo "; fclose FAILED; ";
        clearstatcache(); // needed for filesize and touch
        $fsize = filesize($tname);
        if ($original_size>$fsize)
            {
            echo "; <b>WRITE FAILED, restoring</b>;";
            $original_fname = "$n";
            $result = copy($original_fname, $tname);
            if ($result == false )
              die(" <b>TOTAL FAILTURE: copy failed.</b>");
            else
              echo " <b>RESTORED</b>;";
            }
        else
        {
          if ($fsize === 0)
           echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";    
          if ( $success ) 
              touch("$tname",$fsize,$p);
        }
    } else {
        echo "Couldn't get the lock!";
    }
     $time_elapsed_secs = microtime(true) - $start;
     //usleep( $delay_multiplier + $n*rand(2,6) ); 
     if ( $time_elapsed_secs === 0 )
       echo " FAILED ";
    echo "time: $time_elapsed_secs s<br>"; 
  }
}
// headers to identify originator of the request
switch ( $_SERVER['HTTP_USER_AGENT'] ):
  // FF 1:
  case "Mozilla/5.0 (Windows NT 5.1;) Gecko": 
    $p = 1; break;
  // Chrome:
  case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit Chrome  Safari":
    $p = 2; break;
  // OPERA:
  case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit Chrome Safari":  
    $p = 3; break;
endswitch;

copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");

test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);

The code for T8 (mkdir lock test):

clearstatcache();
$_DEBUG_ = false;

echo "Atomicity tester.".time()."<br>";
$time_constant = 1570787996;
die; // Remove this line when you set time_constant 

while ( time()<$time_constant )
 {
 usleep(500);
 }

/*
c is counter for optimalization
first call must have c = 0;
*/
function atomicFuse($n, $c, $disableDelay = false){
  $start = false;
  if ( !file_exists("$n.t") ) 
   $start = mkdir("$n.t");
  if ( !$disableDelay ){
    if ( $start == false )
     {
     $n = $n*30;
     switch($c):      // Delay example increase:
       case 0: break; // 0,01569 total
       case 1: break; // 0,03138 total
       case 2: $n = $n*2; break; // 0,06276 total
       case 3: $n = $n*4; break; // 0,12552 total
       // case 4: You need at least *6 or *8 to get out of problems with extrem times
       case 4: $n = $n*8; break; // 0,25104 t.(upper limit)
       // In case of heavy traffic:
       case 5: $n = $n*8; break; // 0,36087 total extrem
       case 6: $n = $n*10; break; // 0,51777 total extrem
       case 7: $n = $n*20; break; // 1,03554 total extrem
       default: $n = $n*8; break;
     endswitch;
     usleep($n);
     echo ($n)."<br>";
     }
    }
  return $start;
}
function test($n, $p, $_DEBUG_){
  $fp = null;
  $sname = "$n";    // source
  $tname = "$n.txt";// target
  echo "<h4>$n at ".time()."</h4>";
  for ($i = 0; $i<50; $i++ ){
    $start_time = microtime(true);
      {
      $start = atomicFuse($n,0);
      if (!$start) $start = atomicFuse($n,1);
      if (!$start) $start = atomicFuse($n,2);
      if (!$start) $start = atomicFuse($n,3);
      if (!$start) $start = atomicFuse($n,4);
      if (!$start) $start = atomicFuse($n,5);
      if (!$start) $start = atomicFuse($n,6);
      if (!$start) $start = atomicFuse($n,7);
      if (!$start) $start = atomicFuse($n, false);
      if (!$start) echo "<b>Atomicity failed.</b> ";
      if ( $start )
         {
         echo "<b>Atomicity OK.</b> ";
         /////////////////////////////
         // CHECK FILESIZE VALIDITY //
         /////////////////////////////
         clearstatcache(); // needed for filesize and touch    
         $st = stat("$sname");
         $original_size = $st['size'];
         if ( $_DEBUG_ )
           echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
         $fsize = filesize($sname);
         if ( $original_size <> $fsize )
           die("; fsize total FAILTURE; ");
         if ($fsize === 0)
          echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";    
         ///////////////////
         // OPEN THE FILE //
         ///////////////////
         $fp = fopen($sname, "r");
         $s = fread($fp, $fsize );
         $success = fclose($fp);
         if ( $success === false  )
           die("; fclose failed; ");
         // 10 - loaded data, $p - browser
         if ( $success )
           { 
           $result = touch("$sname",strlen($s),$p);
           if ( $_DEBUG_ )
              echo "; TOUCH: $result;";
           }
         else
           die("fclose FAIL.");
         if ( strlen($s)<60 ) 
            echo "*$s LENGTH:".strlen($s)."<br>";
         }  
      }
    if ( $start )
      {
      clearstatcache();
      $st = stat("$tname");                               
      if ( $_DEBUG_ )
        echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";

      // WRITE OPERATION WITH LOC_EX
      $fp = fopen($tname, "w");
      if ( true ) {  // acquire an exclusive lock
          $success = fwrite($fp, $s);
          if ( $success === false)
            echo "; w FAILED;";
          else
            if ( $_DEBUG_ )
                  echo " $success B written; ";
          $success = fflush($fp);// flush output before releasing the lock
          if ( $success === false ) 
            echo "; flush FAILED; ";
          if ( $success === false ) 
            echo "; release FAILED; ";
          $success = fclose($fp);
          if ( $success === false ) 
            echo "; fclose FAILED; ";
          clearstatcache(); // needed for filesize and touch
          $fsize = filesize($tname);
          if ($original_size>$fsize)
              {
              echo "; <b>WRITE FAILED, restoring</b>;";
              $original_fname = "$n";
              $result = copy($original_fname, $tname);
              if ($result == false )
                die(" <b>TOTAL FAILTURE: copy failed.</b>");
              else
                echo " <b>RESTORED</b>;";
              }
          else
            {
              if ($fsize === 0)
               echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";    
              if ( $success ) 
                  touch("$tname",$fsize,$p);
            }
          } else {
              echo "Couldn't get the lock!";
             }
      $success = rmdir("$n.t"); // remove atomic fuse
      if ( $success )
        echo "<h4>DIR REMOVED</h4>";
      else
        echo "<h4>DIR NOT REMOVED</h4>";
      } // start
     else 
       echo "skipped"; 
     $time_elapsed_secs = microtime(true) - $start_time;
     if ( $time_elapsed_secs === 0 )
       echo " FAILED ";
     echo "time: $time_elapsed_secs s<br>"; 
  } // for
}

switch ( $_SERVER['HTTP_USER_AGENT'] ):
  case "": 
    $p = 1; break;
  case "":
    $p = 2; break;
  case "":  
    $p = 3; break;
endswitch;

copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");

test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);

Note: T5-T7 - I did not determinated whether the file damages were made by fflush or fwrite, but it was in these tests, where these error occures.

Note: T8 - Specific problem with this test is, that it often waits too long on begin of a testing block (on the begin of the testing function). There even delays like 7 seconds waiting. But I also tried to remove these numbers and the avarage does not change too much, so the curve of T8 would stay the same after this change. The problem here is that using the delay in a loop is not ideal solution of the problem, it makes the probability of failture even higher. Note, that by "failture" I do not really mean file corruption but skipping of the given atomic task because time out.

Answer 5

My proposal is to use mkdir() instead of flock(). This is a real-world example for reading/writing caches showing the differences:

$data = false;
$cache_file = 'cache/first_last123.inc';
$lock_dir = 'cache/first_last123_lock';
// read data from cache if no writing process is running
if (!file_exists($lock_dir)) {
    // we suppress error messages as the cache file exists in 99,999% of all requests
    $data = @include $cache_file;
}
// cache file not found
if ($data === false) {
    // get data from database
    $data = mysqli_fetch_assoc(mysqli_query($link, "SELECT first, last FROM users WHERE id = 123"));
    // write data to cache if no writing process is running (race condition safe)
    // we suppress E_WARNING of mkdir() because it is possible in 0,001% of all requests that the dir already exists after calling file_exists()
    if (!file_exists($lock_dir) && @mkdir($lock_dir)) {
        file_put_contents($cache_file, '<?php return ' . var_export($data, true) . '; ?' . '>')) {
        // remove lock
        rmdir($lock_dir);
    }
}

Now, we try to achieve the same with flock():

$data = false;
$cache_file = 'cache/first_last123.inc';
// we suppress error messages as the cache file exists in 99,999% of all requests
$fp = @fopen($cache_file, "r");
// read data from cache if no writing process is running
if ($fp !== false && flock($fp, LOCK_EX | LOCK_NB)) {
    // we suppress error messages as the cache file exists in 99,999% of all requests
    $data = @include $cache_file;
    flock($fp, LOCK_UN);
}
// cache file not found
if (!is_array($data)) {
    // get data from database
    $data = mysqli_fetch_assoc(mysqli_query($link, "SELECT first, last FROM users WHERE id = 123"));
    // write data to cache if no writing process is running (race condition safe)
    $fp = fopen($cache_file, "c");
    if (flock($fp, LOCK_EX | LOCK_NB)) {
        ftruncate($fp, 0);
        fwrite($fp, '<?php return ' . var_export($data, true) . '; ?' . '>');
        flock($fp, LOCK_UN);
    }
}

The important part is LOCK_NB to avoid blocking all consecutive requests:

It is also possible to add LOCK_NB as a bitmask to one of the above operations if you don't want flock() to block while locking.

Without it, the code would produce a huge bottleneck!

An additional important part is if (!is_array($data)) {. This is because $data could contain:

1. array() as a result of the db query
2. false of the failing include
3. or an empty string (race condition)

The race condition happens if the first visitor executes this line:

$fp = fopen($cache_file, "c");

and another visitor executes this line one millisecond later:

if ($fp !== false && flock($fp, LOCK_EX | LOCK_NB)) {

This means the first visitor creates the empty file, but the second visitor creates the lock and so include returns an empty string.

So you saw many pitfalls that can be avoided through using mkdir() and its 7x faster, too:

$filename = 'index.html';
$loops = 10000;
$start = microtime(true);
for ($i = 0; $i < $loops; $i++) {
    file_exists($filename);
}
echo __LINE__ . ': ' . round(microtime(true) - $start, 5) . PHP_EOL;
$start = microtime(true);
for ($i = 0; $i < $loops; $i++) {
    $fp = @fopen($filename, "r");
    flock($fp, LOCK_EX | LOCK_NB);
}
echo __LINE__ . ': ' . round(microtime(true) - $start, 5) . PHP_EOL;

result:

file_exists: 0.00949
fopen/flock: 0.06401

P.S. as you can see I use file_exists() in front of mkdir(). This is because my tests (German) resulted bottlenecks using mkdir() alone.

Answer 6

Here is my "PHP flock() alternative" - build on mkdir().

The idea to do it with mkdir() came from here and here.

My version

• checks if I already have got lock-access. It also prevents blocking myself if I create and use the class multiple times for the same basedir.name
• checks if my locking-file, with which I am asking for lock-access, was created
• lets me get lock-access in the order I came to ask for it
• stops waiting and looping if it could not get lock-access in the time I specified
• removes dead lock-files (= files where the SID of the PID does not exist any more)

You can use the PHP-class like this:

//$dir        (string) = base-directory for the lock-files (with 'files' I mean directories => mode 0644)
// 2       (float/int) = time to wait for lock-access before returning unsuccessful (default is 0 <= try once and return)
//'.my_lock'  (string) = the way you want to name your locking-dirs (default is '.fLock')
$lock = new FileLock($dir, 2, '.my_lock');

//start lock - a locking directory will be created looking like this:
//$dir/.my_lock-1536166146.4997-22796
if ($lock->lock()) {
    //open your file - modify it - write it back
} else { /* write alert-email to admin */ }

//check if I had locked before
if ($lock->is_locked) { /* do something else with your locked file */ }

//unlock - the created dir will be removed (rmdir)
$lock->unlock();

Here is the working class:

//build a file-locking class
define('LOCKFILE_NONE', 0);
define('LOCKFILE_LOCKED', 1);
define('LOCKFILE_ALREADY_LOCKED', 2);
define('LOCKFILE_ALREADY_LOCKED_IN_OTHER_CLASS', 3);
define('LOCKFILE_FAILED_TO_OBTAIN_LOCK', false);
define('LOCKFILE_FAILED_TO_OBTAIN_LOCK_BY_TIMEOUT', '');


class FileLock {
    //FileLock assumes that there are no other directories or files in the
    //lock-base-directory named "$name-(float)-(int)"
    //FileLock uses mkdir() to lock. Why?
    //- mkdir() is atomic, so the lock is atomic and faster then saving files.
    //  Apparently it is faster than flock(), that requires several calls to the
    //  file system.
    //- flock() depends on the system, mkdir() works everywhere.

    private static $locked_memory = array();

    public function __construct($lockbasedir, $wait_sec=0, $name='.fLock') {
        $this->lockbasedir = (string)$lockbasedir;
        $this->wait        = (float)$wait_sec;
        $this->name        = (string)$name;

        $this->pid         = (int)getmypid();

        //if this basedir.name was locked before and is still locked don't try to lock again
        $this->is_locked   = empty(self::$locked_memory[$this->lockbasedir . $this->name]) ? LOCKFILE_NONE : LOCKFILE_ALREADY_LOCKED;
    }

    public function lock() {
        if ($this->is_locked) return $this->is_locked;

        $break_time = microtime(true);

        //create the directory as lock-file NOW
        $this->lockdir = "{$this->name}-" . number_format($break_time, 4, '.', '') . "-{$this->pid}";
        @mkdir("{$this->lockbasedir}/{$this->lockdir}", 0644);

        $break_time += $this->wait;

        //try to get locked
        while ($this->wait == 0 || microtime(true) < $break_time) {

            //get all locks with $this->name
            $files = preg_grep("/^{$this->name}-\d+\.\d+-\d+$/", scandir($this->lockbasedir));

            //since scandir() is sorted asc by default
            //$first_file is the next directory to obtain lock
            $first_file = reset($files);

            if (!$first_file) {
                //no lock-files at all
                return $this->is_locked = LOCKFILE_FAILED_TO_OBTAIN_LOCK;
            } elseif ($first_file == $this->lockdir) {
                //Its me!! I'm getting locked :)
                self::$locked_memory[$this->lockbasedir . $this->name] = 1;
                return $this->is_locked = LOCKFILE_LOCKED;
            } elseif (preg_match("/^{$this->name}-\d+\.\d+-{$this->pid}$/", $first_file)) {
                //my process-ID already locked $this->name in another class before
                rmdir("{$this->lockbasedir}/{$this->lockdir}");
                $this->lockdir = $first_file;
                self::$locked_memory[$this->lockbasedir . $this->name] = 1;
                return $this->is_locked = LOCKFILE_ALREADY_LOCKED_IN_OTHER_CLASS;
            }

            //missing lock-file for this job
            if (array_search($this->lockdir, $files) === false) return LOCKFILE_FAILED_TO_OBTAIN_LOCK;

            //run only once
            if ($this->wait == 0) break;

            //check if process at first place has died
            if (!posix_getsid(explode('-', $first_file)[2])) {
                //remove dead lock
                @rmdir("{$this->lockbasedir}/$first_file");
            } else {
                //wait and try again after 0.1 seconds
                usleep(100000);
            }
        }

        return $this->is_locked = LOCKFILE_FAILED_TO_OBTAIN_LOCK_BY_TIMEOUT;
    }

    public function unlock($force=false) {
        if ($force || $this->is_locked == 1) {
            rmdir("{$this->lockbasedir}/{$this->lockdir}");
            self::$locked_memory[$this->lockbasedir . $this->name] = $this->is_locked = LOCKFILE_NONE;
        }
    }
}