waiting for all pids to exit in php

Question

My issue is this. I am forking a process so that I can speed up access time to files on disk. I store any data from these files in a tmp file on local desk. ideally, after all processes have finished, I need to access that tmp file and get that data into an array. I then unlink the tmp file as it is no longer needed. My problem is that it would seem that pcntl_wait() does not acutally wait until all child processes are done before moving on to the final set of operations. So I end up unlinking that file before some random process can finish up.

I can't seem to find a solid way to wait for all processes to exit cleanly and then access my data.

    $numChild       = 0;    
    $maxChild       = 20;   // max number of forked processes.

    // get a list of "availableCabs"

    foreach ($availableCabs as $cab) {

            // fork the process
            $pids[$numChild] = pcntl_fork();

            if (!$pids[$numChild]) {

                    // do some work     
                    exit(0);

            } else {

                    $numChild++;
                    if ($numChild == $maxChild) {

                            pcntl_wait($status);
                            $numChild--;

                    } 

            } // end fork   

    }

    // Below is where things fall apart. I need to be able to print the complete serialized data. but several child processes don't actually exit before i unlink the file.

    $dataFile = fopen($pid, 'r');

    while(($values = fgetcsv($dataFile,',')) !== FALSE) {
            $fvalues[] = $values;
    }

    print serialize($fvalues);

    fclose($dataFile);
    unlink($file);      

please note that i'm leaving a lot of code out regarding what i'm actually doing, if we need that posted thats not issue.

Answer 1

Try restructuring you code so that you have two loops - one that spawns processes and one that waits for them to finish. You should also use pcntl_waitpid() to check for specific process IDs, rather than the simple child counting approach you are currently using.

Something like this:

<?php

  $maxChildren = 20;   // Max number of forked processes
  $pids = array();     // Child process tracking array

  // Get a list of "availableCabs"

  foreach ($availableCabs as $cab) {

    // Limit the number of child processes
    // If $maxChildren or more processes exist, wait until one exits
    if (count($pids) >= $maxChildren) {
      $pid = pcntl_waitpid(-1, $status);
      unset($pids[$pid]); // Remove PID that exited from the list
    }

    // Fork the process
    $pid = pcntl_fork();

    if ($pid) { // Parent

      if ($pid < 0) {
        // Unable to fork process, handle error here
        continue;
      } else {
        // Add child PID to tracker array
        // Use PID as key for easy use of unset()
        $pids[$pid] = $pid;
      }

    } else { // Child

      // If you aren't doing this already, consider using include() here - it
      // will keep the code in the parent script more readable and separate
      // the logic for the parent and children

      exit(0);

    }

  }

  // Now wait for the child processes to exit. This approach may seem overly
  // simple, but because of the way it works it will have the effect of
  // waiting until the last process exits and pretty much no longer
  foreach ($pids as $pid) {
    pcntl_waitpid($pid, $status);
    unset($pids[$pid]);
  }

  // Now the parent process can do it's cleanup of the results