How to create threads in Perl?

Question

I have got easy Perl script where I have got a BIG loop and inside this I invoke more or less million times function my_fun(). I would like to create pool of th

Answer 1

Take a look at the threads documentation.

Answer 2

We can't give you the fastest way, since that depends on the work, and you didn't tell us what the work is.

But you did ask about threads, so I'll give you the foundation of a threaded application. Here is a worker model. It's robust, maintainable and extendable.

use threads;
use Thread::Queue qw( );   # Version 3.01+ required

my $NUM_WORKERS = 5;

sub worker {
   my ($job) = @_;
   ...
}

my $q = Thread::Queue->new();
my @workers;
for (1..$NUM_WORKERS) {
   push @workers, async {
      while (defined(my $job = $q->dequeue())) {
         worker($job);
      }
   };
}

$q->enqueue($_) for @jobs;        # Send work
$q->end();                        # Tell workers they're done.
$_->join() for @workers;          # Wait for the workers to finish.

This is a basic flow (one-directional), but it's easy to make bi-directional by adding a response queue.

This uses actual threads, but you can switch to using processes by switching use threads; to use forks;.

Parallel::ForkManager can also be used to provide a worker model, but it's continually creating new processes instead of reusing them. This does allow it to handle child death easily, though.

Ref: Thread::Queue (or Thread::Queue::Any)

Answer 3

Have a look at Parallel::ForkManager. It's using fork, not threads, but it should get your job done very simply.

Example lifted from the docs and slightly altered:

use Parallel::ForkManager;

my $pm = Parallel::ForkManager->new(5); # number of parallel processes

for my $i (0 .. 999999) {
  # Forks and returns the pid for the child:
  my $pid = $pm->start and next;

  #... do some work with $data in the child process ...
  my_fun();

  $pm->finish; # Terminates the child process
}

$pm->wait_all_children;