bash: start multiple chained commands in background

Question

I\'m trying to run some commands in paralel, in background, using bash. Here\'s what I\'m trying to do:

forloop {
  //this part is actually written in perl
          


        

Answer 1

I stumbled upon this thread here and decided to put together a code snippet to spawn chained statements as background jobs. I tested this on BASH for Linux, KSH for IBM AIX and Busybox's ASH for Android, so I think it's safe to say it works on any Bourne-like shell.

processes=0;
for X in `seq 0 10`; do
   let processes+=1;
   { { echo Job $processes; sleep 3; echo End of job $processes; } & };
   if [[ $processes -eq 5 ]]; then
      wait;
      processes=0;
   fi;
done;

This code runs a number of background jobs up to a certain limit of concurrent jobs. You can use this, for example, to recompress a lot of gzipped files with xz without having a huge bunch of xz processes eat your entire memory and make your computer throw up: in this case, you use * as the for's list and the batch job would be gzip -cd "$X" | xz -9c > "${X%.gz}.xz".