Using named pipes with bash - Problem with data loss

Question

Did some search online, found simple \'tutorials\' to use named pipes. However when I do anything with background jobs I seem to lose a lot of data.

[[Edit: found a

Answer 1

run say 10 jobs in parallel at most and queue the rest for later processing, but reliably know that they do run

You can do this with GNU Parallel. You will not need a this scripting.

http://www.gnu.org/software/parallel/man.html#options

You can set max-procs "Number of jobslots. Run up to N jobs in parallel." There is an option to set the number of CPU cores you want to use. You can save the list of executed jobs to a log file, but that is a beta feature.