How to get the PID of a process in a pipeline

Question

Consider the following simplified example:

my_prog|awk \'...\' > output.csv &
my_pid=\"$!\" #Gives the PID for awk instead of for my_prog
sleep 10
kill $my         


        

Answer 1

Based on your comment, I still can't see why you'd prefer killing my_prog to having it complete in an orderly fashion. Ten seconds is a pretty arbitrary measurement on a multiprocessing system whereby my_prog could generate 10k lines or 0 lines of output depending upon system load.

If you want to limit the output of my_prog to something more determinate try

my_prog | head -1000 | awk

without detaching from the shell. In the worst case, head will close its input and my_prog will get a SIGPIPE. In the best case, change my_prog so it gives you the amount of output you want.

added in response to comment:

In so far as you have control over my_prog give it an optional -s duration argument. Then somewhere in your main loop you can put the predicate:

if (duration_exceeded()) {
    exit(0);
}

where exit will in turn properly flush the output FILEs. If desperate and there is no place to put the predicate, this could be implemented using alarm(3), which I am intentionally not showing because it is bad.

The core of your trouble is that my_prog runs forever. Everything else here is a hack to get around that limitation.