Checking for status of qsub jobs running within shell script

问题

I have been given a c shell script that launches 800 individual qsubs for a sample. I need to run this script on more than 500 samples (listed in samples.txt). To automate the process, I thought about running the script (named SrchDriver) using the following bash shell script:

#!/bin/sh

for item in $(cat samples.txt)
do
  (cd dir_"$item"/MAPGAPS && SrchDriver "$item"_Out 3)
done

This script would launch the SrchDriver script for all samples one right after another which would result in too many jobs on the server at one time. I would like to run only one sample at a time by waiting for all qsubs to finish for a particular sample.

What is the best way to put in a check for running/waiting jobs for a sample and holding the launch of the Srchdriver script for additional samples until all jobs are finished for the current sample?

I was thinking to first wait for 30 seconds and then check status of the qsubs (name of jobs is mapgaps). Next, I wanted to use a while loop to check the status every 30 seconds. Once the status is no longer 0, then proceed to the next sample. Would this be correct?

sleep 30
qstat | grep mapgaps &> /dev/null

while [ $? -eq 0 ]; 
do
    sleep 30
    qstat | grep mapgaps &> /dev/null
done;

If correct, how would I combine it with my for-loop? Would the following code below be correct?

#!/bin/sh

for item in $(cat samples.txt)
do
    (cd dir_"$item"/MAPGAPS && SrchDriver "$item"_Out 3)

    sleep 30
    qstat | grep mapgaps &> /dev/null
    status=$?

    while [ $status = 0 ]
    do
        sleep 30
        qstat | grep mapgaps &> /dev/null
        status=$?
    done

done

Thanks in advance for help. Please let me know if more information is needed.

回答1:

Your script should work as is, indeed. The logic is sound and the syntax is correct.

A small improvement: the while statement can take the return status of a command directly, without using $?, so you could write your script like this:

#!/bin/sh

for item in $(cat samples.txt)
do
    (cd dir_"$item"/MAPGAPS && SrchDriver "$item"_Out 3)

    sleep 30
    while qstat | grep mapgaps &> /dev/null
    do
        sleep 30
    done
done

来源：https://stackoverflow.com/questions/57424289/checking-for-status-of-qsub-jobs-running-within-shell-script

标签

bash

shell

loops

qsub