How to hold up a script until a slurm job (start with srun) is completely finished?

Question

I am running a job array with SLURM, with the following job array script (that I run with sbatch job_array_script.sh [args]:

#!/bin/bash

#SBATC         


        

Answer 1

You can add the flag --wait to sbatch.

Check the manual page of sbatch for information about --wait.

Answer 2

You can use --wait option in sbatch in combination with wait in bash to send jobs off to the cluster, pause script execution until those are complete, and then continue. E.g.

#!/bin/bash
set -e
date

for((i=0; i<5; i++)); do
    sbatch -W --wrap='echo "hello from $SLURM_ARRAY_TASK_ID eat $VAR"; sleep 10' &
done;
wait

date
echo "I am finished"

Answer 3

You can use the wait bash command. It will wait until whatever lines of code above are finished. Thus you script should look like this:

#!/bin/bash

#SBATCH ... other options ...

#SBATCH --array=0-1000%200

srun ./job_slurm_script.py $1 $2 $3 $4

wait

echo 'open' > status_file.txt