Use Bash variable within SLURM sbatch script

 ̄綄美尐妖づ 提交于 2019-12-08 15:00:54

问题


I'm trying to obtain a value from another file and use this within a SLURM submission script. However, I get an error that the value is non-numerical, in other words, it is not being dereferenced.

Here is the script:

#!/bin/bash
# This reads out the number of procs based on the decomposeParDict                                                                                          
numProcs=`awk '/numberOfSubdomains/ {print $2}' ./meshModel/decomposeParDict`
echo "NumProcs = $numProcs"

#SBATCH --job-name=SnappyHexMesh                                                                                                                            
#SBATCH --output=./logs/SnappyHexMesh.log                                                                                                                   
#                                                                                                                                                           
#SBATCH --ntasks=`$numProcs`                                                                                                                                
#SBATCH --time=240:00                                                                                                                                       
#SBATCH --mem-per-cpu=4000                                                                                                                                  

#First run blockMesh                                                                                                                                        
blockMesh

#Now decompose the mesh                                                                                                                                     
decomposePar

#Now run snappy in parallel                                                                                                                                 
mpirun -np $numProcs snappyHexMesh -parallel -overwrite

When I run this as a normal Bash shell script, it prints out the number of procs correctly and makes the correct mpirun call. Thus the awk command parses out the number of procs correctly and the variable is dereferenced as expected.

However, when I submit this to SLURM using:

sbatch myScript.sh

I get the error:

sbatch: error: Invalid numeric value "`$numProcs`" for number of tasks.

Can anyone help with this?


回答1:


This won't work. What happens when you run

sbatch myscript.sh

is that slurm parses the script for those special #SBATCH lines, generates a job record, stores the batch script somewhere. The batch script is executed only later when the job runs.

So you need to structure you workflow in a slightly different way, and first calculate the number of procs you need before submitting the job. Note that you can use something like

sbatch -n $numProcs myscript.sh

, you don't need to autogenerate the script (also, mpirun should be able to get the number of procs in your allocation automatically, no need to use "-np").



来源:https://stackoverflow.com/questions/24508040/use-bash-variable-within-slurm-sbatch-script

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!