Bash script with multiple papermill commands does not fail on notebook errors

断了今生、忘了曾经 提交于 2021-02-16 20:55:12

问题


I have a refresh_data.sh file which contains multiple papermill commands, for example:

papermill notebook_1.ipynb output_1.ipynb -p start "2017-12-01" -p date "2017-12-31"
papermill notebook_2.ipynb output_2.ipynb -p start "2018-01-01" -p date "2018-01-31"

If I get an error while it is running the first notebook, the process continues executing the second one.

In other words, an error in one of the notebooks doesn't "break" the overall script.

As far as I remember with normal python scripts if there is an error in one of the commands within the bash script it breaks the execution of the entire script.

What is the standard behaviour of a bash script in this case? Can I change it so that it stops as soon as there is an error?


回答1:


If your bash script is configured with: set -e it will fail if a command errors out:

Automatic exit from bash shell script on error

#!/bin/bash
set -e
# Any subsequent(*) commands which fail will cause the shell script to exit immediately

You can run papermill using:

--log-output to get more information about why your notebook fail.

papermill "${INPUT_NOTEBOOK_PATH}" "${OUTPUT_NOTEBOOK_PATH}" --log-output

To capture notebook execution result you can always capture the result of any previous command using $?:

  papermill "${INPUT_NOTEBOOK_PATH}" "${OUTPUT_NOTEBOOK_PATH}" --log-output
  notebook_result=$?
  if [[ ${notebook_result} -eq 0 ]]; then
    echo "All good"
  else
    echo $notebook_result
  fi


来源:https://stackoverflow.com/questions/55478353/bash-script-with-multiple-papermill-commands-does-not-fail-on-notebook-errors

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!