Unit Testing bash scripts

Question

We have a system that has some bash scripts running besides Java code. Since we are trying to Test Everything That Could Possibly Break, and those bash scripts may break, we

Answer 1

I can't believe no one talked about OSHT! It's compatible with both TAP and JUnit, it's pure shell (that is, no other languages involved), it works standalone too, and it's simple and direct.

Testing looks like this (snippets taken from the project page):

#!/bin/bash
. osht.sh

# Optionally, indicate number of tests to safeguard against abnormal exits
PLAN 13

# Comparing stuff
IS $(whoami) != root
var="foobar"
IS "$var" =~ foo
ISNT "$var" == foo

# test(1)-based tests
OK -f /etc/passwd
NOK -w /etc/passwd

# Running stuff
# Check exit code
RUNS true
NRUNS false

# Check stdio/stdout/stderr
RUNS echo -e 'foo\nbar\nbaz'
GREP bar
OGREP bar
NEGREP . # verify empty

# diff output
DIFF <<EOF
foo
bar
baz
EOF

# TODO and SKIP
TODO RUNS false
SKIP test $(uname -s) == Darwin

A simple run:

$ bash test.sh
1..13
ok 1 - IS $(whoami) != root
ok 2 - IS "$var" =~ foo
ok 3 - ISNT "$var" == foo
ok 4 - OK -f /etc/passwd
ok 5 - NOK -w /etc/passwd
ok 6 - RUNS true
ok 7 - NRUNS false
ok 8 - RUNS echo -e 'foo\nbar\nbaz'
ok 9 - GREP bar
ok 10 - OGREP bar
ok 11 - NEGREP . # verify empty
ok 12 - DIFF <<EOF
not ok 13 - TODO RUNS false # TODO Test Know to fail

The last test shows as "not ok", but the exit code is 0 because it's a TODO. One can set verbose as well:

$ OSHT_VERBOSE=1 bash test.sh # Or -v
1..13
# dcsobral \!= root
ok 1 - IS $(whoami) != root
# foobar =\~ foo
ok 2 - IS "$var" =~ foo
# \! foobar == foo
ok 3 - ISNT "$var" == foo
# test -f /etc/passwd
ok 4 - OK -f /etc/passwd
# test \! -w /etc/passwd
ok 5 - NOK -w /etc/passwd
# RUNNING: true
# STATUS: 0
# STDIO <<EOM
# EOM
ok 6 - RUNS true
# RUNNING: false
# STATUS: 1
# STDIO <<EOM
# EOM
ok 7 - NRUNS false
# RUNNING: echo -e foo\\nbar\\nbaz
# STATUS: 0
# STDIO <<EOM
# foo
# bar
# baz
# EOM
ok 8 - RUNS echo -e 'foo\nbar\nbaz'
# grep -q bar
ok 9 - GREP bar
# grep -q bar
ok 10 - OGREP bar
# \! grep -q .
ok 11 - NEGREP . # verify empty
ok 12 - DIFF <<EOF
# RUNNING: false
# STATUS: 1
# STDIO <<EOM
# EOM
not ok 13 - TODO RUNS false # TODO Test Know to fail

Rename it to use a .t extension and put it in a t subdirectory, and you can use prove(1) (part of Perl) to run it:

$ prove
t/test.t .. ok
All tests successful.
Files=1, Tests=13,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.11 cusr  0.16 csys =  0.31 CPU)
Result: PASS

Set OSHT_JUNIT or pass -j to produce JUnit output. JUnit can also be combined with prove(1).

I have used this library both testing functions by sourcing their files and then running assertions with IS/OK and their negatives, and scripts by using RUN/NRUN. For me, this framework provides the most gain for the least overhead.

Answer 2

Epoxy is a Bash test framework I designed mainly for testing other software, but I use it to test bash modules as well, including itself and Carton.

Main advantages are relatively low coding overhead, unlimited assertion nesting and flexible selection of assertions to verify.

I made a presentation comparing it to BeakerLib - a framework used by some at Red Hat.

Answer 3

TAP-compliant Bash testing: Bash Automated Testing System

TAP, the Test Anything Protocol, is a simple text-based interface between testing modules in a test harness. TAP started life as part of the test harness for Perl but now has implementations in C, C++, Python, PHP, Perl, Java, JavaScript, and others.

bats-core

Answer 4

Nikita Sobolev wrote an excellent blog post comparing a few different bash test frameworks: Testing Bash applications

For the impatient: Nikita's conclusion was to use Bats but it appears that Nikita missed the Bats-core project which appear to me to be the one to use going forward as the original Bats project has not been actively maintained since 2013.

Answer 5

I have found it hard to justify using bash for larger scripts when Python has such huge advantages:

• Try/Except allows writing more robust scripts with the ability to undo changes in case of an error.
• You don't have to use obscure syntax such as 'if [ x"$foo" = x"$bar"]; then ...' which is prone to errors.
• Easy parsing of options and arguments using the getopt module (and there's an even easier module for parsing arguments, but the name escapes me).
• Python allows you to work with lists/dicts and objects instead of basic strings and arrays.
• Access to proper language tools such as regex, databases (sure you could pipe everything into the mysql command in bash, but it's not the nicest way to write code).
• No need to worry about using the correct form of $* or "$*" or "$@" or $1 or "$1", spaces in filenames isn't an issue, etc, etc, etc.

Now I only use bash for the simplest of scripts.

Answer 6

Why do you say that it's "hard" to test bash scripts?

What's wrong with test wrappers like:

 #!/bin/bash
 set -e
 errors=0
 results=$($script_under_test $args<<ENDTSTDATA
 # inputs
 # go
 # here
 #
 ENDTSTDATA
 )
 [ "$?" -ne 0 ] || {
     echo "Test returned error code $?" 2>&1
     let errors+=1
     }

 echo "$results" | grep -q $expected1 || {
      echo "Test Failed.  Expected $expected1"
      let errors+=1
 }
 # and so on, et cetera, ad infinitum, ad nauseum
 [ "$errors" -gt 0 ] && {
      echo "There were $errors errors found"
      exit 1
 }