Bash empty array expansion with `set -u`

Question

I\'m writing a bash script which has set -u, and I have a problem with empty array expansion: bash appears to treat an empty array as an unset variable during e

Answer 1

The only safe idiom is ${arr[@]+"${arr[@]}"}

This is already the recommendation in ikegami's answer, but there's a lot of misinformation and guesswork in this thread. Other patterns, such as ${arr[@]-} or ${arr[@]:0}, are not safe across all major versions of Bash.

As the table below shows, the only expansion that is reliable across all modern-ish Bash versions is ${arr[@]+"${arr[@]}"} (column +"). Of note, several other expansions fail in Bash 4.2, including (unfortunately) the shorter ${arr[@]:0} idiom, which doesn't just produce an incorrect result but actually fails. If you need to support versions prior to 4.4, and in particular 4.2, this is the only working idiom.

Unfortunately other + expansions that, at a glance, look the same do indeed emit different behavior. :+ expansion is not safe, because :-expansion treats an array with a single empty element (('')) as "null" and thus doesn't (consistently) expand to the same result.

Quoting the full expansion instead of the nested array ("${arr[@]+${arr[@]}}"), which I would have expected to be roughly equivalent, is similarly unsafe in 4.2.

You can see the code that generated this data along with results for several additional version of bash in this gist.

Answer 2

Interesting inconsistency; this lets you define something which is "not considered set" yet shows up in the output of declare -p

arr=()
set -o nounset
echo ${arr[@]}
 =>  -bash: arr[@]: unbound variable
declare -p arr
 =>  declare -a arr='()'

UPDATE: as others mentioned, fixed in 4.4 released after this answer was posted.

Answer 3

"Interesting" inconsistency indeed.

Furthermore,

$ set -u
$ echo $#
0
$ echo "$1"
bash: $1: unbound variable   # makes sense (I didn't set any)
$ echo "$@" | cat -e
$                            # blank line, no error

While I agree that the current behavior may not be a bug in the sense that @ikegami explains, IMO we could say the bug is in the definition (of "set") itself, and/or the fact that it's inconsistently applied. The preceding paragraph in the man page says

... ${name[@]} expands each element of name to a separate word. When there are no array members, ${name[@]} expands to nothing.

which is entirely consistent with what it says about the expansion of positional parameters in "$@". Not that there aren't other inconsistencies in the behaviors of arrays and positional parameters... but to me there's no hint that this detail should be inconsistent between the two.

Continuing,

$ arr=()
$ echo "${arr[@]}"
bash: arr[@]: unbound variable   # as we've observed.  BUT...
$ echo "${#arr[@]}"
0                                # no error
$ echo "${!arr[@]}" | cat -e
$                                # no error

So arr[] isn't so unbound that we can't get a count of its elements (0), or a (empty) list of its keys? To me these are sensible, and useful -- the only outlier seems to be the ${arr[@]} (and ${arr[*]}) expansion.

Answer 4

I am complementing on @ikegami's (accepted) and @kevinarpe's (also good) answers.

You can do "${arr[@]:+${arr[@]}}" to workaround the problem. The right-hand-side (i.e., after :+) provides an expression that will be used in case the left-hand-side is not defined/null.

The syntax is arcane. Note that the right hand side of the expression will undergo parameter expansion, so extra attention should be paid to having consistent quoting.

: example copy arr into arr_copy
arr=( "1 2" "3" )
arr_copy=( "${arr[@]:+${arr[@]}}" ) # good. same quoting. 
                                    # preserves spaces

arr_copy=( ${arr[@]:+"${arr[@]}"} ) # bad. quoting only on RHS.
                                    # copy will have ["1","2","3"],
                                    # instead of ["1 2", "3"]

Like @kevinarpe mentions, a less arcane syntax is to use the array slice notation ${arr[@]:0} (on Bash versions >= 4.4), which expands to all the parameters, starting from index 0. It also doesn't require as much repetition. This expansion works regardless of set -u, so you can use this at all times. The man page says (under Parameter Expansion):

• ${parameter:offset}

• ${parameter:offset:length}

  ... If parameter is an indexed array name subscripted by @ or *, the result is the length members of the array beginning with ${parameter[offset]}. A negative offset is taken relative to one greater than the maximum index of the specified array. It is an expansion error if length evaluates to a number less than zero.

This is the example provided by @kevinarpe, with alternate formatting to place the output in evidence:

set -u
function count() { echo $# ; };
(
    count x y z
)
: prints "3"

(
    arr=()
    count "${arr[@]}"
)
: prints "-bash: arr[@]: unbound variable"

(
    arr=()
    count "${arr[@]:0}"
)
: prints "0"

(
    arr=(x y z)
    count "${arr[@]:0}"
)
: prints "3"

This behaviour varies with versions of Bash. You may also have noticed that the length operator ${#arr[@]} will always evaluate to 0 for empty arrays, regardless of set -u, without causing an 'unbound variable error'.

Answer 5

According to the documentation,

An array variable is considered set if a subscript has been assigned a value. The null string is a valid value.

No subscript has been assigned a value, so the array isn't set.

But while the documentation suggests an error is appropriate here, this is no longer the case since 4.4.

$ bash --version | head -n 1
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)

$ set -u

$ arr=()

$ echo "foo: '${arr[@]}'"
foo: ''

---

There is a conditional you can use inline to achieve what you want in older versions: Use ${arr[@]+"${arr[@]}"} instead of "${arr[@]}".

$ function args { perl -E'say 0+@ARGV; say "$_: $ARGV[$_]" for 0..$#ARGV' -- "$@" ; }

$ set -u

$ arr=()

$ args "${arr[@]}"
-bash: arr[@]: unbound variable

$ args ${arr[@]+"${arr[@]}"}
0

$ arr=("")

$ args ${arr[@]+"${arr[@]}"}
1
0: 

$ arr=(a b c)

$ args ${arr[@]+"${arr[@]}"}
3
0: a
1: b
2: c

Tested with bash 4.2.25 and 4.3.11.