Properly handling spaces and quotes in bash completion

Question

What is the correct/best way of handling spaces and quotes in bash completion?

Here’s a simple example. I have a command called words (e.g., a dictionar

Answer 1

The question is rather loaded but this answer attempts to explain each aspect:

1. How to handle spaces with COMPREPLY.
2. How does ls do it.

There're also people reaching this question wanting to know how to implement the completion function in general. So:

3. How how do I implement the completion function and correctly set COMPREPLY?

How does ls do it

Moreover, why does it behave differently to when I set COMPREPLY?

Back in '12 (before I updated this answer), I was in a similar situation and searched high and low for the answer to this discrepancy myself. Here's the answer I came up with.

ls, or rather, the default completion routine does it using the -o filenames functionality. This option performs: filename-specific processing (like adding a slash to directory names or suppressing trailing spaces.

To demonstrate:

$ foo () { COMPREPLY=("bar one" "bar two"); }
$ complete -o filenames -F foo words
$ words ░

Tab

$ words bar\ ░          # Ex.1: notice the space is completed escaped

TabTab

bar one  bar two        # Ex.2: notice the spaces are displayed unescaped
$ words bar\ ░

Immediately there are two points I want to make clear to avoid any confusion:

• First of all, your completion function cannot be implemented simply by setting COMPREPLY to an array of your word list! The example above is hard-coded to return candidates starting with b-a-r just to show what happens when TabTab is pressed. (Don't worry, we'll get to a more general implementation shortly.)

• Second, the above format for COMPREPLY only works because -o filenames is specified. For an explanation of how to set COMPREPLY when not using -o filenames, look no further than the next heading.

Also note, there's a downside of using -o filenames: If there's a directory lying about with the same name as the matching word, the completed word automatically gets an arbitrary slash attached to the end. (e.g. bar\ one/)

How to handle spaces with COMPREPLY without using -o filenames

Long story short, it needs to be escaped.

In contrast to the above -o filenames demo:

$ foo () { COMPREPLY=("bar\ one" "bar\ two"); }     # Notice the blackslashes I've added
$ complete -F foo words                             # Notice the lack of -o filenames
$ words ░

Tab

$ words bar\ ░          # Same as -o filenames, space is completed escaped

TabTab

bar\ one  bar\ two      # Unlike -o filenames, notice the spaces are displayed escaped
$ words bar\ ░

How do I actually implement a completion function?

Implementing a completion functions involves:

1. Representing your word list.
2. Filtering your word list to just candidates for the current word.
3. Setting COMPREPLY correctly.

I'm not going to assume to know all the complex requirements there can be for 1 and 2 and the following is only a very basic implementation. I'm providing an explanation for each part so one can mix-and-match to fit their own requirements.

foo() {
    # Get the currently completing word
    local CWORD=${COMP_WORDS[COMP_CWORD]}

    # This is our word list (in a bash array for convenience)
    local WORD_LIST=(foo 'bar one' 'bar two')

    # Commands below depend on this IFS
    local IFS=$'\n'

    # Filter our candidates
    CANDIDATES=($(compgen -W "${WORD_LIST[*]}" -- "$CWORD"))

    # Correctly set our candidates to COMPREPLY
    if [ ${#CANDIDATES[*]} -eq 0 ]; then
        COMPREPLY=()
    else
        COMPREPLY=($(printf '%q\n' "${CANDIDATES[@]}"))
    fi
}

complete -F foo words

In this example, we use compgen to filter our words. (It's provided by bash for this exact purpose.) One could use any solution they like but I'd advise against using grep-like programs simply because of the complexities of escaping regex.

compgen takes the word list with the -W argument and returns the filtered result with one word per line. Since our words can contain spaces, we set IFS=$'\n' beforehand in order to only count newlines as element delimiters when putting the result into our array with the CANDIDATES=(...) syntax.

Another point of note is what we're passing for the -W argument. This argument takes an IFS delimited word list. Again, our words contain spaces so this too requires IFS=$'\n' to prevent our words being broken up. Incidentally, "${WORD_LIST[*]}" expands with elements also delimited with what we've set for IFS and is exactly what we need.

In the example above I chose to define WORD_LIST literally in code.

One could also initialize the array from an external source such as a file. Just make sure to move IFS=$'\n' beforehand if words are going to be line-delimited such as in the original question:

local IFS=$'\n'
local WORD_LIST=($(cat /path/to/words.dat))`

Finally, we set COMPREPLY making sure to escape the likes of spaces. Escaping is quite complicated but thankfully printf's %q format performs all the necessary escaping we need and that's what we use to expand CANDIDATES. (Note we're telling printf to put \n after each element because that's what we've set IFS to.)

Those observant may spot this form for COMPREPLY only applies if -o filenames is not used. No escaping is necessary if it is and COMPREPLY may be set to the same contents as CANDIDATES with COMPREPLY=("$CANDIDATES[@]").

Extra care should be taken when expansions may be performed on empty arrays as this can lead to unexpected results. The example above handles this by branching when the length of CANDIDATES is zero.