How does argparse (and the deprecated optparse) respond to 'tab' keypress after python program name, in bash?

Question

I have tested optcomplete working with the optparse module. Its example is a simple file so I could get that working. I also tested it using the argparse<

Answer 1

To understand what's happening here, let's check what that bash function actually does:

COMPREPLY=( $( \
    COMP_LINE=$COMP_LINE  COMP_POINT=$COMP_POINT \
    COMP_WORDS="${COMP_WORDS[*]}"  COMP_CWORD=$COMP_CWORD \
    OPTPARSE_AUTO_COMPLETE=1 $1 ) )

See the $1 at the end? That means that it actually calls the Python file we want to execute with special environment variables set! To trace what's happening, let's prepare a little script to intercept what optcomplete.autocomplete does:

#!/usr/bin/env python2
import os, sys
import optparse, optcomplete
from cStringIO import StringIO

if __name__ == '__main__':    
    parser = optparse.OptionParser()

    parser.add_option('-s', '--simple', action='store_true',
                      help="Simple really simple option without argument.")

    parser.add_option('-o', '--output', action='store',
                      help="Option that requires an argument.")

    opt = parser.add_option('-p', '--script', action='store',
                            help="Option that takes python scripts args only.")
    opt.completer = optcomplete.RegexCompleter('.*\.py')

    # debug env variables
    sys.stderr.write("\ncalled with args: %s\n" % repr(sys.argv))
    for k, v in sorted(os.environ.iteritems()):
        sys.stderr.write("  %s: %s\n" % (k, v))

    # setup capturing the actions of `optcomplete.autocomplete`
    def fake_exit(i):
      sys.stderr.write("autocomplete tried to exit with status %d\n" % i)
    sys.stdout = StringIO()
    sys.exit = fake_exit

    # Support completion for the command-line of this script.
    optcomplete.autocomplete(parser, ['.*\.tar.*'])

    sys.stderr.write("autocomplete tried to write to STDOUT:\n")
    sys.stderr.write(sys.stdout.getvalue())
    sys.stderr.write("\n")

    opts, args = parser.parse_args()

This gives us the following when we try to autocomplete it:

$ ./test.py [tab]
called with args: ['./test.py']
  ...
  COMP_CWORD: 1
  COMP_LINE: ./test.py 
  COMP_POINT: 10
  COMP_WORDS: ./test.py 
  ...
  OPTPARSE_AUTO_COMPLETE: 1
  ...
autocomplete tried to exit with status 1
autocomplete tried to write to STDOUT:
-o -h -s -p --script --simple --help --output

So optcomplete.autocomplete just reads the environment, prepares the matches, writes them to STDOUT and exits. The result -o -h -s -p --script --simple --help --output is then put into a bash array (COMPREPLY=( ... )) and returned to bash to present the choices to the user. No magic involved :)