argparse subparser monolithic help output

Question

My argparse has only 3 flags (store_true) on the top level, everything else is handled through subparsers. When I run myprog.py --help, the output shows a list of all subcommands like normal, {sub1, sub2, sub3, sub4, ...}. So, the default is working great...

I usually can't remember the exact subcommand name I need, and all of its options. So I end up doing 2 help lookups:

myprog.py --help
myprog.py sub1 --help

I do this so often, I decided to cram this into one step. I would rather have my toplevel help output a huge summary, and then I scroll through the list manually. I find it is much faster (for me at least).

I was using a RawDescriptionHelpFormatter, and typing the long help output by hand. But now I have lots of subcommands, and its becoming a pain to manage.

Is there a way to get a verbose help output with just one program call?

If not, how can I iterate the subparsers of my argparse instance, and then retrieve the help output individually from each one (which I will then later glue together)?

---

Here is a quick outline of my argparse setup. I cleaned/stripped the code a fair bit, so this may not run without a bit of help.

parser = argparse.ArgumentParser(
        prog='myprog.py',
        formatter_class=argparse.RawDescriptionHelpFormatter,
        description=textwrap.dedent(""" You can manually type Help here """) )

parser.add_argument('--debuglog', action='store_true', help='Verbose logging for debug purposes.')
parser.add_argument('--ipyonexit', action='store_true', help='Drop into an embeded Ipython session instead of exiting command.')

subparser = parser.add_subparsers()

### --- Subparser B
parser_b = subparser.add_parser('pdfreport', description="Used to output reports in PDF format.")
parser_b.add_argument('type', type=str, choices=['flatlist', 'nested', 'custom'],
                        help="The type of PDF report to generate.")
parser_b.add_argument('--of', type=str, default='',
                        help="Override the path/name of the output file.")
parser_b.add_argument('--pagesize', type=str, choices=['letter', '3x5', '5x7'], default='letter',
                        help="Override page size in output PDF.")
parser_b.set_defaults(func=cmd_pdf_report)

### ---- Subparser C
parser_c = subparser.add_parser('dbtables', description="Used to perform direct DB import/export using XLS files.")
parser_c.add_argument('action', type=str, choices=['push', 'pull', 'append', 'update'],
                        help="The action to perform on the Database Tables.")
parser_c.add_argument('tablename', nargs="+",
                        help="The name(s) of the DB-Table to operate on.")
parser_c.set_defaults(func=cmd_db_tables)

args = parser.parse_args()
args.func(args)

Answer 1

This is a bit tricky, as argparse does not expose a list of defined sub-parsers directly. But it can be done:

import argparse

# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')

# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')

# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
# print main help
print(parser.format_help())

# retrieve subparsers from parser
subparsers_actions = [
    action for action in parser._actions 
    if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better save than sorry
for subparsers_action in subparsers_actions:
    # get all subparsers and print help
    for choice, subparser in subparsers_action.choices.items():
        print("Subparser '{}'".format(choice))
        print(subparser.format_help())

This example should work for python 2.7 and python 3. The example parser is from Python 2.7 documentation on argparse sub-commands.

The only thing left to do is adding a new argument for the complete help, or replacing the built in -h/--help.

Answer 2

Here is complete soulution with custom help handler (almost all code from @Adaephon answer):

import argparse


class _HelpAction(argparse._HelpAction):

    def __call__(self, parser, namespace, values, option_string=None):
        parser.print_help()

        # retrieve subparsers from parser
        subparsers_actions = [
            action for action in parser._actions
            if isinstance(action, argparse._SubParsersAction)]
        # there will probably only be one subparser_action,
        # but better save than sorry
        for subparsers_action in subparsers_actions:
            # get all subparsers and print help
            for choice, subparser in subparsers_action.choices.items():
                print("Subparser '{}'".format(choice))
                print(subparser.format_help())

        parser.exit()

# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG', add_help=False)  # here we turn off default help action

parser.add_argument('--help', action=_HelpAction, help='help for help if you need some help')  # add custom help

parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')

# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')

# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')

parsed_args = parser.parse_args()

Answer 3

A simpler way to iterate over the subparsers in Adaephon's example is

for subparser in [parser_a, parser_b]:
   subparser.format_help()

Python does allow you to access hidden attributes like parser._actions, but that's not encouraged. It is just as easy to build your own list while defining the parser. Same goes for doing special things with the arguments. add_argument and add_subparser return their respective Action and Parser objects for a reason.

If I were making a subclass of ArgumentParser I would feel free to use _actions. But for a one off application, building my own list would be clearer.

---

An example:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('mainpos')
parser.add_argument('--mainopt')
sp = parser.add_subparsers()
splist = []   # list to collect subparsers
sp1 = sp.add_parser('cmd1')
splist.append(sp1)
sp1.add_argument('--sp1opt')
sp2 = sp.add_parser('cmd2')
splist.append(sp2)
sp2.add_argument('--sp2opt')

# collect and display for helps    
helps = []
helps.append(parser.format_help())
for p in splist:
   helps.append(p.format_help())
print('\n'.join(helps))

# or to show just the usage
helps = []
helps.append(parser.format_usage())
for p in splist:
   helps.append(p.format_usage())
print(''.join(helps))

The combined 'usage' display is:

usage: stack32607706.py [-h] [--mainopt MAINOPT] mainpos {cmd1,cmd2} ...
usage: stack32607706.py mainpos cmd1 [-h] [--sp1opt SP1OPT]
usage: stack32607706.py mainpos cmd2 [-h] [--sp2opt SP2OPT]

The display of the combined helps is long and redundant. It could be edited in various ways, either after formatting, or with special help formatters. But who is going make such choices?

Answer 4

I was also able to print a short help for commands using _choices_actions.

def print_help(parser):
  print(parser.description)
  print('\ncommands:\n')

  # retrieve subparsers from parser
  subparsers_actions = [
      action for action in parser._actions 
      if isinstance(action, argparse._SubParsersAction)]
  # there will probably only be one subparser_action,
  # but better save than sorry
  for subparsers_action in subparsers_actions:
      # get all subparsers and print help
      for choice in subparsers_action._choices_actions:
          print('    {:<19} {}'.format(choice.dest, choice.help))

Answer 5

Perhaps an easier approach is to use parser.epilog:

def define_parser():
    import argparse
    parser = argparse.ArgumentParser(
        prog='main',
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    commands = parser.add_subparsers(
        title="required commands",
        help='Select one of:',
    )    
    command_list = commands.add_parser(
        'list',
        help='List included services',
    )
    command_ensure = commands.add_parser(
        'ensure',
        help='Provision included service',
    )
    command_ensure.add_argument(
        "service",
        help='Service name',
    )
    import textwrap
    parser.epilog = textwrap.dedent(
        f"""\
        commands usage:\n
        {command_list.format_usage()}
        {command_ensure.format_usage()}
        """
    )
    return parser

parser = define_parser()

parser.print_help()

which results in the following output:

usage: main [-h] {list,ensure} ...

optional arguments:
  -h, --help     show this help message and exit

required commands:
  {list,ensure}  Select one of:
    list         List included services
    ensure       Provision included service

commands usage:

usage: main list [-h]

usage: main ensure [-h] service

Answer 6

I have some simple wrappers that store various references (Parser, SubParser, StoreAction) sequentially, for easy iteration during help generation.

I am now getting organized, verbose, automatically generated help output. I will post an overview when I get a chance.

There is one drawback, and that has to do with the generated help content on Optional Arguments: it is not very good. Improving these help outputs will take more than a wrapper (if we wanna keep it clean). But if you want a good help overview for evolving programs, this should satisfy most.