Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using parameter 'action' in function 'argparse.ArgumentParser.add_subparsers' #92811

Open
MatthiasHeinz opened this issue May 14, 2022 · 9 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@MatthiasHeinz
Copy link

MatthiasHeinz commented May 14, 2022

Bug report
The function argparse.ArgumentParser.add_subparsers throws an error regarding parser_class, when passing the parameter action.

Sample-code "ArgParserTest.py":

from argparse import ArgumentParser

parser = ArgumentParser(prog='Testcase for ArgumentParser._subparsers')
# init subparsers for parser
subparsers = parser.add_subparsers(action='count') # Any action works to reproduce.
# Add my_subparser with any "action"
my_subparser = subparsers.add_parser(name='my_subparser')

Terminal output:

> python ArgParserTest.py
Traceback (most recent call last):
  File "REDACTED\ArgParserTest.py", line 5, in <module>
    subparsers = parser.add_subparsers(dest='subcommands', action='count', help='Available subcommands. Run \'<subcommand> -h\' to display further information.')
  File "REDACTED\Python3.10.4\lib\argparse.py", line 1798, in add_subparsers
    action = parsers_class(option_strings=[], **kwargs)
TypeError: _CountAction.__init__() got an unexpected keyword argument 'parser_class'

Possible resolutions

  1. If there is no valid action to subparsers:
    a. Have the error-message complain about the argument action instead of parser_class.
    b. Remove the argument action from the parameter list to the add_subparser-function.
    c. Update the documentation accordingly.
  2. If action can be a valid argument, fix either the error-message or whatever error did occur.

Environment

  • Tested on: Python 3.10.4
  • Operating system: Win10 x64 (Version 21H2, Build 19044.1706)
@hpaulj
Copy link

hpaulj commented May 14, 2022

An 'action' works as long as it's compatible with the default subparsers class. For example using that subclass itself:

 subparsers = parser.add_subparsers(action=argparse._SubParsersAction)

add_subparsers is essentially a variant on add_argument, one that creates a positional and returns an Action. It is special in that it creates a special Action subclass, and supports any necessary parameters.

While I've suggested using 'action' to supply a customized _SUbParsersAction class, I haven't seen other users or questioners try to use it (either on bug/issues or StackOverflow). While refining the docs might be order, it might just add to the confusion.

In other words, 'action' works if the subclass signature is like:

class _SubParsersAction(Action):   
     def __init__(self,
             option_strings,
             prog,
             parser_class,
             dest=SUPPRESS,
             required=False,
             help=None,
             metavar=None):

@MatthiasHeinz
Copy link
Author

I was trying to use subparsers for the first time and basically tried to use action='extend' to be able to supply multiple of those subcommands at the same time (i.e. have a help for both command1 and command2 and offer the ability to either run command1, command2 or both of them in a single call to the program - which apparently does not work this way).

Anyway, it should at least be possible to refine the error-message TypeError: _CountAction.__init__() got an unexpected keyword argument 'parser_class', right? Because that one is really questionable from a user's point of view, when supplying an action argument.

While I do see your point regarding the possible confusion in the docs, I doubt anyone will be able to use this feature/parameter at all, if it stays undocumented. Unfortunately reading the docs and even staring at the sourcecode for a couple of minutes did not yield me any useful information in this situation - just my two cents.

@hpaulj
Copy link

hpaulj commented May 15, 2022

You are trying to get around an intentional limitation. If you try to use add_subparsers twice, argparse raises a "'cannot have multiple subparser arguments" error. When a sub-parser is done, the main parser does not resume parsing. It just cleans up a bit and exits.

The 'action' parameter is designed to be flexible and extensible. Registered words like "extend" map onto Action subclasses (e.g. _ExtendAction). add_argument does not tightly control what parameters it accepts and passes on. Users can define their own subclasses, usually modelled on one of the existing subclasses.

add_subparsers creates the Action with a call:

 action = parsers_class(option_strings=[], **kwargs)

add_argument does something similar

action = action_class(**kwargs)

The add_argument signature is about as general as it gets, add_argument(self, *args, **kwargs). It only does a modest amount of checking before it passes kwargs on. It's the class __init__ that determines what keyword parameters are allowed. To refine the error message that you got, the action __init__ would have to accept a general **kwargs, and test that for keys that it does not recognize or want. None of the subclasses do that.

Use of action='append' and 'extend' make most sense with flagged, optionals. While not prohibited for positionals, they don't make much sense there.

@MatthiasHeinz
Copy link
Author

Sorry, if my previous post was misleading is any way. In the first paragraph, I was just trying to tell you how people, that are new(ish) to argparse, could end up trying to invoke the add_subparsers function with the parameter action and a value which is covered by the documentation.

And my goal certainly wasn't for you to rewrite the entire code because of some required argument. I was just thinking about adding a sanity check for the parameter action before calling into

# create the parsers action and add it to the positionals list
parsers_class = self._pop_action_class(kwargs, 'parsers')
action = parsers_class(option_strings=[], **kwargs)
self._subparsers._add_action(action)

of the add_subparsers function. Maybe something along the lines of

# Alternatively blacklist actions not having the aforementioned subclass signature
if supplied_action_parameter is not None and isinstance(supplied_action_parameter, str):
        raise CustomException('<Some more/precise details than the error message about parser_class...>')

Link to aforementioned subclass signature

This way, the default action strings (like count, append, store_true, ...), which aren't supposed to work in this context (if I understand you correctly), would trigger an error message, that's tailored to this situation.

@AlexWaygood AlexWaygood added the stdlib Python modules in the Lib dir label May 15, 2022
@ruzito
Copy link

ruzito commented Oct 2, 2022

I do not understand why this is intentional limitation.
If I want to create nested subparsers I would potentially want to know the whole chain of commands taken.
So for example:

sub = parent.add_subparsers(dest='subparsers', action='append')
a = sub.add_parser('a')
subsub = a.add_subparsers(dest='subparsers', action='append')
b = subsub.add_parser('b')

would return

Namespace(subparsers=['a', 'b'])

I do not see a simpler way of doing this using the current ideology of argparse library right now.

@hpaulj
Copy link

hpaulj commented Oct 3, 2022 via email

@backerdrive
Copy link

backerdrive commented Jan 1, 2024

@hpaulj I am missing the same feature @ruzito mentioned.

Given is a command-line requirement like:

(cmd1|cmd2) subcmd

This can be expressed by

parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest="command")
subparser_a = subparsers.add_parser("cmd1")
subparser_b = subparsers.add_parser("cmd2")
subsubparsers_a = subparser_a.add_subparsers(dest="command")
subsubparser_a = subsubparsers_a.add_parser("subcmd")
subsubparsers_b = subparser_b.add_subparsers(dest="command")
subsubparser_b = subsubparsers_b.add_parser("subcmd")
print(parser.parse_args(["cmd1", "subcmd"]))

We wouldn't be able to identify, if subsubparser_a or subsubparser_b actually parsed the input, assuming the need to process output further conditionally. The result is ambigious:

Namespace(command='subcmd')

I'd like to assign an identifier or unique tuple (whole chain of commands) to dest, resulting in:

Namespace(command=["cmd1", "subcmd"])

Giving different dest argument names is not scalable for multiple chained parsers and not as easy to process:

Namespace(command1='cmd1',command2='subcmd', ... ,commandXZY='subcmdXYZ')

This is what lists/sequences are naturally used for.

Adding action='append' would be just one of several alternatives. What about being able to additionally pass a callback to dest, which receives dest values of parent parsers?

dest=lambda prev_cmds: ".".join(prev_cmds)

I hope this is comprehensible so far. As this gets a bit orthogonal to OP, should I make up a separate feature request?

@hpaulj
Copy link

hpaulj commented Jan 1, 2024 via email

@backerdrive
Copy link

backerdrive commented Jan 2, 2024

@hpaulj Thanks for pointing out the invoked code locations, it makes sense to me.

Anyways, I don't think the standard argparse should have any added
features to allow/enhance this use of nested subparsers. This nesting is a
kludge used to get around the intended one-subparser per parser
constraint.

This was not clear to me from reading the docs. In this case it probably would be a good idea to document intended usage - only one direct subparser layer, no nesting - more clearly.

If the goal is to identify the (possibly nested) used sub-parser afterwards, I have found a simple workaround for dest via set_defaults:

# Above example, with below additions
subsubparser_a.set_defaults(__ID__=["cmd1", "subcmd"])
subsubparser_b.set_defaults(__ID__=["cmd2", "subcmd"])

, which provides __ID__ in returned Namespace:

$ ./path/to/prog.py cmd1 subcmd
Namespace(__ID__=['cmd1', 'subcmd'])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Status: Bugs
Development

No branches or pull requests

5 participants