Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

argparse: positional with nargs=* results in ignored option argument if they share a dest #101990

Open
stephenfin opened this issue Feb 17, 2023 · 2 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@stephenfin
Copy link

Bug report

Given the following example:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs='*')
parser.add_argument('--foo')

print(vars(parser.parse_args()))

Passing positional arguments works as expected.

❯ python /tmp/test.py bar
{'foo': ['bar']}

However, the option arguments are ignored.

❯ python /tmp/test.py --foo bar
{'foo': []}

As a workaround, you can set a separate dest and handle this in your application.

fwiw, I am using this to deprecate a positional argument in favour of an option argument (using exclusive groups) but this is a minimal reproducer.

Your environment

  • CPython versions tested on: python3.11.1
  • Operating system and architecture: Fedora 37
@stephenfin stephenfin added the type-bug An unexpected behavior, bug, or error label Feb 17, 2023
@stephenfin stephenfin changed the title argparse: positional with nargs=* overrides option argument argparse: positional with nargs=* results in ignored option argument if they share a dest Feb 17, 2023
@b8choi
Copy link
Contributor

b8choi commented Mar 6, 2023

I don't think this is a bug because if there are no positional arguments, we can assume that the positional argument "foo" has a null value at the end of the command, which will then overwrite the value of "--foo".

@hpaulj
Copy link

hpaulj commented Mar 8, 2023

Modifying your example to more clearly show the defaults:

parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs='*', default='pos')
parser.add_argument('--foo', default='opt');

The positional's default has priority:

In [22]: parser.parse_args([]) 
Out[22]: Namespace(foo='pos')

With a positional value:

In [25]: parser.parse_args(['bar'])
Out[25]: Namespace(foo=['bar'])

With both, the last optional's value is kept:

In [26]: parser.parse_args(['bar','--foo=x'])
Out[26]: Namespace(foo='x')

But here, the positional default is used:

In [27]: parser.parse_args(['--foo=x'])
Out[27]: Namespace(foo='pos')

There some subtilties in how positionals and optionals are parsed, and how the '*' positional in particular is handled.

A star positional is 'satisfied' by "nothing", an empty list of strings. That means it is always "seen", in both the [22] and the [27] cases. That empty list will be put in the namespace. Except if it has a not-None default, that is put there instead. That behavior was included in part so that such a positional can be used in a mutually_exclusive_group.

Normally parsing alternates between handling positionals and optionals. That's the key loop in _parse_known_args. But if a positional doesn't consume any strings, it is skipped in this loop. There is a final `consume_positionals' that will cleanup any remaining positionals, including 'star' ones.

In [27], the '--foo' is handled, but the final consumption of the positional ends up over writing that value with its default (or [] in your example).

Hopefully that explains why we get a counter intuitive result. Sharing a dest can be convenient, but it is possible as a consequence of how the overall parsing is done, not as an intentional feature. Nothing in the code takes note that the two Actions share the dest. Each does its own thing, without awareness that it is sharing the dest. The last one 'wins'.

So such sharing must be done with caution. If it causes problems like this, don't do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
Status: Bugs
Development

No branches or pull requests

3 participants