-
-
Notifications
You must be signed in to change notification settings - Fork 32.2k
gh-112903: Fix MRO resolution for GenericAlias
types
#112926
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
isinstance(b, (_BaseGenericAlias, GenericAlias)) | ||
or issubclass(b, Generic) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would happen if we did this instead?
isinstance(b, (_BaseGenericAlias, GenericAlias)) | |
or issubclass(b, Generic) | |
isinstance(b, _BaseGenericAlias | |
or (isinstance(b, type) and issubclass(b, Generic)) |
Previously we've never needed to insert Generic
into the __bases__
tuple if a class had a types.GenericAlias
instance in its __orig_bases__
.
I feel like determining what happens when types.GenericAlias
instances are inherited from should really be the job of types.GenericAlias.__mro_entries__
; typing._BaseGenericAlias.__mro_entries__
shouldn't be modifying the __bases__
tuple except to remove (and appropriately replace) typing._BaseGenericAlias
instances.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I had a play around, and it looks like I was wrong on several counts:
- With my alternative idea, one of your test cases fails (the class-creation machinery complains that it can't find a consistent resolution order for
MyMap1
), whereas they both pass on Python 3.12. So my idea won't work as-is. - We have actually been inserting
Generic
into the__bases__
tuple from this method in earlier Python versions if a class had atypes.GenericAlias
instance in its__orig_bases__
, if the__origin__
of thattypes.GenericAlias
instance hadGeneric
in its mro. So this PR would be restoring the behaviour we already had in 3.12 (even if that behaviour looks to me like it was probably accidental...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think @AlexWaygood's suggestion here is on the right track. The contract of the __mro_entries__
method is that it receives the original bases tuple. That means that the bases tuple it receives can legitimately include non-types (which themselves implement __mro_entries__
). So it is just buggy for an __mro_entries__
method to assume that it can call issubclass
on all bases it sees. Thus I think Alex's fix must be the correct starting point here; if we don't make that fix, it is only a matter of time before someone comes along with some other third-party non-type object in their bases along with a _BaseGenericAlias
and hits this same bug.
To clarify this point, here's a valid case not involving types.GenericAlias
at all, that also hits this same bug in _BaseGenericAlias.__mro_entries__()
:
>>> class O:
... def __mro_entries__(self, bases):
... return (object,)
...
>>> import typing
>>> class A(typing.List[int], O()): ...
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
class A(typing.List[int], O()): ...
File "/home/carljm/cpython/Lib/typing.py", line 1388, in __mro_entries__
return super().__mro_entries__(bases)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/home/carljm/cpython/Lib/typing.py", line 1140, in __mro_entries__
if isinstance(b, _BaseGenericAlias) or issubclass(b, Generic):
~~~~~~~~~~^^^^^^^^^^^^
TypeError: issubclass() arg 1 must be a class
I think the fix here should handle this case also. The correct result in this example should be the same result as for class A(typing.List[int], object): ...
.
Alex's suggestion fixes this case. But clearly there are some follow-on implications we need to resolve as well, to deal with the "can't find consistent resolution order" error in the mapping example; I'm looking into that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AlexWaygood I don't understand your point (2), though. Your suggestion here would also still "insert Generic
into the __bases__
tuple from this method if a class had a types.GenericAlias
instance in its __orig_bases__
." The behavior implemented by your suggestion ends up being "any unrecognized non-type in the MRO does not stop us from adding Generic
." That seems like the right behavior to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AlexWaygood I don't understand your point (2), though. Your suggestion here would also still "insert
Generic
into the__bases__
tuple from this method if a class had atypes.GenericAlias
instance in its__orig_bases__
." The behavior implemented by your suggestion ends up being "any unrecognized non-type in the MRO does not stop us from addingGeneric
." That seems like the right behavior to me.
Correct; I'm not entirely sure what I was talking about in point (2) on Dec 10 2023 :) I agree with all your points!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be more generic, we can call __mro_entries__()
for any non-type in bases
and check whether it returns a Generic
subclass.
We can also move this logic in the type
code that calls __mro_entries__()
, to allow returning duplicated bases, but... the code became complicated. And we do not have real use case for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think this (calling __mro_entries__()
on non-type bases and seeing if they return Generic
or a subclass) is what we have to do. It is the only way I can see to implement the desired behavior here (add Generic
as an MRO entry only if nothing else will add it). The "can't resolve MRO" error Alex observed is indeed because if we don't actually check __mro_entries__()
of non-type bases, we will end up adding Generic
even when another non-type base is already actually a Generic
after resolution of its __mro_entries__()
.
I agree that in principle the other option could be to modify the contract of __mro_entries__()
such that type
itself will remove duplicates, but I think this behavior is hard to explain and hard to implement correctly. (Even the definition of "duplicate" is non-trivial, since it is not just equality but involves subclasses , too.)
This fix does mean that __mro_entries__()
can be called twice on the same non-type base in a single class creation, but I think this is OK; there is no guarantee about this, and the method should not have side effects.
I will push this fix as a new PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New PR is #115191
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, but I suggest to use simpler examples:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this fix is right, because it changes the resulting MRO.
3.12:
>>> import typing
>>> class A(typing.Sized, list[int]): ...
...
>>> A.__mro__
(<class '__main__.A'>, <class 'collections.abc.Sized'>, <class 'typing.Generic'>, <class 'list'>, <class 'object'>)
This PR:
>>> class A(typing.Sized, list[int]): ...
...
>>> A.__mro__
(<class '__main__.A'>, <class 'collections.abc.Sized'>, <class 'list'>, <class 'object'>)
As a result, A[int]
throws on 3.12 (correctly) and on this PR it is allowed.
When you're done making the requested changes, leave the comment: |
#115191 is a better version 👍 |
There are multiple related issues, including #108403
So, let's be very simple here: we had a specific problem, which is now fixed.