`typing.get_type_hints()` works for `Callable[[], None]` from the `typing` module, but not `collections.abc` #91621

Zac-HD · 2022-04-16T22:21:03Z

Bug report

typing.get_type_hints() works for Callable[[], None] from the typing module, but not collections.abc. This is the standard way to annotate a callable which takes no arguments, and the generic types from typing should behave identically to those in collections.abc.

import collections.abc
import typing

def f1(x: typing.Callable[[], None]):
    pass

def f2(x: collections.abc.Callable[[], None]):
    pass

assert "x" in typing.get_type_hints(f1)
typing.get_type_hints(f2)  # Raises TypeError!

Your environment

CPython 3.11.0a7 (and I believe back to a3), found via CI for Hypothesis, and reproduced in Ubuntu/WSL2 locally.
I'm skipping that test for now, and will re-enable when this is fixed.

The text was updated successfully, but these errors were encountered:

AlexWaygood · 2022-04-16T22:46:24Z

Reproduced, and I agree that this looks like a regression in 3.11. On 3.10:

C:\Users\alexw\coding>python
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing, collections.abc
>>> def f(x: collections.abc.Callable[[], None]): ...
...
>>> typing.get_type_hints(f)
{'x': collections.abc.Callable[[], None]}

On 3.11:

C:\Users\alexw\coding>python
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing, collections.abc
>>> def f(x: collections.abc.Callable[[], None]): ...
...
>>> typing.get_type_hints(f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\alexw\coding\cpython\Lib\typing.py", line 2277, in get_type_hints
    hints[name] = _eval_type(value, globalns, localns)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\typing.py", line 353, in _eval_type
    t = t.__origin__[args]
        ~~~~~~~~~~~~^^^^^^
  File "<frozen _collections_abc>", line 430, in __new__
TypeError: Callable must be used as Callable[[arg, ...], result].

hauntsaninja · 2022-04-17T01:04:58Z

Looks like this is a result of #30900 cc @NiklasRosenstein @JelleZijlstra

hauntsaninja · 2022-04-17T01:17:28Z

The complexity here traces back to some of the decisions made in https://bugs.python.org/issue42195

We do this kind of thing a couple times in typing.py, I can open a PR:

diff --git a/Lib/typing.py b/Lib/typing.py
index 3e0fbdb989..af424c2109 100644
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -350,7 +350,11 @@ def _eval_type(t, globalns, localns, recursive_guard=frozenset()):
                 ForwardRef(arg) if isinstance(arg, str) else arg
                 for arg in t.__args__
             )
-            t = t.__origin__[args]
+            if (t.__origin__ is collections.abc.Callable
+                and not (len(args) == 2 and _is_param_expr(args[0]))):
+                t = t.__origin__[(args[:-1], args[-1])]
+            else:
+                t = t.__origin__[args]
         ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__)
         if ev_args == t.__args__:
             return t

Fidget-Spinner · 2022-04-17T16:01:39Z

The complexity here traces back to some of the decisions made in https://bugs.python.org/issue42195

We do this kind of thing a couple times in typing.py, I can open a PR:

🤦 . Seeing this again, I'm sad that the special case has to exist. Unfortunately, this seems like an easy way to trip up new contributors. Please open a PR.

This mirrors logic in typing.get_args. The trickiness comes from how we flatten args in collections.abc.Callable, see https://bugs.python.org/issue42195

) This mirrors logic in typing.get_args. The trickiness comes from how we flatten args in collections.abc.Callable, see https://bugs.python.org/issue42195

JelleZijlstra · 2022-05-02T23:09:23Z

Thanks @Zac-HD for the pre-beta testing and @hauntsaninja for the patch!

Zac-HD added the type-bug An unexpected behavior, bug, or error label Apr 16, 2022

AlexWaygood added stdlib Python modules in the Lib dir 3.11 expert-typing labels Apr 16, 2022

Zac-HD mentioned this issue Apr 16, 2022

Re-enable tests on 3.11 HypothesisWorks/hypothesis#3294

Closed

JelleZijlstra mentioned this issue May 2, 2022

gh-91621: Fix typing.get_type_hints for collections.abc.Callable #91656

Merged

JelleZijlstra closed this as completed May 2, 2022

`typing.get_type_hints()` works for `Callable[[], None]` from the `typing` module, but not `collections.abc` #91621

`typing.get_type_hints()` works for `Callable[[], None]` from the `typing` module, but not `collections.abc` #91621

Zac-HD commented Apr 16, 2022 •

edited

AlexWaygood commented Apr 16, 2022 •

edited

hauntsaninja commented Apr 17, 2022

hauntsaninja commented Apr 17, 2022

Fidget-Spinner commented Apr 17, 2022

JelleZijlstra commented May 2, 2022

typing.get_type_hints() works for Callable[[], None] from the typing module, but not collections.abc #91621

typing.get_type_hints() works for Callable[[], None] from the typing module, but not collections.abc #91621

Comments

Zac-HD commented Apr 16, 2022 • edited

AlexWaygood commented Apr 16, 2022 • edited

hauntsaninja commented Apr 17, 2022

hauntsaninja commented Apr 17, 2022

Fidget-Spinner commented Apr 17, 2022

JelleZijlstra commented May 2, 2022

`typing.get_type_hints()` works for `Callable[[], None]` from the `typing` module, but not `collections.abc` #91621

`typing.get_type_hints()` works for `Callable[[], None]` from the `typing` module, but not `collections.abc` #91621

Zac-HD commented Apr 16, 2022 •

edited

AlexWaygood commented Apr 16, 2022 •

edited