Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop #93896

Open
graingert opened this issue Jun 16, 2022 · 4 comments · May be fixed by #94058
Open

IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop #93896

graingert opened this issue Jun 16, 2022 · 4 comments · May be fixed by #94058
Assignees
@asvetlov
Labels
3.11 3.12 expert-asyncio type-bug

Comments

@graingert
Copy link
Contributor

@graingert graingert commented Jun 16, 2022 •
edited

in https://github.com/python/cpython/pull/31799/files#diff-1f2ae0f6c6010caf9d5f1c80cd6033a796ffe2b60554f5df84f554f4a08e622b the calls to asyncio.set_event_loop() were removed which breaks aiohttp aio-libs/aiohttp#6757 and asyncio.SafeChildWatcher

this looks like an intentional breaking change - and if it is should be documented
@graingert graingert added the type-bug label Jun 16, 2022
@graingert
Copy link
Contributor Author

@graingert graingert commented Jun 16, 2022 •
edited 

import sys
import asyncio


async def run_subprocess():
    proc = await asyncio.create_subprocess_shell(
        "exit 0",
        stdin=asyncio.subprocess.DEVNULL,
        stdout=asyncio.subprocess.DEVNULL,
        stderr=asyncio.subprocess.DEVNULL,
    )
    await proc.wait()
    print("success!")


async def amain():
    await asyncio.to_thread(asyncio.run, run_subprocess())


def main():
    asyncio.get_event_loop_policy().set_child_watcher(asyncio.SafeChildWatcher())
    asyncio.run(amain())


if __name__ == "__main__":
    sys.exit(main())
 graingert@superjacent  ~/projects  python3.10 demo_subprocess.py
success!
 graingert@superjacent  ~/projects  python3.11 demo_subprocess.py 
Traceback (most recent call last):
  File "/home/graingert/projects/demo_subprocess.py", line 26, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/graingert/projects/demo_subprocess.py", line 22, in main
    asyncio.run(amain())
    ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 181, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 115, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/graingert/projects/demo_subprocess.py", line 17, in amain
    await asyncio.to_thread(asyncio.run, run_subprocess())
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 181, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 115, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/graingert/projects/demo_subprocess.py", line 6, in run_subprocess
    proc = await asyncio.create_subprocess_shell(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/subprocess.py", line 205, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1647, in subprocess_shell
    transport = await self._make_subprocess_transport(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/unix_events.py", line 204, in _make_subprocess_transport
    raise RuntimeError("asyncio.get_child_watcher() is not activated, "
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: asyncio.get_child_watcher() is not activated, subprocess support is not installed.

@graingert graingert mentioned this issue Jun 16, 2022
Closed
@graingert
Copy link
Contributor Author

@graingert graingert commented Jun 16, 2022 •
edited

I know there were some efforts to deprecate get/set_event_loop_policy and child watchers #82772 so I think those Deprecations should be applied before asyncio.run/IsolatedAsyncioTestCase stops calling asyncio.set_event_loop()

@graingert
Copy link
Contributor Author

@graingert graingert commented Jun 17, 2022 •
edited

alternatively is there still time to deprecate the child watchers system and the policy system in favor of
asyncio.Runner(loop_factory=asyncio.ProactorEventLoop/asyncio.SelectorEventLoop/uvloop.uvloop.new_event_loop) before the 3.11 release?

that would be deprecating:

asyncio.get_event_loop() # already deprecated unless the loop is running 
asyncio.set_event_loop()  # asyncio.set_event_loop(None) should probably be exempt
asyncio.get_event_loop_policy()
asyncio.set_event_loop_policy()  # asyncio.set_event_loop_policy(None) should probably be exempt
asyncio.set_child_watcher()

asyncio.get_child_watcher() will remain and will always be asyncio.PidfdChildWatcher on old kernels and asyncio.ThreadedChildWatcher on very old kernels

asyncio.new_event_loop() will issue a DeprecationWarning if the current policy is not the default policy, and then become an alias of

if sys.platform == "win32":
    new_event_loop = ProactorEventLoop
else:
    new_event_loop = SelectorEventLoop

graingert added a commit to graingert/cpython that referenced this issue Jun 21, 2022
@graingert
restore the set_event_loop calls to asyncio.run
48596ea 
Fixes python#93896
@graingert graingert linked a pull request Jun 21, 2022 that will close this issue
Open
graingert added a commit to graingert/cpython that referenced this issue Jun 22, 2022
@graingert
restore the set_event_loop calls to asyncio.run
f6b7bab 
Fixes python#93896
@graingert graingert mentioned this issue Jun 22, 2022
Closed
@graingert
Copy link
Contributor Author

@graingert graingert commented Jun 23, 2022

this is also broken with the PidfdChildWatcher

@graingert graingert mentioned this issue Jun 23, 2022
Open
@graingert graingert mentioned this issue Jun 24, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@asvetlov asvetlov

Labels
3.11 3.12 expert-asyncio type-bug
Projects
asyncio
Status: No status
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gh-93896: restore the set_event_loop calls to asyncio.run graingert/cpython
3 participants
@asvetlov @graingert @AlexWaygood