DeprecationWarning scope expanded in asyncio.events #100160

jaraco · 2022-12-10T14:02:04Z

As discovered in prompt-toolkit/python-prompt-toolkit#1696, it appears that the DeprecationWarning introduced in Python 3.10 has expanded its scope, now with 3.11.1 and 3.10.9 emitting during get_event_loop_policy() where it did not before:

 ~ $ py -3.11 -W error
Python 3.11.0 (main, Oct 26 2022, 19:06:18) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> asyncio.get_event_loop_policy().get_event_loop()
<_UnixSelectorEventLoop running=False closed=False debug=False>
>>> ^D
 ~ $ docker run -it jaraco/multipy-tox py -3.11 -W error
Python 3.11.1 (main, Dec  7 2022, 01:11:34) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> asyncio.get_event_loop_policy().get_event_loop()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.11/asyncio/events.py", line 687, in get_event_loop
    warnings.warn('There is no current event loop',
DeprecationWarning: There is no current event loop

It's not obvious if it was intentional to expand the scope of the DeprecationWarning, but it does come as a surprise as the calling code had previously attempted to address the deprecation.

I think there are two concerns to be addressed:

Does this symptom indicate an unintentional but real additional path that was previously unprotected by the DeprecationWarning? And if so, does that imply that the behavior's first true deprecation is in Python 3.12 and thus will delay the removal?
What is a user expected to do to properly address the deprecation? I read the what's new for Python 3.10 and it indicates that the call is deprecated, but it provides little guidance on how a user can adapt to the new behavior. Maybe there should be a note to the effect of "programs relying on a non-running event loop must ensure that there is a running event loop before attempting to get the event loop."

Linked PRs

The text was updated successfully, but these errors were encountered:

gvanrossum · 2022-12-10T16:07:03Z

@kumaraditya303 @serhiy-storchaka Looks like the 3.12 code somehow made it into 3.11.1 (and 3.10.9). We should roll this back.

the culprit appears to be #93453.

kumaraditya303 · 2022-12-10T16:21:12Z

I am confused, yes there is a change in behavior but it is correct. You need to set the event loop before getting it alternatively use the runner APIs.

I looked at the code and it is missing setting the event loop:
https://github.com/prompt-toolkit/python-prompt-toolkit/blob/da05f669d00817655f76b82972272d4d5f4d4225/src/prompt_toolkit/eventloop/utils.py#L106-L118 The last line gets the event loop without setting it. Am I missing something?

gvanrossum · 2022-12-10T17:34:58Z

In pre-3.12 semantics, the first ever get_event_loop() call (if in the main thread) creates a new loop and sets it as the current loop. Apparently those semantics were lost?

gvanrossum · 2022-12-10T17:35:18Z

We shouldn't have a change in behavior between 3.11.0 and 3.11.1.

serhiy-storchaka · 2022-12-10T18:33:19Z

It was an intentional change based on results of discussion in #93453. BaseDefaultEventLoopPolicy.get_event_loop() will raise an exception in 3.12.

A user is expected to only use get_event_loop() if an event loop was previously set by set_event_loop(), and use new_event_loop() for explicit creation of the event loop. But if you do not know whether an even loop was previously set, the only workaround is silencing DeprecationWarning for the line with get_event_loop().

gvanrossum · 2022-12-10T21:56:36Z

I'm guessing it's unfortunate that we have a deprecation warning in 3.11.1 that wasn't there in 3.11.0, and ditto for 3.10.9 vs. 3.10.8. That's not normally how we do deprecations (unless it's security related).

I'd like an opinion from an SC member -- let me tag @gpshead, he can decide whether to escalate this to the SC agenda or whatever.

gpshead · 2022-12-11T08:45:24Z

New warnings from existing code paths should not normally happen in patch releases. It is unfortunate that this change already shipped in 3.10.9 and 3.11.1. I recommend undoing it in both branches.

Release manager @pablogsal, Is this worth rolling 3.10.10 and 3.11.2 releases for? At least two projects disrupted by the warning change have linked to this issue so far (python-prompt-toolkit and tqdm). There will presumably always be people using 3.10.9 and 3.11.1 so impacted projects may need to carry workarounds forever regardless. Questions to consider:

Is that just in those project's tests where DeprecationWarning is enabled?
Or does it impact their transitive users?
Who else calls the API in this situation getting implicit event loop creation that we don't yet know about?

From #93453:

But then we will have a weird situation, when some code emits a Deprecation warning in 3.10-3.11, but works without warning before 3.10 and after 3.11. Should we undeprecate this case in the future bugfix releases of 3.10 and/or 3.11? It would be not so easy.

The thing to do in this kind of situation is to either leave the existing warning as is in the stable releases, or remove it entirely. (It is fine if we marked something as deprecated in a release and later change our mind and opt not to deprecate it).

What I think happened here appears to be that an attempt to narrow the warning to reflect a new plan was made. This wound up moving the warning to happen from different API calls. The original released warning was removed (fine), but a new location raising the warning was added to a different call path. As seen in the tests: test_get_event_loop() added a new assertion that a DeprecationWarning was raised. That should be a red flag in stable releases. This kind of hiccup might be expected in a first patch 3.11.1 if it were correcting an accident for a warning just added in 3.11, but is never expected in a later lifecycle patch like 3.10.9.

We still need a Deprecation warning in 3.10-3.11 for the case when a new event loop is implicitly created.

Strictly speaking this was a bug that the warning wasn't raised in those releases. The lack of warning about an upcoming behavior change should be considered a reason to delay making the change. ie: restore the implicit event loop creation in 3.12 and remove it in 3.14. I reopened #93453 with such a comment.

serhiy-storchaka · 2022-12-11T11:16:19Z

Plan A:

Remove any deprecation warnings in 3.10 and 3.11.
Restore implicit creation of an event loop in 3.12 and add a deprecation warning.
Keep a warning in 3.13.
Remove implicit creation of an event loop in 3.14.

Plan B:

Remove any deprecation warnings in 3.10.
Keep a deprecation warning in 3.11.
Restore implicit creation of an event loop in 3.12 and add a deprecation warning.
Remove implicit creation of an event loop in 3.13.

Plan C:

Do nothing (except maybe changing the documentation more).

jaraco · 2022-12-11T15:10:43Z

Or does it impact their transitive users?

For prompt toolkit, it does affect transitive users including those of xonsh, who see the warning emitted when the shell starts.

gvanrossum · 2022-12-11T15:21:40Z

I think it has to be plan A.

blink1073 · 2022-12-19T12:58:21Z

FWIW you can add several jupyter packages to affected users. We would also prefer plan A.

gvanrossum · 2022-12-21T05:24:01Z

@serhiy-storchaka Is there a PR yet? Should I label this as release blocker for the next 3.10 and 3.11 releases?

pablogsal · 2022-12-21T14:19:57Z

I think it has to be plan A.

I concur with this

Should I label this as release blocker for the next 3.10 and 3.11 releases?

I went ahead and add it myself 👍

Partially revert changes made in pythonGH-93453. asyncio.DefaultEventLoopPolicy.get_event_loop() now emits a DeprecationWarning and creates and sets a new event loop instead of raising a RuntimeError if there is no current event loop set.

…_loop() They are deferred to Python 3.12.

jaraco added the type-bug An unexpected behavior, bug, or error label Dec 10, 2022

jaraco assigned serhiy-storchaka Dec 10, 2022

jaraco mentioned this issue Dec 10, 2022

Finish deprecation in asyncio.get_event_loop() #93453

Open

This was referenced Dec 11, 2022

python39: 3.9.15 -> 3.9.16; python310: 3.10.8 -> 3.10.9 NixOS/nixpkgs#204902

Merged

upstream asyncio changes cause breakage tqdm/tqdm#1407

Open

amotl mentioned this issue Dec 13, 2022

CI croaks on Python 3.12.0a3 panodata/grafana-wtf#46

Open

pablogsal added release-blocker 3.11 3.10 labels Dec 21, 2022

bedevere-bot mentioned this issue Dec 21, 2022

gh-100160: Deprecate implicit creation of an event loop #100410

Open

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Dec 21, 2022

pythongh-100160: Remove any deprecation warnings in asyncio.get_event…

19c9a03

…_loop() They are deferred to Python 3.12.

bedevere-bot mentioned this issue Dec 21, 2022

[3.11] gh-100160: Remove any deprecation warnings in asyncio.get_event_loop() #100412

Open

DeprecationWarning scope expanded in asyncio.events #100160

DeprecationWarning scope expanded in asyncio.events #100160

jaraco commented Dec 10, 2022 •

edited by bedevere-bot

gvanrossum commented Dec 10, 2022

kumaraditya303 commented Dec 10, 2022

gvanrossum commented Dec 10, 2022

gvanrossum commented Dec 10, 2022

serhiy-storchaka commented Dec 10, 2022

gvanrossum commented Dec 10, 2022

gpshead commented Dec 11, 2022

serhiy-storchaka commented Dec 11, 2022

jaraco commented Dec 11, 2022

gvanrossum commented Dec 11, 2022

blink1073 commented Dec 19, 2022

gvanrossum commented Dec 21, 2022

pablogsal commented Dec 21, 2022 •

edited

DeprecationWarning scope expanded in asyncio.events #100160

DeprecationWarning scope expanded in asyncio.events #100160

Comments

jaraco commented Dec 10, 2022 • edited by bedevere-bot

Linked PRs

gvanrossum commented Dec 10, 2022

kumaraditya303 commented Dec 10, 2022

gvanrossum commented Dec 10, 2022

gvanrossum commented Dec 10, 2022

serhiy-storchaka commented Dec 10, 2022

gvanrossum commented Dec 10, 2022

gpshead commented Dec 11, 2022

serhiy-storchaka commented Dec 11, 2022

Plan A:

Plan B:

Plan C:

jaraco commented Dec 11, 2022

gvanrossum commented Dec 11, 2022

blink1073 commented Dec 19, 2022

gvanrossum commented Dec 21, 2022

pablogsal commented Dec 21, 2022 • edited

jaraco commented Dec 10, 2022 •

edited by bedevere-bot

pablogsal commented Dec 21, 2022 •

edited