This also makes refactoring asyncio.wait_for to use asyncio.timeout difficult: #31847 (comment) difficult

Sorry, deleted my last comment since it was wrong. I'm going to try to dive into this a little to get some understanding.

This is a race condition. Two things need to happen on the same event loop iteration, but it turns out A happens before B, and not B before A. I don't think asyncio has a spec anywhere saying this is wrong, so a legitimate response might be to just live with it.

Maybe it's confusing to users though so we want to do something about it.

The event loop maintains two callback lists:

• self._ready - these are callbacks ready to be run right now. (actually a deque)
• self._scheduled - these are callbacks that will be ready at a certain time t. t might be in the past. (a list, but actually a heapq)

When processing a loop step, the event loop looks at _scheduled first, taking all callbacks that are due to run and appending them to _ready. So callbacks from _scheduled will run after the callbacks in _ready. Then it executes all callbacks in _ready.

timeout uses loop.call_at under the hood, and call_at uses self._scheduled.
sleep has a special case for 0 and negative values; this case uses loop.call_soon, and call_soon uses self._ready.

So this explains the issue:

0. loop iteration starts, amain() starts running
1. timeout puts a callback into loop._scheduled
2. sleep stops amain() and puts a callback into loop._ready
3. Another loop iteration starts
4. timeout callback gets appended to loop._ready, but the sleep callback is first
5. sleep callback runs, drives the coroutine to completion
6. no timeout exception, since when the timeout callback runs, the coroutine/task is done

A couple of proposals how to handle this:

A. do nothing, leave it up to the event loop how to resolve cases like this
B. change timeout to special case delays <= 0 to use loop.call_soon instead of loop.call_at. This would make the timeout callback go first, but the fix feels narrowly scoped, might leave the race condition still to happen under a different timeout.
C. special case loop.call_at to internally call loop._call_soon if the deadline is in the past.
D. change how the event loop processes callbacks. The scheduled callbacks could be prepended to self._ready instead of appended. This would again make the timeout callback go first. Also feels more intuitive to me as far as callback scheduling goes. I guess it'd make the event loop do slightly more work. Might also be backwards-incompatible, but it doesn't matter since it's unspecified?

My personal preference might be D, and then A.

@gvanrossum I know you've been in this space recently, would appreciate your feedback.