Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

asyncio subprocess stdout occasionally lost (3.11.0 → 3.11.1 regression) #100133

Closed
AMDmi3 opened this issue Dec 9, 2022 · 7 comments · Fixed by #100154
Closed

asyncio subprocess stdout occasionally lost (3.11.0 → 3.11.1 regression) #100133

AMDmi3 opened this issue Dec 9, 2022 · 7 comments · Fixed by #100154
Assignees
Labels
3.11 3.12 expert-asyncio type-bug An unexpected behavior, bug, or error

Comments

@AMDmi3
Copy link
Contributor

AMDmi3 commented Dec 9, 2022

Bug report

So, I've updated python from 3.11.0 to 3.11.1 and one of my utilities which runs a lot of external processes with asyncio.create_subprocess_exec started failing in different places in weird ways. It turned out that with some probability asyncio.subprocess.Process.communicate() would now return an empty stdout. Here's a repro:

import asyncio

async def main():
    attempt = 1
    while True:
        proc = await asyncio.create_subprocess_exec('/bin/echo', 'test', stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)

        stdout, stderr = await proc.communicate()

        text = stdout.decode('utf-8').strip()

        if text != 'test':
            raise RuntimeError(f'FAIL on attempt {attempt}: output="{text}"')

        attempt += 1

asyncio.run(main())

You may have to wait somewhat for the problem to reproduce, but for me it fails in under 15 seconds more or less reliably. Possible output:

RuntimeError: FAIL on attempt 3823: output=""

Your environment

  • CPython versions tested on: 3.11.1
  • Operating system and architecture: FreeBSD 13.1 amd64

Linked PRs

@AMDmi3 AMDmi3 added the type-bug An unexpected behavior, bug, or error label Dec 9, 2022
@AMDmi3 AMDmi3 changed the title asyncio subprocess stdout lost (3.11.0 → 3.11.1 regression) asyncio subprocess stdout occasionally lost (3.11.0 → 3.11.1 regression) Dec 9, 2022
@AMDmi3
Copy link
Contributor Author

AMDmi3 commented Dec 9, 2022

@kumaraditya303 FYI, there were some commits to asyncio subprocess from you recently.

@mportesdev
Copy link

mportesdev commented Dec 9, 2022

I experienced this too, reported it here with some additional info:
https://discuss.python.org/t/python-3-11-1-async-subprocess-stdout-lost-when-piped/21788

@mportesdev
Copy link

mportesdev commented Dec 9, 2022

For me, the following script always results with an unexpected [b'', b'', b'baz\n'] both in Python 3.11.1 and 3.12.0a3.

import asyncio


async def get_command_stdout(cmd, *args):
    proc = await asyncio.create_subprocess_exec(
        cmd, *args, stdout=asyncio.subprocess.PIPE,
    )
    stdout, _ = await proc.communicate()
    return stdout


async def main():
    return await asyncio.gather(
        get_command_stdout('echo', 'foo'),
        get_command_stdout('echo', 'bar'),
        get_command_stdout('echo', 'baz'),
    )


if __name__ == '__main__':
    print(asyncio.run(main()))

@gvanrossum
Copy link
Member

gvanrossum commented Dec 9, 2022

Let’s bisect that repro to a specific commit.

@gvanrossum
Copy link
Member

gvanrossum commented Dec 12, 2022

The bisection points to

commit 7015e1379791cbf19908cd1a7c668a5d6e7165d5
Date:   Wed Oct 5 10:15:31 2022

    gh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (#32073)

We will get it fixed in 3.11.2.

@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Dec 12, 2022

Thanks for the bug report, I have a fix for this #100154, can you verify it that it works as you expect? Thanks @AMDmi3

@AMDmi3
Copy link
Contributor Author

AMDmi3 commented Dec 12, 2022

The fix works for me, thank you!

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Dec 21, 2022
…` output (pythonGH-100154)

(cherry picked from commit a7715cc)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
miss-islington added a commit that referenced this issue Dec 21, 2022
…ut (GH-100154)

(cherry picked from commit a7715cc)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
freebsd-git pushed a commit to freebsd/freebsd-ports that referenced this issue Dec 23, 2022
Fix losing stdout of asyncio.subprocess spawned processes.

PR:		268502
Approved by:	wen (python@)
Upstream issue:	python/cpython#100133
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 3.12 expert-asyncio type-bug An unexpected behavior, bug, or error
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

5 participants