Skip to content

Subprocess module calling shell commands only using one physical core #103555

Closed as not planned
@cap-jmk

Description

@cap-jmk

Bug report

When invoking subprocess in either pytest or invoking pytest from a subprocess the command is using only one physical core.
However invoking the commands myself makes use of all cores. It seems like only heavy loads experience the problem as small loads distribute the calculations across physical cores.

Your environment

  • CPython versions tested on: 3.10
  • Operating system and architecture: Ubuntu 22.04, Endeavor OS

I also tried to diagnose the bug a little bit. You can refer to these two issues as a reference

Summary of debugging done so far

* using os -> is also not recognized by the kernel properly
* not invoking a shell
* invoking a shell 
* installing cmake and gcc
* using multithreading 
* using multiprocessing 
* using a bash script with subprocess

They all happen to work on one physical core.

Yet, running pytest in the plain shell works! From what I see, the system is ready for high performance but subprocess does not communicate well with the kernel for some reason.

Metadata

Metadata

Assignees

No one assigned

    Labels

    pendingThe issue will be closed if no feedback is providedtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions