make altinstall for PGO is not parallel-safe #93584
Comments
|
My PR solved the race condition in @pablogsal 's problem is caused by the fact that PGO builds use recursive make. The |
All install targets use the "all" target as synchronization point to prevent race conditions with PGO builds. PGO builds use recursive make, which can lead to two parallel `./python setup.py build` processes that step on each others toes. "test" targets now correctly compile PGO build in a clean repo.
All install targets use the "all" target as synchronization point to prevent race conditions with PGO builds. PGO builds use recursive make, which can lead to two parallel `./python setup.py build` processes that step on each others toes. "test" targets now correctly compile PGO build in a clean repo.
|
Weird, I never hit a compilation failure after 2 hours of trying on my 32 core machine, maybe this is an edge case you can only hit with the macOS scheduler. |
|
Did you create a build with |
|
Yeah, I was running a script in a loop using the steps from Pablo's reproducer here: #84461 (comment) After 2 hours it still hadn't failed |
That's odd. I have been able to reproduce in at least 3 systems, including Linux with arch Linux and RHEL7 |
|
In comment #93589 (comment) @nascheme suggested to rename some of the targets to make the intention more clear. I like the idea, but I don't want to mix improvements with a release blocker fix. |
tiran
changed the title
All install targets use the "all" target as synchronization point to prevent race conditions with PGO builds. PGO builds use recursive make, which can lead to two parallel `./python setup.py build` processes that step on each others toes. "test" targets now correctly compile PGO build in a clean repo.
…ythonGH-93589) All install targets use the "all" target as synchronization point to prevent race conditions with PGO builds. PGO builds use recursive make, which can lead to two parallel `./python setup.py build` processes that step on each others toes. "test" targets now correctly compile PGO build in a clean repo. (cherry picked from commit 243ed54) Co-authored-by: Christian Heimes <christian@python.org>
) (GH-93603) All install targets use the "all" target as synchronization point to prevent race conditions with PGO builds. PGO builds use recursive make, which can lead to two parallel `./python setup.py build` processes that step on each others toes. "test" targets now correctly compile PGO build in a clean repo. (cherry picked from commit 243ed54) Co-authored-by: Christian Heimes <christian@python.org>
gpshead
changed the title
|
@pablogsal As far as I know this bug has been fixed by GH-93589. |
Bug report
setup.pyhas a helper methodadd_multiarch_paths. The method creates a temporary filebuild/lib.{platform}/multiarchand unlinks it at the end of the function call. This is not parallel-safe.PGO builds of Python use
$(MAKE), so called recursive make. Recursive makes are considered harmful because the main make process has no understanding what the child make process is doing. With heavy parallel makes this can cause race conditions.The combination of unsafe
add_multiarch_paths(), recursive make and loooots of CPU cores can lead to build issues like #84461 (comment).Possible workarounds
$(MAKE)CC -print-multiarchanddpkg-architecture ... -qDEB_HOST_MULTIARCHcalls toconfigure.ac_bootsubprocess.check_output()andadd_multiarch_paths()to use a more safe tmp fileOption (3) is the simplest approach. Even
tmpfile = os.path.join(self.build_temp, f'multiarch-{os.getpid()}')would be good enough to avoid file conflicts in parallel builds.The text was updated successfully, but these errors were encountered: