bpo-29877: compileall: import ProcessPoolExecutor only when needed #4856
Conversation
This comment has been minimized.
This comment has been minimized.
JulienPalard
changed the title
|
Hi, thanks for your contribution and welcome! Can't this be simplified by complexity removing the global and removing the |
|
@JulienPalard @methane if you remove the global, the import will happen for each file that is compiled. I know the second import isn't as expensive as the first import, but I'm not really sure how much overhead it would be. If you're fine with an reimport per-file, then I can change it and this will be much less complex. |
Importing ProcessPoolExecutor may hang or cause an error when the import accesses urandom on a low resource platform
virtuald
force-pushed
the
compileall-fix
branch
from
aec9815 to
5058e6d
Compare
|
Something wrong with the tests, I'll look at it later. |
|
Fixed the tests, and other improvements as suggested. |
It's negligible compared to compiling Python source. |
really? I thought |
|
Oops, this dropped off my radar. Still an issue though. :) Updated to simplify as asked. Maybe needs a rebase? |
|
We recommend "merge master" than "rebase". We use "Squash and merge" when merging PRs. CIs are failing. It likely because some tests for compilall are added. You need to merge master |
|
Yeah, I had just noticed that. I fixed the tests (had broken the ability to run unlimited workers with max_workers=0.. oops!), but haven't merged in master yet. I was running the tests locally by executing the test file directly (since it has a main) -- it seems that a6b3ec5 broke the ability to do that by adding a relative import to the file. If the tests pass on travis I'll merge in master and see what happens. |
|
Looks like it's all working finally. |
|
Thanks @virtuald for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7. |
…ythonGH-4856) Importing ProcessPoolExecutor may hang or cause an error when the import accesses urandom on a low resource platform https://bugs.python.org/issue29877 (cherry picked from commit 1d817e4) Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
|
GH-10686 is a backport of this pull request to the 3.7 branch. |
|
GH-10687 is a backport of this pull request to the 3.6 branch. |
…ythonGH-4856) Importing ProcessPoolExecutor may hang or cause an error when the import accesses urandom on a low resource platform https://bugs.python.org/issue29877 (cherry picked from commit 1d817e4) Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
…H-4856) Importing ProcessPoolExecutor may hang or cause an error when the import accesses urandom on a low resource platform https://bugs.python.org/issue29877 (cherry picked from commit 1d817e4) Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
…H-4856) Importing ProcessPoolExecutor may hang or cause an error when the import accesses urandom on a low resource platform https://bugs.python.org/issue29877 (cherry picked from commit 1d817e4) Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Importing ProcessPoolExecutor may hang or cause an error when the import
accesses urandom on a low resource platform
https://bugs.python.org/issue29877