-
-
Notifications
You must be signed in to change notification settings - Fork 30.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
forked process in multiprocessing does not honour atexit #83856
Comments
I've talked with Pablo about this in person, and as advised opening the issue here now. I've discovered that forked processes do not honour atexit registrations. See the following example code: from multiprocessing import Process, set_start_method
import time
import os
import atexit
def cleanup():
print(f"cleanup {os.getpid()}")
atexit.register(cleanup)
def run():
time.sleep(0.1)
print(f"process done {os.getpid()}")
# atexit._run_exitfuncs()
if __name__ == "__main__":
set_start_method("fork")
process = Process(target=run)
process.start()
process.join()
print("app finished") In case of a forked process childs the atexit is never executed (note it works if I ran them manually at the end of the child process; so they're registered correctly). Switching to spawn method makes it work as expected. The behaviour is the same even if you call register within the child process (as opposed to being inherited during forking). Also found this StackOverflow question that mentions this https://stackoverflow.com/a/26476585. At the very least the documentation should explain this; though I'd expect atexit to be called before finalization of the fork processes (assuming the child process exits with 0 exit code). d |
See the PR and this comment for more context |
The discussion on the PR was useful. This is merged in time for 3.13beta1. It is a bit of a behavior change and a corner case that has existed for a long time and most code does not hit so I do not believe we should be backporting it as a bugfix despite it also being reasonably to call a bug. Thanks for the discussion and PR! |
…ythonGH-114279) Use atexit for all multiprocessing start methods to cleanup. See the pythonGH-114279 PR discussion and related issue for details as to why.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
The text was updated successfully, but these errors were encountered: