Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RawArray causes FileNotFoundError at cleanup #79429

Closed
MathieuLamarre mannequin opened this issue Nov 14, 2018 · 3 comments
Closed

RawArray causes FileNotFoundError at cleanup #79429

MathieuLamarre mannequin opened this issue Nov 14, 2018 · 3 comments
Labels
stdlib Python modules in the Lib dir topic-multiprocessing type-bug An unexpected behavior, bug, or error

Comments

@MathieuLamarre
Copy link
Mannequin

MathieuLamarre mannequin commented Nov 14, 2018 •
edited by bedevere-app bot

BPO 35248

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-11-14.17:50:30.017>
labels = ['type-bug', 'library']
title = 'RawArray causes FileNotFoundError at cleanup'
updated_at = <Date 2018-11-14.18:17:03.678>
user = 'https://bugs.python.org/MathieuLamarre'

bugs.python.org fields:

activity = <Date 2018-11-14.18:17:03.678>
actor = 'Mathieu Lamarre'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-11-14.17:50:30.017>
creator = 'Mathieu Lamarre'
dependencies = []
files = []
hgrepos = []
issue_num = 35248
keywords = []
message_count = 2.0
messages = ['329920', '329924']
nosy_count = 1.0
nosy_names = ['Mathieu Lamarre']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35248'
versions = ['Python 3.6']

Linked PRs
@MathieuLamarre MathieuLamarre mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Nov 14, 2018
@MathieuLamarre
Copy link
Mannequin Author

MathieuLamarre mannequin commented Nov 14, 2018

Running:

from multiprocessing.sharedctypes import RawArray
from ctypes import c_uint32
if __name__ == '__main__':
    shared_array = RawArray(c_uint32, 1500)
Causes:
Traceback (most recent call last):
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers
    finalizer()
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/multiprocessing/util.py", line 186, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/shutil.py", line 480, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'pym-10314-v8aznmmb'

Python 3.6.6 |Anaconda, Inc.| (default, Oct 9 2018, 12:34:16)
[GCC 7.3.0] on linux

@MathieuLamarre
Copy link
Mannequin Author

MathieuLamarre mannequin commented Nov 14, 2018

To repro, the temp directory returned by tempfile must be a network mount to reproduce this error. I have this on server with minimal disk space in /home and TMPDIR points to CIFS mounted volume. So maybe can only be reproduced if shutils operations on the temp dir are slow (or buggy).

We have been using a network temp dir in a large project and RawArray is the only object giving us error so far. Maybe the error could be ignored, if it's only dual delete issue.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Dec 8, 2023
@serhiy-storchaka
pythongh-79429: Ignore FileNotFoundError when remove a temporary dire…
a29ef64 
…ctory in the multiprocessing finalizer
@bedevere-app bedevere-app bot mentioned this issue Dec 8, 2023
Merged
serhiy-storchaka added a commit that referenced this issue Dec 9, 2023
@serhiy-storchaka
[3.12] gh-79429: Ignore FileNotFoundError when remove a temporary dir…
7e82c62 
…ectory in the multiprocessing finalizer (GH-112865)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Dec 9, 2023
@serhiy-storchaka @miss-islington
[3.12] pythongh-79429: Ignore FileNotFoundError when remove a tempora…
3386d97 
…ry directory in the multiprocessing finalizer (pythonGH-112865)

(cherry picked from commit 7e82c62)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Dec 9, 2023
Merged
@serhiy-storchaka
Copy link
Member

shutil.rmtree() ignores FileNotFound errors by default, so it should only be fixed in 3.11 and 3.12.

serhiy-storchaka added a commit that referenced this issue Dec 9, 2023
@miss-islington @serhiy-storchaka
[3.11] gh-79429: Ignore FileNotFoundError when remove a temporary dir…
2d8012f 
…ectory in the multiprocessing finalizer (GH-112865) (GH-112897)

(cherry picked from commit 7e82c62)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stdlib Python modules in the Lib dir topic-multiprocessing type-bug An unexpected behavior, bug, or error
Projects
Multiprocessing issues
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vstinner @serhiy-storchaka