gh-96819: check if the length of a pipe write is not greater than 512 #96890

saito828koki · 2022-09-17T13:02:46Z

Issue: Multiprocessing resource tracker incorrectly checks pipe write length #96819

bedevere-bot · 2022-09-17T13:02:49Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

DanielNoord · 2022-09-19T21:47:13Z

Lib/multiprocessing/resource_tracker.py

@@ -161,7 +161,7 @@ def unregister(self, name, rtype):
    def _send(self, cmd, name, rtype):
        self.ensure_running()
        msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('ascii')
-        if len(name) > 512:
+        if len(msg) > 512:
            # posix guarantees that writes to a pipe of less than PIPE_BUF
            # bytes are atomic, and that PIPE_BUF >= 512
            raise ValueError('name too long')


Shouldn't this also be updated? ie. msg too long?

@DanielNoord
Thanks for your comment!
The error message should also be updated. I will fix it.

DanielNoord · 2022-09-20T07:39:03Z

Misc/NEWS.d/next/Library/2022-09-17-13-15-10.gh-issue-96819.6RfqM7.rst

@@ -0,0 +1 @@
+Fixed the block to guarantee that the length of a write to a pipe is not greater than PIPE_BUF


Suggested change

Fixed the block to guarantee that the length of a write to a pipe is not greater than PIPE_BUF

Fixed the check that guarantees that the length of a write to a pipe is not greater than PIPE_BUF

Thank you for correcting my English! Your suggestion seems to be great.

No worries! I struggle with writing good and short changelog entries myself many times! 😄

JelleZijlstra · 2022-09-30T03:44:48Z

Is it possible to write a unit test for this change?

saito828koki · 2022-10-01T12:35:47Z

Hello @JelleZijlstra ,

Thanks for your comment.
I added a test for this change. Could you review this PR?

kumaraditya303 · 2022-10-01T17:19:37Z

Misc/NEWS.d/next/Library/2022-09-17-13-15-10.gh-issue-96819.6RfqM7.rst

@@ -0,0 +1 @@
+Fixed the check that guarantees that the length of a write to a pipe is not greater than PIPE_BUF


Mention multiprocessing.resource_tracker in news.

Misc/NEWS.d/next/Library/2022-09-17-13-15-10.gh-issue-96819.6RfqM7.rst

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

miss-islington · 2022-10-03T00:41:04Z

Thanks @saito828koki for the PR, and @JelleZijlstra for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

bedevere-bot · 2022-10-03T00:41:15Z

GH-97745 is a backport of this pull request to the 3.11 branch.

…pipe write <= 512 (pythonGH-96890) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> (cherry picked from commit 19ca114) Co-authored-by: Koki Saito <49419225+saito828koki@users.noreply.github.com>

bedevere-bot · 2022-10-03T00:41:20Z

GH-97746 is a backport of this pull request to the 3.10 branch.

…pipe write <= 512 (pythonGH-96890) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> (cherry picked from commit 19ca114) Co-authored-by: Koki Saito <49419225+saito828koki@users.noreply.github.com>

…rite <= 512 (GH-96890) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> (cherry picked from commit 19ca114) Co-authored-by: Koki Saito <49419225+saito828koki@users.noreply.github.com>

* main: (2069 commits) pythongh-96512: Move int_max_str_digits setting to PyConfig (python#96944) pythongh-94808: Coverage: Check picklablability of calliter (python#95923) pythongh-94808: Add test coverage for PyObject_HasAttrString (python#96627) pythongh-94732: Fix KeyboardInterrupt race in asyncio run_forever() (python#97765) Fix typos in `bltinmodule.c`. (pythonGH-97766) pythongh-94808: `_PyLineTable_StartsLine` was not used (pythonGH-96609) pythongh-97681: Remove Tools/demo/ directory (python#97682) Fix typo in unittest docs (python#97742) pythongh-97728: Argument Clinic: Fix uninitialized variable in the Py_UNICODE converter (pythonGH-97729) pythongh-95913: Fix PEP number in PEP 678 What's New ref label (python#97739) pythongh-95913: Copyedit/improve New Modules What's New section (python#97721) pythongh-97740: Fix bang in Sphinx C domain ref target syntax (python#97741) pythongh-96819: multiprocessing.resource_tracker: check if length of pipe write <= 512 (python#96890) pythongh-97706: multiprocessing tests: Delete unused variable `rand` (python#97707) pythonGH-85447: Clarify docs about awaiting future multiple times (python#97738) [docs] Update logging cookbook with recipe for using a logger like an output… (pythonGH-97730) pythongh-97607: Fix content parsing in the impl-detail reST directive (python#97652) pythongh-95975: Move except/*/finally ref labels to more precise locations (python#95976) pythongh-97591: In `Exception.__setstate__()` acquire strong references before calling `tp_hash` slot (python#97700) pythongh-95588: Drop the safety claim from `ast.literal_eval` docs. (python#95919) ...

fix: check the length of msg, not the length of name

2efc372

bedevere-bot added the awaiting review label Sep 17, 2022

blurb-it bot and others added 2 commits Sep 17, 2022

📜🤖 Added by blurb_it.

476bed0

fix: remove backticks

ee548f9

DanielNoord reviewed Sep 19, 2022

View changes

fix: update error message

1566b72

DanielNoord approved these changes Sep 20, 2022

View changes

bedevere-bot added awaiting core review and removed awaiting review labels Sep 20, 2022

modify NEWS entry

3f32cb6

Add a test for too long name resources

1039648

kumaraditya303 requested changes Oct 1, 2022

View changes

kumaraditya303 added type-bug An unexpected behavior, bug, or error needs backport to 3.10 needs backport to 3.11 labels Oct 1, 2022

Modify NEWS entry

30f651c

kumaraditya303 reviewed Oct 1, 2022

View changes

Misc/NEWS.d/next/Library/2022-09-17-13-15-10.gh-issue-96819.6RfqM7.rst Outdated Show resolved Hide resolved

kumaraditya303 approved these changes Oct 1, 2022

View changes

Modify NEWS entry

6c3afb9

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

kumaraditya303 requested a review from JelleZijlstra Oct 1, 2022

JelleZijlstra approved these changes Oct 3, 2022

View changes

bedevere-bot added awaiting merge and removed awaiting core review labels Oct 3, 2022

JelleZijlstra merged commit 19ca114 into python:main Oct 3, 2022
14 checks passed

bedevere-bot removed the awaiting merge label Oct 3, 2022

bedevere-bot removed the needs backport to 3.11 label Oct 3, 2022

bedevere-bot removed the needs backport to 3.10 label Oct 3, 2022

gh-96819: check if the length of a pipe write is not greater than 512 #96890

gh-96819: check if the length of a pipe write is not greater than 512 #96890

saito828koki commented Sep 17, 2022 •

edited by bedevere-bot

bedevere-bot commented Sep 17, 2022

DanielNoord Sep 19, 2022

saito828koki Sep 20, 2022

DanielNoord Sep 20, 2022

saito828koki Sep 20, 2022

DanielNoord Sep 20, 2022

JelleZijlstra commented Sep 30, 2022

saito828koki commented Oct 1, 2022

kumaraditya303 Oct 1, 2022

miss-islington commented Oct 3, 2022

bedevere-bot commented Oct 3, 2022

bedevere-bot commented Oct 3, 2022

		@@ -0,0 +1 @@
		Fixed the block to guarantee that the length of a write to a pipe is not greater than PIPE_BUF

	Fixed the block to guarantee that the length of a write to a pipe is not greater than PIPE_BUF
	Fixed the check that guarantees that the length of a write to a pipe is not greater than PIPE_BUF

gh-96819: check if the length of a pipe write is not greater than 512 #96890

gh-96819: check if the length of a pipe write is not greater than 512 #96890

Conversation

saito828koki commented Sep 17, 2022 • edited by bedevere-bot

bedevere-bot commented Sep 17, 2022

DanielNoord Sep 19, 2022

Choose a reason for hiding this comment

saito828koki Sep 20, 2022

Choose a reason for hiding this comment

DanielNoord Sep 20, 2022

Choose a reason for hiding this comment

saito828koki Sep 20, 2022

Choose a reason for hiding this comment

DanielNoord Sep 20, 2022

Choose a reason for hiding this comment

JelleZijlstra commented Sep 30, 2022

saito828koki commented Oct 1, 2022

kumaraditya303 Oct 1, 2022

Choose a reason for hiding this comment

miss-islington commented Oct 3, 2022

bedevere-bot commented Oct 3, 2022

bedevere-bot commented Oct 3, 2022

saito828koki commented Sep 17, 2022 •

edited by bedevere-bot