Skip to content

/cpython

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-36051: drop GIL during large b''.join operations #17757

Open
wants to merge 3 commits into
base: master
from
Open

bpo-36051: drop GIL during large b''.join operations #17757

wants to merge 3 commits into from
+45 −19

Conversation

@bmerry
Copy link

bmerry commented Dec 30, 2019 •
edited by bedevere-bot

Improve multi-threaded performance by dropping the GIL in the fast path
of bytes.join. To avoid increasing overhead for small joins, it is only
done if the output size exceeds a threshold.

https://bugs.python.org/issue36051
@bmerry
bpo-36051: drop GIL during large b''.join operations
Loading status checks…
03d07fd 
Improve multi-threaded performance by dropping the GIL in the fast path
of bytes.join. To avoid increasing overhead for small joins, it is only
done if the output size exceeds a threshold.
@bmerry

This comment has been minimized.

Sign in to view
Copy link
Author

bmerry commented Dec 30, 2019

For now I've only made the change for the b''.join fast-path, but if there is agreement I could also add it to the general case (non-empty separator).

I think the code is safe against another thread mutating the sequence while the GIL is dropped (because the setup code takes it own refs to all the items), but it would be good if someone could double-check it.
bmerry added 2 commits Dec 31, 2019
@bmerry
bpo-36051: add extra tests for join()
0feabcf 
The existing stress test is increased in size to exceed GIL_THRESHOLD,
and a new stress test is added with an empty separator (since that is a
separate code path).
@bmerry
bpo-36051: adjustments to bytes.join GIL-dropping
Loading status checks…
4e783c1 
The GIL is now also dropped in the code path with non-empty separator,
but it is *not* dropped if any of the items is not a `bytes` object (see
discussion in bpo-36051 for reasoning).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA signed awaiting review
Projects
None yet
Milestone
No milestone
3 participants
@bmerry @the-knights-who-say-ni @bedevere-bot
You can’t perform that action at this time.