Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory #30434

Merged
merged 1 commit into from Jan 6, 2022
Merged

bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory #30434

merged 1 commit into from Jan 6, 2022

Conversation

@tiran
Copy link
Member

@tiran tiran commented Jan 6, 2022 •
edited by miss-islington

https://bugs.python.org/issue46263

Automerge-Triggered-By: GH:tiran
@tiran
Copy link
Member Author

@tiran tiran commented Jan 6, 2022

https://www.freebsd.org/cgi/man.cgi?query=jemalloc&apropos=0&sektion=0&manpath=FreeBSD+14.0-current&arch=default&format=html

  Additionally, --enable-debug is enabled in development versions of FreeBSD

   opt.junk	(const char *) r- [--enable-fill]
	   Junk	filling. If set	to "alloc", each byte of uninitialized
	   allocated memory will be initialized	to 0xa5. If set	to "free", all
	   deallocated memory will be initialized to 0x5a. If set to "true",
	   both	allocated and deallocated memory will be initialized, and if
	   set to "false", junk	filling	be disabled entirely. This is intended
	   for debugging and will impact performance negatively. This option
	   is "false" by default unless	--enable-debug is specified during
	   configuration, in which case	it is "true" by	default.

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Jan 6, 2022

@tiran: Status check is done, and it's a success ✅ .

@miss-islington miss-islington merged commit a4aa52d into python:main Jan 6, 2022
12 checks passed
@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Jan 6, 2022

Thanks @tiran for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒⛏🤖

miss-islington added a commit to miss-islington/cpython that referenced this issue Jan 6, 2022
@tiran @miss-islington
bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed m…
17bc26f 
…emory (pythonGH-30434)

Automerge-Triggered-By: GH:tiran
(cherry picked from commit a4aa52d)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Jan 6, 2022

GH-30436 is a backport of this pull request to the 3.10 branch.

miss-islington added a commit to miss-islington/cpython that referenced this issue Jan 6, 2022
@tiran @miss-islington
bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed m…
f25a2f3 
…emory (pythonGH-30434)

Automerge-Triggered-By: GH:tiran
(cherry picked from commit a4aa52d)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Jan 6, 2022

GH-30437 is a backport of this pull request to the 3.9 branch.

miss-islington added a commit that referenced this issue Jan 6, 2022
@tiran @miss-islington
bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed m…
b951dec 
…emory (GH-30434)

Automerge-Triggered-By: GH:tiran
(cherry picked from commit a4aa52d)

Co-authored-by: Christian Heimes <christian@python.org>
tiran added a commit that referenced this issue Jan 6, 2022
@tiran @miss-islington
[3.9] bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of f…
b259015 
…reed memory (GH-30434) (GH-30437)

Co-authored-by: Christian Heimes <christian@python.org>
gpshead
gpshead reviewed Jan 6, 2022
View changes
Lib/test/test_capi.py
out = assert_python_failure(
'-c', code,
PYTHONMALLOC=self.PYTHONMALLOC,
# FreeBSD: instruct jemalloc to not fill freed() memory
Copy link
Member

@gpshead gpshead Jan 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a rather fragile test as written. depending on the undefined behavior of what the contents of free()d memory will be is our problem. jemalloc is doing something perfectly reasonable, it's our test that is not. expect this to come up again at a random future point even if this workaround does something useful today.

Copy link
Member Author

@tiran tiran Jan 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other memory allocators may use different constants. I have an open PR for mimalloc microsoft/mimalloc#517

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gpshead gpshead

@pablogsal pablogsal

Assignees
No one assigned
Labels
CLA signed 🤖 automerge type-tests
Projects
None yet
Milestone
No milestone
6 participants
@tiran @miss-islington @bedevere-bot @gpshead @pablogsal @the-knights-who-say-ni