New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bpo-45210: _Py_Dealloc() checks tp_dealloc exception #32357
base: main
Are you sure you want to change the base?
Conversation
If Python is built in debug mode, _Py_Dealloc() now ensures that the tp_dealloc function leaves the current exception unchanged.
See https://bugs.python.org/issue45210#msg416849 for a manual test on this PR. |
@serhiy-storchaka @methane @pablogsal: is it this change acceptable for a debug build? The intent is to detect bugs in C extensions. It's similar to _Py_CheckFunctionResult() and _Py_CheckSlotResult(). For example, if a deallocator function clears the current exception, it can be very problematic for code like:
|
old_exc_type
is a borrowed reference. If the exception type is deallocated, comparing a pointer is an UB.- What if the deallocator raised an exception with the same popular type?
- What if the exception was not normalized and the deallocator normalized it? Normalization of OSError changes the type.
Well, I made the assumption that a tp_dealloc function is not going to allocate new types which would get he same memory address. Are you suggesting to store a strong reference to the exception type? I tried to minimize _Py_Dealloc() overhead, since it's called often. I tried to avoid INCREF/DECREF on the type, but IMO the error message would be useless if we cannot dump the freed object and we cannot at least mention the type name.
If the old and new exception types are the same, the test doesn't notice that tp_dealloc raised a new exception. Do you mean that I should store the pointer to the exception instance ("exc value") and compare it to the new value as well?
The deallocator must not normalize the current exception: it must not change the current exception. IMO the test must fail in this case. Either it doesn't touch exceptions at all, or it must save and restore the current exception. See GH #28358 and https://bugs.python.org/issue45210. |
If tp_dealloc deallocates the old exception type, allocates a new new exception type which gets the same address and tp_dealloc raises a new exception (bug!) with the new type: my test fails to detect the bug. Well. It's unfortunate. But at least, the test cannot crash Python (fatal error) with a false positive: it cannot fail even if the code is correct. If you prefer, I can told strong references to the old exception type and to the old exception value. |
Practically, it perhaps does not cause problems on common platforms. Not sure about WebAssembly. And UB is UB -- the result is unpredictable, it can be not only crash. |
|
I'm generally in favor of these kinds of checks in debug builds. They help ferret out issues in our code. If possible for this check, consider keying off of #ifndef NDEBUG
rather than #ifdef Py_DEBUG
. More builds use NDEBUG than full --with-pydebug
.
#ifdef Py_TRACE_REFS | ||
_Py_ForgetReference(op); | ||
#endif | ||
(*dealloc)(op); | ||
|
||
#ifdef Py_DEBUG | ||
// bpo-45210: The tp_dealloc function must leave the current exception |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update to the gh-XXX issue reference
@@ -559,7 +564,12 @@ def test_c_subclass_of_heap_ctype_with_del_modifying_dunder_class_only_decrefs_o | |||
del subclass_instance | |||
|
|||
# Test that setting __class__ modified the reference counts of the types | |||
self.assertEqual(type_refcnt - 1, _testcapi.HeapCTypeSubclassWithFinalizer.refcnt_in_del) | |||
if Py_DEBUG: | |||
# bpo-45210: In debug mode, _Py_Dealloc() keeps a strong reference |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update to the gh-XXX reference
If Python is built in debug mode, _Py_Dealloc() now ensures that the
tp_dealloc function leaves the current exception unchanged.
https://bugs.python.org/issue45210