Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-45210: _Py_Dealloc() checks tp_dealloc exception #32357

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Copy link
Member

@vstinner vstinner commented Apr 6, 2022

If Python is built in debug mode, _Py_Dealloc() now ensures that the
tp_dealloc function leaves the current exception unchanged.

https://bugs.python.org/issue45210

If Python is built in debug mode, _Py_Dealloc() now ensures that the
tp_dealloc function leaves the current exception unchanged.
@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 6, 2022

See https://bugs.python.org/issue45210#msg416849 for a manual test on this PR.

@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 6, 2022

@serhiy-storchaka @methane @pablogsal: is it this change acceptable for a debug build? The intent is to detect bugs in C extensions. It's similar to _Py_CheckFunctionResult() and _Py_CheckSlotResult(). For example, if a deallocator function clears the current exception, it can be very problematic for code like:

PyErr_SetString(...);
Py_DECREF(obj);  // Call PyErr_Clear() !!!
return NULL;  // return NULL with no exception raised!?

Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

  1. old_exc_type is a borrowed reference. If the exception type is deallocated, comparing a pointer is an UB.
  2. What if the deallocator raised an exception with the same popular type?
  3. What if the exception was not normalized and the deallocator normalized it? Normalization of OSError changes the type.

@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 6, 2022

old_exc_type is a borrowed reference. If the exception type is deallocated, comparing a pointer is an UB.

Well, I made the assumption that a tp_dealloc function is not going to allocate new types which would get he same memory address. Are you suggesting to store a strong reference to the exception type? I tried to minimize _Py_Dealloc() overhead, since it's called often. I tried to avoid INCREF/DECREF on the type, but IMO the error message would be useless if we cannot dump the freed object and we cannot at least mention the type name.

What if the deallocator raised an exception with the same popular type?

If the old and new exception types are the same, the test doesn't notice that tp_dealloc raised a new exception. Do you mean that I should store the pointer to the exception instance ("exc value") and compare it to the new value as well?

What if the exception was not normalized and the deallocator normalized it? Normalization of OSError changes the type.

The deallocator must not normalize the current exception: it must not change the current exception. IMO the test must fail in this case.

Either it doesn't touch exceptions at all, or it must save and restore the current exception. See GH #28358 and https://bugs.python.org/issue45210.

@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 6, 2022

old_exc_type is a borrowed reference. If the exception type is deallocated, comparing a pointer is an UB.

If tp_dealloc deallocates the old exception type, allocates a new new exception type which gets the same address and tp_dealloc raises a new exception (bug!) with the new type: my test fails to detect the bug. Well. It's unfortunate. But at least, the test cannot crash Python (fatal error) with a false positive: it cannot fail even if the code is correct.

If you prefer, I can told strong references to the old exception type and to the old exception value.

@serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Apr 6, 2022

Are you suggesting to store a strong reference to the exception type?

Practically, it perhaps does not cause problems on common platforms. Not sure about WebAssembly. And UB is UB -- the result is unpredictable, it can be not only crash.

@tiran @mdickinson

@tiran
Copy link
Member

@tiran tiran commented Apr 6, 2022

Py_Debug builds don't work on WASM yet. The default stack size is too small.

@gpshead gpshead added 🔨 test-with-buildbots interpreter-core labels Apr 15, 2022
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Apr 15, 2022

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 3c57ddb 🤖

If you want to schedule another build, you need to add the "🔨 test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots label Apr 15, 2022
Copy link
Member

@gpshead gpshead left a comment

I'm generally in favor of these kinds of checks in debug builds. They help ferret out issues in our code. If possible for this check, consider keying off of #ifndef NDEBUG rather than #ifdef Py_DEBUG. More builds use NDEBUG than full --with-pydebug.

#ifdef Py_TRACE_REFS
_Py_ForgetReference(op);
#endif
(*dealloc)(op);

#ifdef Py_DEBUG
// bpo-45210: The tp_dealloc function must leave the current exception
Copy link
Member

@gpshead gpshead Apr 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update to the gh-XXX issue reference

@@ -559,7 +564,12 @@ def test_c_subclass_of_heap_ctype_with_del_modifying_dunder_class_only_decrefs_o
del subclass_instance

# Test that setting __class__ modified the reference counts of the types
self.assertEqual(type_refcnt - 1, _testcapi.HeapCTypeSubclassWithFinalizer.refcnt_in_del)
if Py_DEBUG:
# bpo-45210: In debug mode, _Py_Dealloc() keeps a strong reference
Copy link
Member

@gpshead gpshead Apr 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update to the gh-XXX reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting core review CLA signed interpreter-core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants