Reference counting in Python is complicated. It's never obvious if we are holding a strong reference or a borrowed reference to a Python object.
If we have a borrowed reference to a Python object, to make sure that it cannot be deleted while we execute arbitrary Python code, it's needed to creating a temporary strong reference by increasing temporarily its reference counter. Example from Modules/_abc.c:
subclasses = PyObject_CallMethod(self, "__subclasses__", NULL);
(...)
PyObject *scls = PyList_GET_ITEM(subclasses, pos);
Py_INCREF(scls); // create a temporary strong reference
int r = PyObject_IsSubclass(subclass, scls);
Py_DECREF(scls); // delete the temporary strong reference
The scls variable is assigned to a borrowed reference to a list item. The list is the result of the __subclasses__() method. The code creates a temporarily strong reference to call PyObject_IsSubclass().
IMO in terms of semantics, such code is hard to understand if read aloud in English, since INCREF and DECREF modify the object in-place. Py_INCREF() and Py_DECREF() are like the low-level implementation, I would prefer to have an abstraction on top of it.
That's also why I added Py_NewRef() in Python 3.10: Py_NewRef() semantics is simpler: it creates a new strong reference. In terms of semantics, it doesn't modify the object in-place, even if it's possible to write something like var = Py_NewRef(var) (I discourage to write such code). See issue #99300 for the usage of the Py_NewRef() function.
It would be nice to have a macro to clarify the scope of the temporary strong reference rather than having to INCREF/DECREF temporarily. Something like:
In programming languages like C++, it would be possible to create such API by relying on destructors. There are CPython C++ binding which uses such API to hold the GIL for example: the GIL is released at the end of the scope of a variable which "holds the GIL".
vstinner commentedNov 14, 2022
Reference counting in Python is complicated. It's never obvious if we are holding a strong reference or a borrowed reference to a Python object.
If we have a borrowed reference to a Python object, to make sure that it cannot be deleted while we execute arbitrary Python code, it's needed to creating a temporary strong reference by increasing temporarily its reference counter. Example from
Modules/_abc.c
:The scls variable is assigned to a borrowed reference to a list item. The list is the result of the
__subclasses__()
method. The code creates a temporarily strong reference to callPyObject_IsSubclass()
.IMO in terms of semantics, such code is hard to understand if read aloud in English, since INCREF and DECREF modify the object in-place. Py_INCREF() and Py_DECREF() are like the low-level implementation, I would prefer to have an abstraction on top of it.
That's also why I added
Py_NewRef()
in Python 3.10: Py_NewRef() semantics is simpler: it creates a new strong reference. In terms of semantics, it doesn't modify the object in-place, even if it's possible to write something likevar = Py_NewRef(var)
(I discourage to write such code). See issue #99300 for the usage of the Py_NewRef() function.It would be nice to have a macro to clarify the scope of the temporary strong reference rather than having to INCREF/DECREF temporarily. Something like:
Example of usage:
Example holding two (different) strong references:
Alternative compact syntax:
Obviously, the macro name is open for bikeshedding :-)
The text was updated successfully, but these errors were encountered: