Skip to content

Subtle issue with borrowed references in extensions. #95797

Closed as not planned
Closed as not planned
@anamax22

Description

@anamax22

I think that the discussion of borrowing in Section 1.10 makes an untrue assumption.

In specific, I think that "When you pass an object reference into another function, in general, the function borrows the reference from you" is incomplete/misleading.

More to the point, "When a C function is called from Python, it borrows references to its arguments from the caller. The caller owns a reference to the object, so the borrowed reference’s lifetime is guaranteed until the function returns." seems wrong.

I don't have a clean documentation fix but I do have an example that demonstrates the problem.

Section 1.10.3 of https://docs.python.org/3/extending/extending.html ends with:

void
bug(PyObject *list)
{
    PyObject *item = PyList_GetItem(list, 0);
    Py_BEGIN_ALLOW_THREADS
    ...some blocking I/O call...
    Py_END_ALLOW_THREADS
    PyObject_Print(item, stdout, 0); /* BUG! */
}

It's a second example of "Could it perhaps do something to invalidate the reference to item in bug()? You bet! Assuming that the list passed into bug() is accessible to the del() method, it could execute a statement to the effect of del list[0], and assuming this was the last reference to that object, it would free the memory associated with it, thereby invalidating item." (from earlier in that section).

However, I'm reasonably certain that the following is also broken in a way that the discussion of borrowing (from the caller) doesn't address.

void
bug2(PyObject *list)
{
    Py_BEGIN_ALLOW_THREADS
    ...let other threads run during some blocking I/O call...
    Py_END_ALLOW_THREADS

    # Can the last reference to list be deleted by a thread that ran during
    # said blocking I/O call?
    # If so, this next line is likely to cause problems.
    PyObject *item = PyList_GetItem(list, 0);
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions