Skip to content

Regression in 3.14: setting __dict__ on custom type broken (mypyc, pybind11) #133912

Open
@henryiii

Description

@henryiii

Bug report

Bug description:

The work in #115776 by @markshannon broke mypyc and pybind11's tests related to pickling and setting __dict__ on custom types. Specially, #117750 introduced the regression for pybind11, I tested all the released for 3.14 then bisected between the commit after the fork point and 3.14.0a1 to find the commit in the PR above introduced the issue. More types are now inlined, but this breaks dict setting.

I've prepared a MWE without pybind11 or mypyc:

src/main.c

#define PY_SSIZE_T_CLEAN
#include <Python.h>

typedef struct {
    PyObject_VAR_HEAD
} ManagedDictObject;

int ManagedDict_traverse(PyObject *self, visitproc visit, void *arg) {
    PyObject_VisitManagedDict(self, visit, arg);
    Py_VISIT(Py_TYPE(self));
    return 0;
}

int ManagedDict_clear(PyObject *self) {
    PyObject_ClearManagedDict(self);
    return 0;
}

static PyGetSetDef ManagedDict_getset[] = {
    {"__dict__", PyObject_GenericGetDict, PyObject_GenericSetDict, NULL, NULL},
    {NULL, NULL, NULL, NULL, NULL},
};

static PyType_Slot ManagedDict_slots[] = {
    {Py_tp_new, (void *)PyType_GenericNew},
    {Py_tp_getset, (void *)ManagedDict_getset},
    {Py_tp_traverse, (void *)ManagedDict_traverse},
    {Py_tp_clear, (void *)ManagedDict_clear},
    {0}
};

static PyType_Spec ManagedDict_spec = {
    "manageddictbug.ManagedDict",
    sizeof(ManagedDictObject),
    0, // itemsize
    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_HEAPTYPE | Py_TPFLAGS_HAVE_GC,
    ManagedDict_slots
};

static PyModuleDef manageddictbugmodule = {
    PyModuleDef_HEAD_INIT,
    "manageddictbug",
    NULL,
    -1,
    NULL,
};

PyMODINIT_FUNC
PyInit_manageddictbug(void) {
    PyObject *m = PyModule_Create(&manageddictbugmodule);
    if (m == NULL)
        return NULL;

    PyObject *ManagedDictType = PyType_FromSpec(&ManagedDict_spec);
    if (ManagedDictType == NULL) {
        Py_DECREF(m);
        return NULL;
    }

    if (PyModule_AddObject(m, "ManagedDict", ManagedDictType) < 0) {
        Py_DECREF(ManagedDictType);
        Py_DECREF(m);
        return NULL;
    }

    return m;
}

pyproject.toml

[build-system]
requires = ["scikit-build-core"]
build-backend = "scikit_build_core.build"

[project]
name = "example-broken"
version = "0.1.0"

CMakeLists.txt

cmake_minimum_required(VERSION 3.15...4.0)
project(${SKBUILD_PROJECT_NAME} LANGUAGES C)

find_package(Python REQUIRED COMPONENTS Development.Module)

python_add_library(manageddictbug MODULE WITH_SOABI src/main.c)
install(TARGETS manageddictbug DESTINATION .)

example.py

import manageddictbug

obj = manageddictbug.ManagedDict()
obj.foo = 42
print(obj.foo)
print(obj.__dict__)

obj.__dict__ = {"bar": 3}
print(obj.__dict__)
print(obj.bar)
$ uv venv -p 3.13 -q && uv pip install . -q && .venv/bin/python example.py
42
{'foo': 42}
{'bar': 3}
3
$ uv venv -p ~/git/software/pybind11/.venv/bin/python3.14 -q && uv pip install . -q && .venv/bin/python example.py
42
{'foo': 42}
{'bar': 3}
Traceback (most recent call last):
  File "/Users/henryschreiner/git/scikit-build-proj/example_broken/example.py", line 13, in <module>
    print(obj.bar)
          ^^^^^^^
AttributeError: 'manageddictbug.ManagedDict' object has no attribute 'bar'

There is custom code in PyObject_GenericSetDict that is supposed to be handling the inline case, but maybe it wasn't hit before and is faulty?

CPython versions tested on:

3.14.0b1 (and back to 3.14.0a1)

Operating systems tested on:

macOS (and Linux in CI too; Windows CI broken due to https://gitlab.kitware.com/cmake/cmake/-/issues/26926, which I haven't opened a CPython issue for yet).

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)release-blockertype-bugAn unexpected behavior, bug, or error

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions