Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Callable iterator can SystemError if call exhausts the iterator #101892

Closed
ionite34 opened this issue Feb 13, 2023 · 0 comments
Closed

Callable iterator can SystemError if call exhausts the iterator #101892

ionite34 opened this issue Feb 13, 2023 · 0 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@ionite34
Copy link
Contributor

ionite34 commented Feb 13, 2023

Bug report

If the callable of a callable iterator (with sentinel) exhausts the iterator itself during the call, a SystemError occurs.

Example discovered by @chilaxan

def bug():
    if bug.clear:
        return 1
    else:
        bug.clear = True
        list(bug.iterator)
        return 0

bug.iterator = iter(bug, 1)
bug.clear = False
next(bug.iterator)
SystemError: Objects\object.c:722: bad argument to internal function

Likely cause

iterobject.calliter_iternext does not check whether it->it_sentinel is NULL after the _PyObject_CallNoArgs call
https://github.com/python/cpython/blob/main/Objects/iterobject.c#L207-L237

static PyObject *
calliter_iternext(calliterobject *it)
{
    PyObject *result;

    if (it->it_callable == NULL) {
        return NULL;
    }

+   result = _PyObject_CallNoArgs(it->it_callable);
    if (result != NULL) {
        int ok;

+       ok = PyObject_RichCompareBool(it->it_sentinel, result, Py_EQ);
        if (ok == 0) {
            return result; /* Common case, fast path */
        }
(...)

Your environment

Appears to affect 3.7-3.12

Linked PRs

@ionite34 ionite34 added the type-bug An unexpected behavior, bug, or error label Feb 13, 2023
workingpayload added a commit to workingpayload/cpython that referenced this issue Feb 14, 2023
Updated Cython/objects/iterobject.c Line 217
@arhadthedev arhadthedev added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Feb 14, 2023
workingpayload added a commit to workingpayload/cpython that referenced this issue Feb 15, 2023
kumaraditya303 pushed a commit that referenced this issue Mar 4, 2023
…he iterator (#101896)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 4, 2023
…usts the iterator (pythonGH-101896)

(cherry picked from commit 705487c)

Co-authored-by: Raj <51259329+workingpayload@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
miss-islington added a commit that referenced this issue Mar 4, 2023
…he iterator (GH-101896)

(cherry picked from commit 705487c)

Co-authored-by: Raj <51259329+workingpayload@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
arhadthedev pushed a commit to arhadthedev/cpython that referenced this issue Mar 4, 2023
…usts the iterator (python#101896)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
(cherry picked from commit 705487c)
kumaraditya303 pushed a commit that referenced this issue Mar 4, 2023
…austs the iterator (GH-101896) (#102422)

gh-101892: Fix `SystemError` when a callable iterator call exhausts the iterator (#101896)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
(cherry picked from commit 705487c)

Co-authored-by: Raj <51259329+workingpayload@users.noreply.github.com>
hugovk pushed a commit to hugovk/cpython that referenced this issue Mar 6, 2023
…usts the iterator (python#101896)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants