Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash on AST with misordered linenos #92597

Open
JelleZijlstra opened this issue May 10, 2022 · 7 comments
Open

Crash on AST with misordered linenos #92597

JelleZijlstra opened this issue May 10, 2022 · 7 comments
Labels
3.11 3.12 type-crash

Comments

@JelleZijlstra
Copy link
Member

@JelleZijlstra JelleZijlstra commented May 10, 2022

  • Compile 3.11 branch
  • ./python.exe -m ensurepip
  • ./python.exe -m pip install pytest-cov
  • ./python.exe -m pytest

Crashes with Assertion failed: (i->i_end_lineno >= i->i_lineno), function write_location_info_long_form, file compile.c, line 7532.

lldb stack trace:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #0: 0x00007fff203cd92e libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff203fc5bd libsystem_pthread.dylib`pthread_kill + 263
    frame #2: 0x00007fff20351406 libsystem_c.dylib`abort + 125
    frame #3: 0x00007fff203507d8 libsystem_c.dylib`__assert_rtn + 314
  * frame #4: 0x000000010027c183 python.exe`write_location_info_long_form.cold.2 at compile.c:7532:5 [opt]
    frame #5: 0x000000010018f469 python.exe`write_location_info_long_form(a=<unavailable>, i=<unavailable>, length=<unavailable>) at compile.c:7532:5 [opt]
    frame #6: 0x000000010018f27d python.exe`write_location_info_entry(a=0x00007ffeefbfe5d0, i=0x000000010388ac98, isize=1) at compile.c:7590:5 [opt]
    frame #7: 0x000000010018d2cd python.exe`assemble_emit_location(a=0x00007ffeefbfe5d0, i=0x000000010388ac98) at compile.c:7605:12 [opt]
    frame #8: 0x000000010017e9fa python.exe`assemble(c=<unavailable>, addNone=<unavailable>) at compile.c:8411:18 [opt]
    frame #9: 0x000000010017d4a3 python.exe`compiler_mod(c=0x00007ffeefbfe6b8, mod=<unavailable>) at compile.c:2203:10 [opt]
    frame #10: 0x000000010017d2b3 python.exe`_PyAST_Compile(mod=0x00000001038852b0, filename=0x000000010654d630, flags=<unavailable>, optimize=<unavailable>, arena=0x00000001064b0100) at compile.c:581:10 [opt]
    frame #11: 0x00000001001634e0 python.exe`builtin_compile_impl(module=<unavailable>, source=0x00000001065a3fc0, filename=0x000000010654d630, mode=<unavailable>, flags=<unavailable>, dont_inherit=1, optimize=-1, feature_version=-1) at bltinmodule.c:818:33 [opt]
    frame #12: 0x0000000100161fb6 python.exe`builtin_compile(module=<unavailable>, args=0x00007ffeefbfe7f0, nargs=<unavailable>, kwnames=<unavailable>) at bltinmodule.c.h:328:20 [opt]
    frame #13: 0x00000001000cbb08 python.exe`cfunction_vectorcall_FASTCALL_KEYWORDS(func=0x0000000100b9f4d0, args=0x0000000100c6fe00, nargsf=<unavailable>, kwnames=0x0000000102c72260) at methodobject.c:443:24 [opt]
    frame #14: 0x0000000100087018 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x0000000100b9f4d0, args=0x0000000100c6fe00, nargsf=9223372036854775811, kwnames=0x0000000102c72260) at pycore_call.h:92:11 [opt]
    frame #15: 0x0000000100086fa0 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:299:12 [opt]
    frame #16: 0x0000000100170d7c python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6fd70, throwflag=<unavailable>) at ceval.c:4826:23 [opt]
    frame #17: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
    frame #18: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000104737d80, locals=0x0000000000000000, args=0x00007ffeefbfeb80, argcount=3, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
    frame #19: 0x0000000100087330 python.exe`_PyFunction_Vectorcall(func=0x0000000104737d80, stack=0x00007ffeefbfeb80, nargsf=<unavailable>, kwnames=0x0000000000000000) at call.c:0:45 [opt]
    frame #20: 0x0000000100089fd8 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x0000000104737d80, args=0x00007ffeefbfeb80, nargsf=3, kwnames=0x0000000000000000) at pycore_call.h:92:11 [opt]
    frame #21: 0x0000000100089644 python.exe`method_vectorcall(method=<unavailable>, args=0x00000001064c9788, nargsf=<unavailable>, kwnames=0x0000000000000000) at classobject.c:89:18 [opt]
    frame #22: 0x0000000100086f3d python.exe`_PyVectorcall_Call(tstate=0x0000000100486930, func=(python.exe`method_vectorcall at classobject.c:44), callable=0x000000010640d310, tuple=0x00000001064c9770, kwargs=0x0000000000000000) at call.c:245:16 [opt]
    frame #23: 0x0000000100087148 python.exe`_PyObject_Call(tstate=0x0000000100486930, callable=0x000000010640d310, args=0x00000001064c9770, kwargs=0x0000000000000000) at call.c:328:16 [opt]
    frame #24: 0x0000000100087209 python.exe`PyObject_Call(callable=<unavailable>, args=<unavailable>, kwargs=<unavailable>) at call.c:355:12 [opt]
    frame #25: 0x0000000100177419 python.exe`do_call_core(tstate=0x0000000100486930, func=0x000000010640d310, callargs=0x00000001064c9770, kwdict=0x0000000000000000, use_tracing=<unavailable>) at ceval.c:0:13 [opt]
    frame #26: 0x00000001001733b0 python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6f580, throwflag=<unavailable>) at ceval.c:5431:22 [opt]
    frame #27: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
    frame #28: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000104053490, locals=0x0000000000000000, args=0x00000001064c0a48, argcount=1, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
    frame #29: 0x0000000100087330 python.exe`_PyFunction_Vectorcall(func=0x0000000104053490, stack=0x00000001064c0a48, nargsf=<unavailable>, kwnames=0x00000001064c8f50) at call.c:0:45 [opt]
    frame #30: 0x00000001000866b2 python.exe`_PyObject_FastCallDictTstate(tstate=0x0000000100486930, callable=0x0000000104053490, args=0x00007ffeefbfef70, nargsf=1, kwargs=0x00000001064c6d50) at call.c:152:15 [opt]
    frame #31: 0x00000001000875e4 python.exe`_PyObject_Call_Prepend(tstate=<unavailable>, callable=<unavailable>, obj=0x000000010622f3a0, args=<unavailable>, kwargs=<unavailable>) at call.c:482:24 [opt]
    frame #32: 0x00000001000ee1f9 python.exe`slot_tp_call(self=0x000000010622f3a0, args=0x000000010046c5a0, kwds=0x00000001064c6d50) at typeobject.c:7604:15 [opt]
    frame #33: 0x000000010008689c python.exe`_PyObject_MakeTpCall(tstate=0x0000000100486930, callable=0x000000010622f3a0, args=0x0000000100c6f408, nargs=<unavailable>, keywords=0x0000000104694960) at call.c:214:18 [opt]
    frame #34: 0x0000000100087049 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x000000010622f3a0, args=0x0000000100c6f408, nargsf=9223372036854775808, kwnames=0x0000000104694960) at pycore_call.h:90:16 [opt]
    frame #35: 0x0000000100086fa0 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:299:12 [opt]
    frame #36: 0x0000000100170d7c python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6f380, throwflag=<unavailable>) at ceval.c:4826:23 [opt]
    frame #37: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
    frame #38: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000102881b20, locals=0x0000000102830350, args=0x0000000000000000, argcount=0, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
    frame #39: 0x0000000100166615 python.exe`PyEval_EvalCode(co=<unavailable>, globals=0x0000000102830350, locals=0x0000000102830350) at ceval.c:1207:21 [opt]
    frame #40: 0x0000000100163b09 python.exe`builtin_exec_impl(module=<unavailable>, source=0x000000010617f230, globals=0x0000000102830350, locals=0x0000000102830350, closure=<unavailable>) at bltinmodule.c:1080:17 [opt]
    frame #41: 0x000000010016224d python.exe`builtin_exec(module=<unavailable>, args=0x0000000100c6f180, nargs=2, kwnames=0x0000000000000000) at bltinmodule.c.h:465:20 [opt]
    frame #42: 0x00000001000cbb08 python.exe`cfunction_vectorcall_FASTCALL_KEYWORDS(func=0x0000000100b9f6b0, args=0x0000000100c6f180, nargsf=<unavailable>, kwnames=0x0000000000000000) at methodobject.c:443:24 [opt]
    frame #43: 0x0000000100087018 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x0000000100b9f6b0, args=0x0000000100c6f180, nargsf=9223372036854775810, kwnames=0x0000000000000000) at pycore_call.h:92:11 [opt]
    frame #44: 0x0000000100086fa0 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:299:12 [opt]
    frame #45: 0x0000000100170d7c python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6f0d8, throwflag=<unavailable>) at ceval.c:4826:23 [opt]
    frame #46: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
    frame #47: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000102c5eaf0, locals=0x0000000000000000, args=0x0000000102894ba8, argcount=2, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
    frame #48: 0x0000000100087330 python.exe`_PyFunction_Vectorcall(func=0x0000000102c5eaf0, stack=0x0000000102894ba8, nargsf=<unavailable>, kwnames=0x0000000000000000) at call.c:0:45 [opt]
    frame #49: 0x0000000100086f3d python.exe`_PyVectorcall_Call(tstate=0x0000000100486930, func=(python.exe`_PyFunction_Vectorcall at call.c:385), callable=0x0000000102c5eaf0, tuple=0x0000000102894b90, kwargs=0x0000000000000000) at call.c:245:16 [opt]
    frame #50: 0x0000000100087148 python.exe`_PyObject_Call(tstate=0x0000000100486930, callable=0x0000000102c5eaf0, args=0x0000000102894b90, kwargs=0x0000000000000000) at call.c:328:16 [opt]
    frame #51: 0x0000000100087209 python.exe`PyObject_Call(callable=<unavailable>, args=<unavailable>, kwargs=<unavailable>) at call.c:355:12 [opt]
    frame #52: 0x00000001001d9f3d python.exe`pymain_run_module(modname=<unavailable>, set_argv0=<unavailable>) at main.c:300:14 [opt]
    frame #53: 0x00000001001d982d python.exe`pymain_run_python(exitcode=0x00007ffeefbff7dc) at main.c:595:21 [opt]
    frame #54: 0x00000001001d9718 python.exe`Py_RunMain at main.c:680:5 [opt]
    frame #55: 0x00000001001d99af python.exe`pymain_main(args=<unavailable>) at main.c:710:12 [opt]
    frame #56: 0x00000001001d99fb python.exe`Py_BytesMain(argc=<unavailable>, argv=<unavailable>) at main.c:734:12 [opt]
    frame #57: 0x0000000100001999 python.exe`main(argc=<unavailable>, argv=<unavailable>) at python.c:15:12 [opt]
    frame #58: 0x00007fff20417f3d libdyld.dylib`start + 1

Your environment

  • CPython versions tested on: 3.11 current head
  • Operating system and architecture: macOS

I put print statements in that show the file it crashes on is '/usr/local/lib/python3.11/site-packages/pytest_cov/plugin.py', but just running Python on that file doesn't reproduce the crash. I'll try to debug some more.
@JelleZijlstra JelleZijlstra added type-crash 3.11 3.12 labels May 10, 2022
@JelleZijlstra
Copy link
Member Author

@JelleZijlstra JelleZijlstra commented May 10, 2022 •
edited

Simple repro:

from ast import *
tree = Module(body=[
    Import(names=[alias(name='builtins', lineno=1, col_offset=0)], lineno=1, col_offset=0), 
    Import(names=[alias(name='traceback', lineno=0, col_offset=0)], lineno=0, col_offset=1)
], type_ignores=[])
compile(tree, "x", "exec", dont_inherit=True)
% ./python.exe wronglinenos.py 
Assertion failed: (i->i_end_lineno >= i->i_lineno), function write_location_info_long_form, file compile.c, line 7532.
zsh: abort      ./python.exe wronglinenos.py

@JelleZijlstra JelleZijlstra changed the title Compiler crash on pytest-cov Crash on AST with misordered linenos May 10, 2022
@kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented May 10, 2022

944fffe is the first bad commit

commit 944fffee8916cb94321fa33cd3a43f4108717746
Author: Mark Shannon <mark@hotpy.org>
Date:   Thu Apr 21 16:10:37 2022 +0100

    GH-88116: Use a compact format to represent end line and column offsets. (GH-91666)
    
    * Stores all location info in linetable to conform to PEP 626.
    
    * Remove column table from code objects.
    
    * Remove end-line table from code objects.
    
    * Document new location table format

@iritkatriel
Copy link
Member

@iritkatriel iritkatriel commented May 10, 2022

@markshannon

Looks like the ast defaults the end_lineno to None, which then becomes 0 in the compiler. Maybe the ast should default it be equal to lineno? Or should None become -1 in c?

@markshannon
Copy link
Member

@markshannon markshannon commented May 10, 2022

Both, perhaps.
If no end_lineno is specified, then it should probably default to lineno
If the end_lineno is None then we should convert that to -1, as we do for lineno.

@iritkatriel
Copy link
Member

@iritkatriel iritkatriel commented May 11, 2022

cc @pablogsal

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 11, 2022

Both, perhaps.
If no end_lineno is specified, then it should probably default to lineno
If the end_lineno is None then we should convert that to -1, as we do for lineno.

That's what we do in some parts that deal with these information. I can prepare a PR

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 11, 2022

The question is if we should manage this in the AST creation routines or in the compiler. I assume that the first is more resilient but what do others think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.11 3.12 type-crash
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@JelleZijlstra @iritkatriel @markshannon @pablogsal @kumaraditya303