Putting a null byte into a Python string causes a SystemError in Python 3.10, due to a call to strlen in the string parsing library. In Python 3.9, the following example runs without errors:
# -*- coding: latin-1 -*-
"""
<NULL>
"""
In Python 3.10, it raises SystemError: ../Parser/string_parser.c:219: bad argument to internal function.
Internally, the new string_parser library introduced in v3.10.0a1 uses a call to strlen to determine the string size, which is getting thrown off by the null byte. This call is actually unnecessary, as the length has already been calculated by the calling parser and can be retrieved with PyBytes_AsStringAndSize.
Error messages
For single line strings, the error is SystemError: Negative size passed to PyUnicode_New
For multiline strings, the error is SystemError: ../Parser/string_parser.c:219: bad argument to internal function
The text was updated successfully, but these errors were encountered:
Actually, I just re-read that issue, and I was a bit over-eager with closing this. I feel like this is a very specific fix which doesn't actually solve their use case: This particular issue only shows up in 3.10, and causes an internal parse error. That linked issue is present in every 3.x version I've tested, including after this fix. I believe a patch elsewhere in the parser is needed for that.
apccurtiss commentedSep 26, 2022
Crash report
Putting a null byte into a Python string causes a SystemError in Python 3.10, due to a call to strlen in the string parsing library. In Python 3.9, the following example runs without errors:
In Python 3.10, it raises
SystemError: ../Parser/string_parser.c:219: bad argument to internal function
.Internally, the new string_parser library introduced in v3.10.0a1 uses a call to strlen to determine the string size, which is getting thrown off by the null byte. This call is actually unnecessary, as the length has already been calculated by the calling parser and can be retrieved with
PyBytes_AsStringAndSize
.Error messages
For single line strings, the error is
SystemError: Negative size passed to PyUnicode_New
For multiline strings, the error is
SystemError: ../Parser/string_parser.c:219: bad argument to internal function
The text was updated successfully, but these errors were encountered: