Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Null characters in strings cause a C SystemError #97556

Open
apccurtiss opened this issue Sep 26, 2022 · 5 comments
Open

Null characters in strings cause a C SystemError #97556

apccurtiss opened this issue Sep 26, 2022 · 5 comments
Labels
3.9 3.10 3.11 3.12 type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@apccurtiss
Copy link

apccurtiss commented Sep 26, 2022

Crash report

Putting a null byte into a Python string causes a SystemError in Python 3.10, due to a call to strlen in the string parsing library. In Python 3.9, the following example runs without errors:

# -*- coding: latin-1 -*-
"""
<NULL>
"""

In Python 3.10, it raises SystemError: ../Parser/string_parser.c:219: bad argument to internal function.

Internally, the new string_parser library introduced in v3.10.0a1 uses a call to strlen to determine the string size, which is getting thrown off by the null byte. This call is actually unnecessary, as the length has already been calculated by the calling parser and can be retrieved with PyBytes_AsStringAndSize.

Error messages

For single line strings, the error is SystemError: Negative size passed to PyUnicode_New

For multiline strings, the error is SystemError: ../Parser/string_parser.c:219: bad argument to internal function

@apccurtiss apccurtiss added the type-crash A hard crash of the interpreter, possibly with a core dump label Sep 26, 2022
@vstinner
Copy link
Member

vstinner commented Sep 26, 2022

This issue seems like a duplicate of the issue #96670.

@apccurtiss
Copy link
Author

apccurtiss commented Sep 26, 2022

Ah, you're totally right. I found that one earlier, but misunderstood the core bug and re-discovered it. Thanks!

@apccurtiss
Copy link
Author

apccurtiss commented Sep 26, 2022

Actually, I just re-read that issue, and I was a bit over-eager with closing this. I feel like this is a very specific fix which doesn't actually solve their use case: This particular issue only shows up in 3.10, and causes an internal parse error. That linked issue is present in every 3.x version I've tested, including after this fix. I believe a patch elsewhere in the parser is needed for that.

@apccurtiss apccurtiss reopened this Sep 26, 2022
@gvanrossum
Copy link
Member

gvanrossum commented Sep 26, 2022

Can you submit a fix?

@apccurtiss
Copy link
Author

apccurtiss commented Sep 26, 2022

Submitted a fix in this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.9 3.10 3.11 3.12 type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Development

No branches or pull requests

3 participants