Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

re: undocumented exception is raised #109747

Closed
fuzzah opened this issue Sep 22, 2023 · 6 comments
Closed

re: undocumented exception is raised #109747

fuzzah opened this issue Sep 22, 2023 · 6 comments
Assignees
@serhiy-storchaka
Labels
3.11 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes docs Documentation in the Doc dir topic-regex type-bug An unexpected behavior, bug, or error

Comments

@fuzzah
Copy link

fuzzah commented Sep 22, 2023 •
edited by bedevere-app bot

Bug report

Bug description:

The re module's documentation says it only raises the re.error exception, but the regex "\x00(?<!\x00{2147483648})" causes RuntimeError:

Python 3.11.5 (main, Sep 20 2023, 10:46:56) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> re.compile("\x00(?<!\x00{2147483647})")
re.compile('\x00(?<!\x00{2147483647})')
>>>
>>>
>>> re.compile("\x00(?<!\x00{2147483648})")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.11/re/__init__.py", line 227, in compile
    return _compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/re/_compiler.py", line 759, in compile
    return _sre.compile(
           ^^^^^^^^^^^^^
RuntimeError: invalid SRE code

Other re methods, such as match or split show the same result.

For brevity:
"\x00(?<!\x00{2147483648})" -> RuntimeError
"\x00(?<!\x00{2147483647})" -> no errors

I have found this with libFuzzer by testing the fuzz_sre_compile binary.

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Linked PRs
@fuzzah fuzzah added the type-bug An unexpected behavior, bug, or error label Sep 22, 2023
@hugovk hugovk added docs Documentation in the Doc dir topic-regex labels Sep 22, 2023
@AA-Turner
Copy link
Member

cc: @serhiy-storchaka @ezio-melotti

@AA-Turner
Copy link
Member

@fuzzah are you able to reproduce the error with 3.12.0rc3 or 3.13.0a0?

A

@fuzzah
Copy link
Author

fuzzah commented Sep 22, 2023

@AA-Turner , for both versions the answer is yes.

My checked out commit of 3.13.0a0 is this:

commit 7c553991724d8d537f8444db73f016008753d77a (grafted, HEAD -> main, origin/main, origin/HEAD)
Author: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Date:   Fri Sep 22 17:59:35 2023 +0100

    gh-109719: Fix missing jump target labels when compiler reorders cold/warm blocks (#109734)

@AA-Turner AA-Turner added 3.11 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes labels Sep 22, 2023
@fuzzah
Copy link
Author

fuzzah commented Sep 22, 2023

Might be related: #91700, #98740

@serhiy-storchaka
Copy link
Member

Confirm the bug. The RE engine has limitation on the width of look-behind pattern. But there were no checks for this or they did not work. I will see how it can be fixed.

@serhiy-storchaka serhiy-storchaka self-assigned this Sep 22, 2023
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Sep 25, 2023
@serhiy-storchaka
pythongh-109747: Improve errors for unsupported look-behind patterns
940b85f 
Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
@bedevere-app bedevere-app bot mentioned this issue Sep 25, 2023
Merged
serhiy-storchaka added a commit that referenced this issue Oct 14, 2023
@serhiy-storchaka
gh-109747: Improve errors for unsupported look-behind patterns (GH-10…
e2b3d83 
…9859)

Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 14, 2023
@serhiy-storchaka @miss-islington
pythongh-109747: Improve errors for unsupported look-behind patterns (p…
deb42aa 
…ythonGH-109859)

Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
(cherry picked from commit e2b3d83)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Oct 14, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 14, 2023
@serhiy-storchaka @miss-islington
pythongh-109747: Improve errors for unsupported look-behind patterns (p…
8f63581 
…ythonGH-109859)

Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
(cherry picked from commit e2b3d83)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Oct 14, 2023
Merged
serhiy-storchaka added a commit that referenced this issue Oct 14, 2023
@miss-islington @serhiy-storchaka
[3.12] gh-109747: Improve errors for unsupported look-behind patterns (…
a2cc9a4 
…GH-109859) (GH-110859)

Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
(cherry picked from commit e2b3d83)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this issue Oct 14, 2023
@miss-islington @serhiy-storchaka
[3.11] gh-109747: Improve errors for unsupported look-behind patterns (…
1c26f1c 
…GH-109859) (GH-110860)

Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
(cherry picked from commit e2b3d83)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@serhiy-storchaka
Copy link
Member

Now you will get better error message. Also, the limit is increased.

aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@serhiy-storchaka @aisk
pythongh-109747: Improve errors for unsupported look-behind patterns (p…
02b6c25 
…ythonGH-109859)

Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.

The limit is increased to 2**32-1 (was 2**31-1).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@serhiy-storchaka serhiy-storchaka

Labels
3.11 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes docs Documentation in the Doc dir topic-regex type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hugovk @serhiy-storchaka @AA-Turner @fuzzah