Pasting 🏳️‍🌈 crashes the new Python REPL #121609

treyhunner · 2024-07-11T00:48:27Z

Crash report

What happened?

To reproduce, start the new Python REPL in 3.13.0b3 and start to assign a string:

$ python3.13
Python 3.13.0b3 (main, Jul  2 2024, 13:24:06) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> flag = '

Then paste 🏳️‍🌈 and this happens:

$ python3.13
Python 3.13.0b3 (main, Jul  2 2024, 13:24:06) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> flag = 'Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/__main__.py", line 3, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/main.py", line 55, in interactive_console
    run_interactive(namespace)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/simple_interact.py", line 144, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/readline.py", line 385, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 768, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 751, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 695, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 672, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 360, in calc_screen
    l, l2 = disp_str(line)
            ~~~~~~~~^^^^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 61, in disp_str
    b.append(str_width(c))
             ~~~~~~~~~^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/utils.py", line 10, in str_width
    if ord(c) < 128:
       ~~~^^^
TypeError: ord() expected a character, but string of length 6 found

This looks like an issue with multiple-character glyphs, as 🏴󠁧󠁢󠁥󠁮󠁧󠁿 causes the same issue.

I found myself at my system command prompt after this (Python had exited).

I initially tested this on Python 3.13.0b3, but I also reproduced the issue on the current main branch.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0b3 (main, Jul 2 2024, 13:24:06) [GCC 11.4.0]

Linked PRs

The text was updated successfully, but these errors were encountered:

itamaro · 2024-07-11T01:48:14Z

confirmed the reproducer indeed crashes the repl on 3.14 (main branch) on macOS too (with a debug build and GIL disabled, ftr)

itamaro · 2024-07-11T01:50:46Z

for comparison, this is how it behaves on 3.12:

$ python3                                                                                                                                  [18:48:47]
Python 3.12.3 (v3.12.3:f6650f9ad7, Apr  9 2024, 08:18:47) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> flag = '🏳️\U+200D🌈'
>>> flag
'🏳️\u200d🌈'
>>> print(flag)
🏳️‍🌈

the '🏳️\U+200D🌈' sequence is what gets pasted when I paste the pride flag, not something I typed manually

Eclips4 · 2024-07-11T06:21:57Z

Hello! The type-crash label is intended for bugs where the interpreter exits with a segmentation fault or something similar.
The type-bug label is more applicable here since there is no hard crash of the interpreter.

mgmacias95 · 2024-07-13T08:32:10Z

I'm working on this :)

ods · 2024-07-13T09:04:05Z

@mgmacias95 It's just a typo, one extra copy-pasted line

--- i/Lib/_pyrepl/reader.py
+++ w/Lib/_pyrepl/reader.py
@@ -58,7 +58,6 @@ def disp_str(buffer: str) -> tuple[str, list[int]]:
         elif unicodedata.category(c).startswith("C"):
             c = r"\u%04x" % ord(c)
             s.append(c)
-            b.append(str_width(c))
             b.extend([0] * (len(c) - 1))
         else:
             s.append(c)

mgmacias95 · 2024-07-13T09:09:04Z

If that line is removed the emoji is pasted as:

mgmacias@mgmacias:~/cpython$ ./python
Python 3.14.0a0 (remotes/cpython/main:dc03ce797ae, Jul 13 2024, 08:30:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 🏳️\u200d🌈🏳️\u200d🌈🏳️\u200d🌈🏳️\u200d🌈🏳️\u200d🌈

In prior versions of python it was pasted correctly:

Python 3.12.2 (main, Feb  7 2024, 20:47:03) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 🏳️‍🌈🏳️‍🌈🏳️‍🌈🏳️‍🌈🏳️‍🌈🏳️‍🌈

ods · 2024-07-13T09:25:07Z

It looks like intended behaviour: the code explicitly replaces unprintable control/format/etc characters with their unicode escape sequence.

…ner (#121667)

…er joiner (pythonGH-121667) (cherry picked from commit e745996) Co-authored-by: Marta Gómez Macías <mgmacias@google.com>

…ter joiner (GH-121667) (#121733) Co-authored-by: Marta Gómez Macías <mgmacias@google.com>

hugovk · 2024-07-14T09:03:29Z

Thank you @treyhunner for the report, @mgmacias95 for the fix, and everyone else for the investigation!

…er joiner (python#121667)

treyhunner added the type-crash A hard crash of the interpreter, possibly with a core dump label Jul 11, 2024

itamaro added 3.13 bugs and security fixes 3.14 new features, bugs and security fixes topic-repl Related to the interactive shell labels Jul 11, 2024

Eclips4 added type-bug An unexpected behavior, bug, or error and removed type-crash A hard crash of the interpreter, possibly with a core dump labels Jul 11, 2024

bedevere-app bot mentioned this issue Jul 13, 2024

gh-121609: Fix pasting of characters containing unicode character joiner #121667

Merged

pablogsal pushed a commit that referenced this issue Jul 13, 2024

gh-121609: Fix pasting of characters containing unicode character joi…

e745996

…ner (#121667)

bedevere-app bot mentioned this issue Jul 13, 2024

[3.13] gh-121609: Fix pasting of characters containing unicode character joiner (GH-121667) #121733

Merged

hugovk pushed a commit that referenced this issue Jul 14, 2024

[3.13] gh-121609: Fix pasting of characters containing unicode charac…

e28f863

…ter joiner (GH-121667) (#121733) Co-authored-by: Marta Gómez Macías <mgmacias@google.com>

hugovk closed this as completed Jul 14, 2024

estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024

pythongh-121609: Fix pasting of characters containing unicode charact…

0b48231

…er joiner (python#121667)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pasting 🏳️‍🌈 crashes the new Python REPL #121609

Pasting 🏳️‍🌈 crashes the new Python REPL #121609

treyhunner commented Jul 11, 2024 •

edited by bedevere-app bot

Loading

itamaro commented Jul 11, 2024

itamaro commented Jul 11, 2024

Eclips4 commented Jul 11, 2024

mgmacias95 commented Jul 13, 2024

ods commented Jul 13, 2024

mgmacias95 commented Jul 13, 2024

ods commented Jul 13, 2024

hugovk commented Jul 14, 2024

Pasting 🏳️‍🌈 crashes the new Python REPL #121609

Pasting 🏳️‍🌈 crashes the new Python REPL #121609

Comments

treyhunner commented Jul 11, 2024 • edited by bedevere-app bot Loading

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

itamaro commented Jul 11, 2024

itamaro commented Jul 11, 2024

Eclips4 commented Jul 11, 2024

mgmacias95 commented Jul 13, 2024

ods commented Jul 13, 2024

mgmacias95 commented Jul 13, 2024

ods commented Jul 13, 2024

hugovk commented Jul 14, 2024

treyhunner commented Jul 11, 2024 •

edited by bedevere-app bot

Loading