Skip to content

Pasting 🏳️‍🌈 crashes the new Python REPL #121609

Closed
Closed
Pasting 🏳️‍🌈 crashes the new Python REPL#121609
Labels
3.13only security fixes3.14bugs and security fixestopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

Description

@treyhunner
treyhunner
opened on Jul 11, 2024

Crash report

What happened?

To reproduce, start the new Python REPL in 3.13.0b3 and start to assign a string:

$ python3.13
Python 3.13.0b3 (main, Jul  2 2024, 13:24:06) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> flag = '

Then paste 🏳️‍🌈 and this happens:

$ python3.13
Python 3.13.0b3 (main, Jul  2 2024, 13:24:06) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> flag = 'Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/__main__.py", line 3, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/main.py", line 55, in interactive_console
    run_interactive(namespace)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/simple_interact.py", line 144, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/readline.py", line 385, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 768, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 751, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 695, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 672, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 360, in calc_screen
    l, l2 = disp_str(line)
            ~~~~~~~~^^^^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/reader.py", line 61, in disp_str
    b.append(str_width(c))
             ~~~~~~~~~^^^
  File "/home/trey/.pyenv/versions/3.13.0b3/lib/python3.13/_pyrepl/utils.py", line 10, in str_width
    if ord(c) < 128:
       ~~~^^^
TypeError: ord() expected a character, but string of length 6 found

This looks like an issue with multiple-character glyphs, as 🏴󠁧󠁢󠁥󠁮󠁧󠁿 causes the same issue.

I found myself at my system command prompt after this (Python had exited).

I initially tested this on Python 3.13.0b3, but I also reproduced the issue on the current main branch.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0b3 (main, Jul 2 2024, 13:24:06) [GCC 11.4.0]

Linked PRs

Activity

  1. treyhunner
    added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Jul 11, 2024
  2. itamaro
    added
    3.13only security fixes
    3.14bugs and security fixes
    topic-replRelated to the interactive shell
    on Jul 11, 2024

  3. itamaro commented on Jul 11, 2024

    @itamaro
    itamaro
    on Jul 11, 2024
    Contributor

    confirmed the reproducer indeed crashes the repl on 3.14 (main branch) on macOS too (with a debug build and GIL disabled, ftr)

  4. itamaro commented on Jul 11, 2024

    @itamaro
    itamaro
    on Jul 11, 2024
    Contributor

    for comparison, this is how it behaves on 3.12:

    $ python3                                                                                                                                  [18:48:47]
Python 3.12.3 (v3.12.3:f6650f9ad7, Apr  9 2024, 08:18:47) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> flag = '🏳️\U+200D🌈'
>>> flag
'🏳️\u200d🌈'
>>> print(flag)
🏳️‍🌈

    the '🏳️\U+200D🌈' sequence is what gets pasted when I paste the pride flag, not something I typed manually

  5. Eclips4 commented on Jul 11, 2024

    @Eclips4
    Eclips4
    on Jul 11, 2024
    Member

    Hello! The type-crash label is intended for bugs where the interpreter exits with a segmentation fault or something similar.
    The type-bug label is more applicable here since there is no hard crash of the interpreter.

  6. Eclips4
    added
    type-bugAn unexpected behavior, bug, or error
    and removed
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Jul 11, 2024

  7. mgmacias95 commented on Jul 13, 2024

    @mgmacias95
    mgmacias95
    on Jul 13, 2024
    Contributor

    I'm working on this :)

  8. ods commented on Jul 13, 2024

    @ods
    ods
    on Jul 13, 2024

    @mgmacias95 It's just a typo, one extra copy-pasted line

    --- i/Lib/_pyrepl/reader.py
+++ w/Lib/_pyrepl/reader.py
@@ -58,7 +58,6 @@ def disp_str(buffer: str) -> tuple[str, list[int]]:
         elif unicodedata.category(c).startswith("C"):
             c = r"\u%04x" % ord(c)
             s.append(c)
-            b.append(str_width(c))
             b.extend([0] * (len(c) - 1))
         else:
             s.append(c)

  9. mgmacias95 commented on Jul 13, 2024

    @mgmacias95
    mgmacias95
    on Jul 13, 2024
    Contributor

    If that line is removed the emoji is pasted as:

    mgmacias@mgmacias:~/cpython$ ./python
Python 3.14.0a0 (remotes/cpython/main:dc03ce797ae, Jul 13 2024, 08:30:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 🏳️\u200d🌈🏳️\u200d🌈🏳️\u200d🌈🏳️\u200d🌈🏳️\u200d🌈

    In prior versions of python it was pasted correctly:

    Python 3.12.2 (main, Feb  7 2024, 20:47:03) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 🏳️‍🌈🏳️‍🌈🏳️‍🌈🏳️‍🌈🏳️‍🌈🏳️‍🌈

  10. ods commented on Jul 13, 2024

    @ods
    ods
    on Jul 13, 2024

    It looks like intended behaviour: the code explicitly replaces unprintable control/format/etc characters with their unicode escape sequence.

  11. pablogsal
    added a commit that references this issue on Jul 13, 2024

    gh-121609: Fix pasting of characters containing unicode character joi…

    e745996
  12. miss-islington
    added a commit that references this issue on Jul 13, 2024

    pythongh-121609: Fix pasting of characters containing unicode charact…

    7c1ceb9
  13. hugovk
    added a commit that references this issue on Jul 14, 2024

    [3.13] gh-121609: Fix pasting of characters containing unicode charac…

    e28f863

  14. hugovk commented on Jul 14, 2024

    @hugovk
    hugovk
    on Jul 14, 2024
    Member

    Thank you @treyhunner for the report, @mgmacias95 for the fix, and everyone else for the investigation!

  15. hugovk
    closed this as completedon Jul 14, 2024
  16. estyxx
    added a commit that references this issue on Jul 17, 2024

    pythongh-121609: Fix pasting of characters containing unicode charact…

    0b48231
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixestopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions