Skip to content

f-string string_parser.c regression: Assertion *str < end failed #93418

Closed

Description

@gpshead

Build an interpreter with assertions enabled (--with-pydebug works) and have it try to parse the following: (execute it, paste it in a repl, call literal_eval, use -c, etc.)

./python -c 'f"{i="'
(gdb) run -c 'f"{i="'
... the usual ...
python: ../gpshead/Parser/string_parser.c:806: fstring_find_expr: Assertion `*str < end' failed.

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49
49      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49
#1  0x00007ffff7c86546 in __GI_abort () at abort.c:79
#2  0x00007ffff7c8642f in __assert_fail_base (fmt=0x7ffff7dfcdf8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=0x55555590ca64 "*str < end", file=0x55555590c3d0 "../gpshead/Parser/string_parser.c", line=806, function=<optimized out>)
    at assert.c:92
#3  0x00007ffff7c95222 in __GI___assert_fail (assertion=assertion@entry=0x55555590ca64 "*str < end", 
    file=file@entry=0x55555590c3d0 "../gpshead/Parser/string_parser.c", line=line@entry=806, 
    function=function@entry=0x55555590cc30 <__PRETTY_FUNCTION__.6> "fstring_find_expr") at assert.c:101
#4  0x00005555556af7f2 in fstring_find_expr (p=p@entry=0x7ffff7761840, str=str@entry=0x7fffffffcfe0, end=end@entry=0x7ffff773cd25 "\"", 
    raw=raw@entry=0, recurse_lvl=recurse_lvl@entry=0, expr_text=expr_text@entry=0x7fffffffcf50, expression=0x7fffffffcf58, 
    first_token=0x7ffff773ad40, t=0x7ffff773ad40, last_token=0x7ffff773ad40) at ../gpshead/Parser/string_parser.c:783
#5  0x00005555556af92e in fstring_find_literal_and_expr (p=p@entry=0x7ffff7761840, str=str@entry=0x7fffffffcfe0, 
    end=end@entry=0x7ffff773cd25 "\"", raw=raw@entry=0, recurse_lvl=recurse_lvl@entry=0, literal=literal@entry=0x7fffffffcf48, 
    expr_text=0x7fffffffcf50, expression=0x7fffffffcf58, first_token=0x7ffff773ad40, t=0x7ffff773ad40, last_token=0x7ffff773ad40)
    at ../gpshead/Parser/string_parser.c:917
#6  0x00005555556afb17 in _PyPegen_FstringParser_ConcatFstring (p=p@entry=0x7ffff7761840, state=state@entry=0x7fffffffcff0, 
    str=str@entry=0x7fffffffcfe0, end=0x7ffff773cd25 "\"", raw=0, recurse_lvl=recurse_lvl@entry=0, first_token=0x7ffff773ad40, t=0x7ffff773ad40, 
    last_token=0x7ffff773ad40) at ../gpshead/Parser/string_parser.c:1142
#7  0x00005555556480dd in _PyPegen_concatenate_strings (p=p@entry=0x7ffff7761840, strings=strings@entry=0x555555c43560)
    at ../gpshead/Parser/action_helpers.c:911
#8  0x0000555555652607 in strings_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:15467
#9  0x0000555555661df1 in atom_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:14273
#10 0x000055555567ac83 in t_primary_raw (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:18112
#11 0x000055555567afdf in t_primary_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:17902
#12 0x000055555567f196 in single_subscript_attribute_target_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:17791
#13 0x000055555567fbb0 in _tmp_12_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:24111
#14 0x0000555555699f99 in assignment_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:2241
#15 0x000055555569af35 in simple_stmt_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:1648
#16 0x000055555569c67e in simple_stmts_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:1543
#17 0x00005555556a2bfe in statement_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:1366
#18 0x00005555556a2e61 in _loop1_3_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:23542
#19 0x00005555556a3040 in statements_rule (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:1298
#20 0x00005555556aa782 in file_rule (p=0x7ffff7761840) at ../gpshead/Parser/parser.c:1057
#21 0x00005555556ad811 in _PyPegen_parse (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/parser.c:38559
#22 0x0000555555645162 in _PyPegen_run_parser (p=p@entry=0x7ffff7761840) at ../gpshead/Parser/pegen.c:817
#23 0x0000555555645445 in _PyPegen_run_parser_from_string (str=str@entry=0x7ffff788f8a0 "f\"{i=\"\n", start_rule=start_rule@entry=257, 
    filename_ob=filename_ob@entry='<string>', flags=flags@entry=0x7fffffffd628, arena=arena@entry=0x7ffff7787940)
    at ../gpshead/Parser/pegen.c:932
#24 0x00005555556afc7c in _PyParser_ASTFromString (str=str@entry=0x7ffff788f8a0 "f\"{i=\"\n", filename='<string>', mode=mode@entry=257, 
    flags=flags@entry=0x7fffffffd628, arena=arena@entry=0x7ffff7787940) at ../gpshead/Parser/peg_api.c:14
#25 0x000055555584fa0e in PyRun_StringFlags (str=str@entry=0x7ffff788f8a0 "f\"{i=\"\n", start=start@entry=257, 
    globals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x555555bde730>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7907350>}, 
    locals={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x555555bde730>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7ffff7907350>}, flags=flags@entry=0x7fffffffd628) at ../gpshead/Python/pythonrun.c:1600
#26 0x000055555584fa8e in PyRun_SimpleStringFlags (command=0x7ffff788f8a0 "f\"{i=\"\n", flags=flags@entry=0x7fffffffd628)
    at ../gpshead/Python/pythonrun.c:486
#27 0x00005555558705a7 in pymain_run_command (command=<optimized out>) at ../gpshead/Modules/main.c:255

This comes from oss-fuzz https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47764 aka https://oss-fuzz.com/testcase-detail/5010707545980928

I minimized the input to be trivial. Do we really not have a regression test for this? We will! 😄

A simplified version of the full oss-fuzz input is:

f'{i=}{i=}{i=}{i=}{i:m{i=}{i=}{i=}{i=}{h=}{i=}i{i=}{i=}{i=}{i=}{i=}{i=}{i=}{i=}{i=}{i:}{y5}{i=}{i=}{i=}{i='/{''x\ިr]\r\srr@\n\]\]\25}=}{i=}{i:}{i=}{i=}{i=}{E=XXXE,bqE=XX,E=bE,bqE}{i=}{i=}{i=}{}{i=}{i=}{i=}{i5}as{i=}Bi=}{i=}{i=}{i=}{i=}{i=}{i=}{i}}y}}}}}}}}}}}}}}6r\r\n\rrr\]srr@\n\r\]'\r\srr\]\r\r@\n\Y\r\.2n\]\r\r'\n\\r\n\r\]\r\srr@\n\]\]128r\r\n\rrr\]srr@\n\r\]\r\srr@\n\]\r\rx\ިr\srr@\n\r\]\r\srr@\n\]\r\n\]\r\r@\n\\r\n\r\]\r\srr@\n\]\]\0r\r\n\rrr\]srr@\n\r\]\r\srr@\n\]\r\rx\ިr\srr@\n\r\]\\r\]\r\srr@\n\]\r\\ddddddddddddddd\r\]\r\srr@\n\]\dddddddddddddddddd]\2r\]\r\r\~'

If you want extra torture, but it really appears to just be the simple unclosed } ?

I did not test assertion enabled release builds to determine what backports are needed. probably 3.11, unknown others. needs verifying.

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Jun 1, 2022
  2. ericvsmith commented on Jun 1, 2022

    @ericvsmith
    Member

    I can duplicate this on Windows on main. I'm working on a fix.

  3. ericvsmith commented on Jun 1, 2022

    @ericvsmith
    Member

    Thanks for the catch! I should note that the code would have worked find in non-debug mode without the assert, but it's of course best to fix it to avoid the assert.

  4. added a commit that references this issue on Jun 1, 2022
  5. ericvsmith commented on Jun 1, 2022

    @ericvsmith
    Member

    Closed via #93419.

  6. added 2 commits that reference this issue on Jun 2, 2022
  7. added 2 commits that reference this issue on Jun 2, 2022
  8. vincedani commented on Feb 13, 2023

    @vincedani

    Hello @gpshead, it looks like you are (or were) fuzzing this repository, and you’ve found some interesting bugs. 🥇

    I would like to create a Python based test case reduction test suite that contains fuzzer generated outputs, and benchmark automatic test case reducers how they perform on Python inputs. It looks like to me you have opened this issue with the already reduced input that caused malfunction. Is it possible that you still have the output of the fuzzer, which is free of any reduction?

    I’m also interested in these issues of yours:

    with the same motivation.

    Thanks in advance,
    Daniel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

type-bugAn unexpected behavior, bug, or errortype-crashA hard crash of the interpreter, possibly with a core dump

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions