Conversation
Fixes Keats#1035. The parser already bounds its own recursion via MAX_RECURSION_DEPTH, but that guard only fires for right-nested forms built through inner_parse_expression (parentheses, unary operators, **). Left-associative chains -- `1 + 1 + ...`, `a | f | f | ...`, `a.b.c...` -- are built iteratively by the Pratt loop and never increment recursion_depth, so they sail past the parser untouched. Compiler::compile_expr then walks that same left-leaning AST recursively with no guard of its own, so a template with a few thousand chained terms aborts the process with a stack overflow instead of returning an error. This adds the same MAX_RECURSION_DEPTH guard to compile_expr (now Compiler::compile_expr, wrapping the renamed compile_expr_inner), returning the same "The expression is too complex" error the parser already uses for right-nested forms. Making compile_expr fallible means propagating TeraResult through everything that calls it, directly or transitively: compile_kwargs, compile_map_entries, compile_block, compile_node, and Compiler::compile, plus their two call sites in Template::new. Second bug found and fixed while verifying this: once the depth guard returns early, the still-mostly-intact remainder of the expression tree it was handed needs to be dropped -- and Expression's default, compiler-derived Drop is itself recursive over that same boxed tree, so a sufficiently large chain (tested: ~55,000+ terms) traded a stack overflow during *compilation* for one during *drop*, on the very error path meant to prevent the crash. Expression::drop_iteratively walks the tree with an explicit heap-allocated stack instead of the call stack, and compile_expr's guard now calls it explicitly on the rejected expression instead of letting it drop implicitly. Verified directly: templates up to 5,000,000 chained terms (about 20MB of source) now return a clean error with no crash, versus overflowing before this second fix at around 55-60k terms. Tested: `cargo test` and `cargo test --all-features` both pass (139 tests across all_features), `cargo fmt --all --check` and `cargo clippy --all-targets --all-features -- -D warnings` both clean. Verified against the exact reproduction from Keats#1035 plus a range of chain lengths from 100 to 5,000,000 terms to find and confirm both the original threshold and the drop-related one; happy to share the verification script if useful for review.
Owner
|
Thanks but I don't think this is the correct approach. I've pushed #1040 instead |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1035. The parser already bounds its own recursion via MAX_RECURSION_DEPTH, but that guard only fires for right-nested forms built through inner_parse_expression (parentheses, unary operators, **). Left-associative chains --
1 + 1 + ...,a | f | f | ...,a.b.c...-- are built iteratively by the Pratt loop and never increment recursion_depth, so they sail past the parser untouched. Compiler::compile_expr then walks that same left-leaning AST recursively with no guard of its own, so a template with a few thousand chained terms aborts the process with a stack overflow instead of returning an error.This adds the same MAX_RECURSION_DEPTH guard to compile_expr (now Compiler::compile_expr, wrapping the renamed compile_expr_inner), returning the same "The expression is too complex" error the parser already uses for right-nested forms. Making compile_expr fallible means propagating TeraResult through everything that calls it, directly or transitively: compile_kwargs, compile_map_entries, compile_block, compile_node, and Compiler::compile, plus their two call sites in Template::new.
Second bug found and fixed while verifying this: once the depth guard returns early, the still-mostly-intact remainder of the expression tree it was handed needs to be dropped -- and Expression's default, compiler-derived Drop is itself recursive over that same boxed tree, so a sufficiently large chain (tested: ~55,000+ terms) traded a stack overflow during compilation for one during drop, on the very error path meant to prevent the crash. Expression::drop_iteratively walks the tree with an explicit heap-allocated stack instead of the call stack, and compile_expr's guard now calls it explicitly on the rejected expression instead of letting it drop implicitly. Verified directly: templates up to 5,000,000 chained terms (about 20MB of source) now return a clean error with no crash, versus overflowing before this second fix at around 55-60k terms.
Tested:
cargo testandcargo test --all-featuresboth pass (139 tests across all_features),cargo fmt --all --checkandcargo clippy --all-targets --all-features -- -D warningsboth clean. Verified against the exact reproduction from #1035 plus a range of chain lengths from 100 to 5,000,000 terms to find and confirm both the original threshold and the drop-related one.