Add a regression test for NOT_FOUND_COLUMN_IN_BLOCK with LEFT JOIN and a constant predicate#111608
Conversation
…d a constant predicate A constant-true predicate (AND 1) combined with a LEFT JOIN, ORDER BY and LIMIT used to throw NOT_FOUND_COLUMN_IN_BLOCK: the optimizer looked for the whole merged filter expression as a materialized input column instead of evaluating it. Regression from ClickHouse#104268, fixed on master. This test locks the fix in, pinning the three default-on optimizations that must all be enabled to trigger the defect. Closes: ClickHouse#111452 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-PR validation gate (click to expand)
Session id: cron:clickhouse-author-slot-3:20260723-111600 |
Internal second-model review — adjudication log (click to expand)Pre-publication review by an independent model (engine: codex; 3 findings across one full pass + one delta recheck; bounded: 1 full pass + 1 fix round + 1 delta recheck).
Severity: ❌ blocker / Session id: cron:clickhouse-author-slot-3:20260723-111600 |
|
cc @PedroTadim — as requested. Test-only PR adding the regression test for #111452 (the NOT_FOUND_COLUMN_IN_BLOCK is already fixed on master); it pins the optimizations that reach the lazy-materialization-through-join path and asserts via EXPLAIN that the path is exercised. |
|
Workflow [PR], commit [1531dcd] AI ReviewSummaryThis PR adds a stateless regression test for issue Findings
Final VerdictChanges requested. |
…c marker The old liveness oracle only asserted that a JoinLazyColumnsStep is present in the plan. That step is the generic lazy-materialization stitch step: it is built whenever optimizeLazyMaterialization (or optimizeLazyFinal) splits a read into a main and a lazy half, including for single-table plans (e.g. it appears in 02354_vector_search_queries.reference). So the oracle could stay green even if the LEFT JOIN / top-k-through-join rewrite this test targets regressed, and the test would no longer prove the code path described in its comment. Replace it with a join-specific oracle that asserts three properties at once: the SQL Join step is present; top-k-through-join pushed a Sorting onto the preserved (left) input, i.e. a Sorting appears below the Join in the plan; and lazy materialization split the read (JoinLazyColumnsStep). Kept as a single EXPLAIN over one CTE, in the SELECT ... FROM (EXPLAIN ...) form. Verified on a debug build: the oracle returns 1/1/1 on fixed master, and goes red (0 in the relevant column) when top-k-through-join or lazy materialization is disabled. 30/30 runner passes with randomization enabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Head branch was pushed to by a user without write access
| -- in the lazy half of the plan, referencing a column no input block provides. Regression from #104268, | ||
| -- fixed on master. Issue #111452. | ||
| -- | ||
| -- All optimizations that must be enabled to reach the fixed code path are pinned so the runner cannot |
There was a problem hiding this comment.
tests/clickhouse-test still randomizes query_plan_join_swap_table between auto and false, so this test is not actually pinning every plan-shaping knob it depends on. The sibling regression test 04230_top_k_through_join_join_swap_gate.sql documents why that matters: once swap is allowed, optimizeJoinLegacy can rewrite the logical LEFT JOIN into RIGHT JOIN. The current oracle only checks for Join, Sorting somewhere below it, and JoinLazyColumnsStep, so it can stay green on that swapped path too.
Please add SET query_plan_join_swap_table = false; here so the regression stays on the LEFT JOIN lazy-materialization path from #111452 instead of whichever join orientation the randomizer leaves behind.
Closes: #111452
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
...
Description
A constant-true predicate (
AND 1) combined with aLEFT JOIN,ORDER BYandLIMITused to throwNOT_FOUND_COLUMN_IN_BLOCK. With filter push down, merge expressions, top-k-through-join and lazy materialization all active, the merged outer filter was left as a dangling input pass-through in the lazy half of the plan, referencing a column no input block provides. Regression from #104268 (shipped in 26.5/26.6), already fixed on master.Test-only PR adding a stateless regression test that locks the fix in. The bug lives in the lazy-materialization-through-join path, so the test pins every optimization that must be enabled to reach that path (
query_plan_filter_push_down,query_plan_merge_expressions,query_plan_top_k_through_join,query_plan_optimize_lazy_materialization, both*_max_limit_*caps set to 0 = unlimited so they never drop below theLIMIT) and includes anEXPLAINliveness oracle. The oracle is join-specific: it asserts the SQLJoinstep is present, that top-k-through-join pushed aSortingonto the preserved (left) input, and that lazy materialization split the read (JoinLazyColumnsStep).JoinLazyColumnsStepalone is the generic lazy-materialization stitch step (it also appears in single-table plans), so asserting only it would let the oracle stay green if the join rewrite regressed; the three-column form goes red when top-k-through-join or lazy materialization is disabled. Taggedno-parallel-replicas(like the other lazy-materialization tests).Verified: the query throws
Code: 10 NOT_FOUND_COLUMN_IN_BLOCKon a pre-fix master build and returns the expected single row on current master; the test passes under CI setting randomization.