comment to release

refactor(ddl): destructure Column for clarity + future-proofing in co…

…mment-only check

Address review feedback from @callicles:
- Replace clone-and-normalise trick with exhaustive destructuring.
  Adding a new Column field is now a compile error rather than silently
  joining the "no drop" set.
- Lead the doc with the rule (drop required unless only `comment` differs),
  demote ClickHouse rationale below.

No behaviour change; all 50 ddl_ordering tests still pass.

fix(ddl): skip dependent index/projection rebuild for comment-only co…

…lumn changes

ModifyTableColumn previously dropped and re-added every dependent skip-index
and projection on any ColumnChange::Updated, regardless of which field
diverged. ClickHouse only requires that drop when the modification changes
on-disk layout or evaluated values (data_type, default, codec, materialized);
a pure comment change is accepted in place. The over-conservative drop forced
ClickHouse to rebuild skip-indexes lazily on merges, degrading data-skipping
on existing parts -- hours of slower time-windowed reads for a metadata-only
migration.

Gate drop_column_dependents / readd_column_dependents on a new
column_modify_requires_dependent_drop helper that compares before/after with
the comment field normalised. Any other field difference (including
annotations) keeps the existing PR 514-labs#3813 drop path, so new Column fields
default to the safe behaviour.

Adds regression tests for comment-only changes against both minmax indexes
and projections; existing PR 514-labs#3813 type-change tests still cover the drop
path.

feat(dictionaries): add structural mismatch detection to check_realit…

…y() (514-labs#4059)

## Summary

- Implements ENG-2765 step 3: structurally compare dictionaries present
in both the infra map and ClickHouse via `SHOW CREATE DICTIONARY`
- Mismatched dicts are removed from the reconciled map so the subsequent
diff generates an `Added` change, resolved by `CREATE OR REPLACE
DICTIONARY` on next apply
- Switched `execute_create_dictionary` from `IF NOT EXISTS` to `OR
REPLACE` to overwrite drifted CH definitions without a manual drop

## Changes

- **`OlapOperations` trait / ClickHouse impl**: new
`show_create_dictionary(db, name)` method
- **`dicts_ddl_equivalent()`**: DDL body comparator that sorts
post-column clauses (PRIMARY KEY, SOURCE, LAYOUT, LIFETIME) to tolerate
ordering differences between our generator and `SHOW CREATE DICTIONARY`
output
- **`check_reality()`**: replaces the `Vec::new()` placeholder with a
full mismatch-detection loop
- **`reconcile_with_reality()`**: removes structurally mismatched dicts
from the reconciled map

## Test plan

- [ ] `cargo test test_dict` — 19 dictionary tests pass (4
DDL-equivalence, 3 detection, 12 existing)
- [ ] `cargo test test_reconcile` — 11 reconciliation tests pass (2 new
dict tests)
- [ ] `cargo clippy --all-targets -- -D warnings` — clean

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Adds new dictionary DDL introspection/comparison logic and changes
dictionary creation to `CREATE OR REPLACE`, which can overwrite existing
definitions; parsing edge-cases or false positives could trigger
unexpected replaces.
> 
> **Overview**
> Adds **structural mismatch detection for ClickHouse dictionaries**
during `check_reality()` by fetching `SHOW CREATE DICTIONARY` and
comparing it against infra-map generated DDL (with whitespace
normalization, clause reordering tolerance, LIFETIME short/long-form
normalization, and ClickHouse Cloud `[HIDDEN]` secret substitution).
> 
> Updates reconciliation so **mismatched dictionaries are removed from
the reconciled map**, causing the next diff to recreate them, and
switches dictionary creation to use **`CREATE OR REPLACE DICTIONARY`**
for both create/replace paths to overwrite drifted definitions without
manual drops.
> 
> Extends the `OlapOperations` trait + ClickHouse implementation with
`show_create_dictionary()`, and refactors tests by introducing shared
`test_helpers` (mock OLAP client now supports per-dictionary DDL) plus
new unit tests covering DDL equivalence and mismatch/reconcile behavior.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
00010bf. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

refactor: extract QuoteDepthState to eliminate duplicate DDL scanner …

…logic

The quote-and-depth-aware character scanner was implemented identically in
two places: the column block walker in `extract_body` and the
`find_top_level_keyword` inner function. Extract a `QuoteDepthState` struct
with a single `advance(ch) -> bool` method that owns all five state
variables (depth, in_single/double/backtick, skip_next_single). Both call
sites are reduced to a loop over char_indices that delegates state management
to `QuoteDepthState::advance`, eliminating the risk of fixes landing in one
copy but not the other.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: handle backslash-escaped quotes in column block and keyword scan…

…ners

The single-quote state handlers in the column block walker and
`find_top_level_keyword` would exit on any `'` character, including
backslash-escaped ones (`\'`). If `)` appeared after a false quote-close
inside a SOURCE credential, depth tracking became corrupted, causing
incorrect clause boundary detection. Add a `skip_next_single` flag
(matching the approach used in `find_closing_quote`) so `\'` is treated
as an embedded quote rather than a string terminator. Add tests for both
scanners covering escaped-quote + `)` sequences.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: handle escaped quotes in fill_hidden_credentials credential extr…

…action

`fill_hidden_credentials` used `find('\'')` to locate the closing quote
of a credential value, which stops prematurely on backslash-escaped quotes
(`\'`) that `escape_clickhouse_string` emits for passwords containing
single quotes. Replace with `find_closing_quote`, which skips `\'` pairs,
so the full credential value is extracted correctly. Add a test to cover
this case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: use saturating_sub and remove redundant trim_end in infra_realit…

…y_checker

Prevent potential debug-mode panic from usize underflow in the column
block depth walker by switching bare `depth -= 1` to
`depth.saturating_sub(1)`, matching the pattern already used in
`find_top_level_keyword`. Also remove the redundant `trim_end()` call
before `split_whitespace()` flagged by Clippy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: correct credential rotation comment for ClickHouse Cloud

The previous comment claimed credential rotation is always detected as a
mismatch. This is only true when display_secrets_in_show_and_select is ON
(self-hosted default). On ClickHouse Cloud (OFF by default), credentials
appear as [HIDDEN] and fill_hidden_credentials() substitutes them from
desired — rotation cannot be detected in that configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: substitute [HIDDEN] credentials before DDL comparison

ClickHouse Cloud's display_secrets_in_show_and_select=0 (the default)
replaces credential values with [HIDDEN] in SHOW CREATE DICTIONARY output.
Without this fix, every dictionary with SOURCE credentials would be flagged
as structurally mismatched on every reconcile cycle.

fill_hidden_credentials() replaces each '[HIDDEN]' in actual with the
corresponding value from desired, keyed by the keyword immediately preceding
the placeholder (e.g. PASSWORD '[HIDDEN]' → PASSWORD 'real_pass'). Structural
fields (TABLE, HOST, PORT) are still compared normally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>