Skip to content

fix(api): one script-guard definition for every search entry point - #124

Merged
xerj-org merged 2 commits into
mainfrom
fix/script-guard-symmetry
Aug 2, 2026
Merged

fix(api): one script-guard definition for every search entry point#124
xerj-org merged 2 commits into
mainfrom
fix/script-guard-symmetry

Conversation

@xerj-org

@xerj-org xerj-org commented Aug 1, 2026

Copy link
Copy Markdown
Owner

PR #115 closed a real bypass — /_msearch skipped the request-time script guard entirely — but it closed it with a different check than _search uses, and that asymmetry is a live defect on main.

build_search_request runs the guard on six specific fields (query, rescore, sort, script_fields, runtime_mappings, aggs). The _msearch, _search/template and _msearch/template guards ran it on the whole request body. So _msearch became stricter than _search: an over-limit script under any unrelated key returned 400 where the identical search returned 200.

Measured with a 45-case router probe run against both commits, fingerprinting status, hits.total, hit ids, agg keys and error type at all four entry points: 12 bodies on main where _search answers 200 and the three raw paths answer 400.

The fix

One definition. enum GuardedField names the six fields once; in_raw_body projects it onto a raw JSON body and in_search_body onto the typed EsSearchBody. Both matches are exhaustive, so a field cannot be guarded on one path and skipped on the other.

The aggs/aggregations pair is a single value resolved the way the executor resolves it. This matters because an earlier attempt at this listed both spellings as independent keys, which produced a new asymmetry: a body with an ordinary aggs and an over-limit script under aggregations got 200 from _search and 400 from _msearch, even though neither endpoint executes that value — parse_request also does obj.get("aggs").or_else(|| obj.get("aggregations")).

The precedence rule is key-name, not document order. serde_json preserves order workspace-wide, so this is observable: {"aggregations": <over-limit>, "aggs": <ordinary>} answers 200 everywhere and executes the aggs value despite it being written second.

Behaviour change

On _msearch, _search/template and _msearch/template, an over-limit script under any key other than the six guarded ones goes 400 → 200. Beyond highlight, suggest, docvalue_fields and collapse.inner_hits, that was measured on post_filter, _source, fields, ext, stored_fields, pit, indices_boost and a top-level inner_hits object.

None of these is a script sink — parse_request reads only query, sort, aggs and script_fields among them — so nothing executes and the direction is the ES-compatible one. _search itself is unchanged: every status in the probe table is identical before and after on the _search column.

Two bodies now disagree between the typed and raw paths where main happened to agree (a knn.filter carrying a script, and {"aggs": null, "aggregations": <over-limit>}). Main's agreement was accidental — the whole-body walk 400'd everything — and no script executes on the raw paths in either case.

The #115 bypass stays closed

Re-verified independently with a 240,531-byte three-hostile-item _msearch body: still rejected per-item.

A gap verification found, now closed

Adding a GuardedField variant was already a compile error. Removing one was not, and that is the direction that reopens #115. Every test over the guarded set iterates ALL, so they prove the guard covers whatever ALL holds, not six specific fields — measured, deleting Self::Rescore left all 129 tests green while an 80,001-byte rescore script went 400 → 200 on every entry point.

const _: () = assert!(GuardedField::ALL.len() == 6) makes that a build failure. Verified by mutation: evaluation panicked: assertion failed: GuardedField::ALL.len() == 6.

Unrelated flake worth knowing about

es_compat::scripted_update_publication_tests::concurrent_scripted_update_by_query_preserves_every_increment fails roughly 1 in 60 full-suite runs on both this branch and main, and 0 in 60 targeted runs, so it only trips under suite concurrency. Introduced by dc24130; this diff does not touch it. Flagging so it is not misattributed to whichever PR is in flight when it eventually turns CI red.

@xerj-org xerj-org closed this Aug 1, 2026
@xerj-org xerj-org reopened this Aug 1, 2026
PR #115 closed a real bypass — `_msearch`, `_search/template` and
`_msearch/template` skip `build_search_request`, so the request-time script
guard never ran there — but it closed it with a walk over the WHOLE
sub-request body, while `_search` guards six specific fields (`query`,
`rescore`, `sort`, `script_fields`, `runtime_mappings`, `aggs`). That made
the multi-search API stricter than `_search`.

Measured on main through the real router, identical body to both endpoints:

  * an over-limit `script` under `highlight`, `suggest`, `docvalue_fields`
    or `collapse.inner_hits`: `_search` 200, `_msearch` 400
  * `aggs` ordinary + `aggregations` over-limit: `_search` 200, `_msearch`
    400 — and neither endpoint even executes that value, since the typed and
    the raw resolver both take the FIRST spelling present
    (`parse_request`: `obj.get("aggs").or_else(|| obj.get("aggregations"))`)

`GuardedField` is now the single definition of what the guard walks. Both
resolvers — `in_search_body` for the typed `EsSearchBody` (`_search`, scroll,
async search) and `in_raw_body` for the raw bodies handed straight to
`parse_request` — are exhaustive matches over it, so a field cannot be
guarded on one path and skipped on another. The `aggs`/`aggregations` pair is
one value in both, resolved the way the executor resolves it, not two keys
checked independently.

Not a loosening of the guard: an over-limit script in any of the six fields
is still rejected on all four entry points, and a hostile `_msearch` item is
still rejected on its own while its siblings run. Top-level `knn` stays
outside the set with a tripwire test: `_search` folds it into the guarded
`query`, and `_msearch` does not execute it at all (a `knn.filter` matching
nothing returns 1 hit there vs 0 on `_search`).

The tests fire one body at `_search`, `_msearch`, `_search/template` and
`_msearch/template` through the real router and require the verdicts to
match; the field table is driven off `GuardedField::ALL`, so a new variant
cannot ship without a fixture every entry point is checked against.
Reverting only the `_msearch` call site to the whole-body walk fails three of
them.
…rule exactly

Follow-up to the guard-unification commit, closing three things verification
found. No change to what any request answers.

A COMPILE-TIME WIDTH ASSERTION. Adding a `GuardedField` variant was already a
compile error in both exhaustive matches. Removing one was not, and that is the
dangerous direction: every test over the guarded set iterates `ALL`, so they
prove the guard covers whatever `ALL` currently holds rather than six specific
fields. Measured — deleting `Self::Rescore` leaves the entire suite green while
an 80,001-byte `rescore` script goes 400 -> 200 on all four entry points, which
is exactly the bypass #111 closed. `const _: () = assert!(ALL.len() == 6)` now
makes that a build failure; verified by mutation, it fails with
"evaluation panicked: assertion failed: GuardedField::ALL.len() == 6".

THE AGGS PRECEDENCE RULE WAS AMBIGUOUS. Three places said "first spelling
present wins", which reads as document order. `serde_json` preserves key order
workspace-wide, so document order is observable and it is NOT the rule: with
`{"aggregations": <over-limit>, "aggs": <ordinary>}` all four endpoints answer
200 and execute the `aggs` value even though it is written second. The rule is
key-name precedence — `aggs` wins whenever both are present — and the docs now
say that.

THE BLAST RADIUS WAS UNDER-NAMED. The behaviour-delta list named four unguarded
fields plus the aggs pair. The actual change is the whole class: on `_msearch`,
`_search/template` and `_msearch/template`, an over-limit script under ANY key
other than the six guarded ones goes 400 -> 200. Measured on eight further
top-level keys beyond those declared: post_filter, _source, fields, ext,
stored_fields, pit, indices_boost, and a top-level inner_hits object. None is a
script sink, so nothing executes and the direction is the ES-compatible one,
but a reviewer reading the old list would have expected four fields.

Two bodies now disagree between the typed and raw paths where main happened to
agree: a `knn.filter` carrying a script, and `{"aggs": null, "aggregations":
<over-limit>}`. Main's agreement was accidental — the whole-body walk 400'd
everything — and no script executes on the raw paths in either case.
@xerj-org
xerj-org force-pushed the fix/script-guard-symmetry branch from 7441d9f to bce7713 Compare August 1, 2026 23:34
@xerj-org
xerj-org merged commit 755c612 into main Aug 2, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant