fix(api): one script-guard definition for every search entry point - #124
Merged
Conversation
PR #115 closed a real bypass — `_msearch`, `_search/template` and `_msearch/template` skip `build_search_request`, so the request-time script guard never ran there — but it closed it with a walk over the WHOLE sub-request body, while `_search` guards six specific fields (`query`, `rescore`, `sort`, `script_fields`, `runtime_mappings`, `aggs`). That made the multi-search API stricter than `_search`. Measured on main through the real router, identical body to both endpoints: * an over-limit `script` under `highlight`, `suggest`, `docvalue_fields` or `collapse.inner_hits`: `_search` 200, `_msearch` 400 * `aggs` ordinary + `aggregations` over-limit: `_search` 200, `_msearch` 400 — and neither endpoint even executes that value, since the typed and the raw resolver both take the FIRST spelling present (`parse_request`: `obj.get("aggs").or_else(|| obj.get("aggregations"))`) `GuardedField` is now the single definition of what the guard walks. Both resolvers — `in_search_body` for the typed `EsSearchBody` (`_search`, scroll, async search) and `in_raw_body` for the raw bodies handed straight to `parse_request` — are exhaustive matches over it, so a field cannot be guarded on one path and skipped on another. The `aggs`/`aggregations` pair is one value in both, resolved the way the executor resolves it, not two keys checked independently. Not a loosening of the guard: an over-limit script in any of the six fields is still rejected on all four entry points, and a hostile `_msearch` item is still rejected on its own while its siblings run. Top-level `knn` stays outside the set with a tripwire test: `_search` folds it into the guarded `query`, and `_msearch` does not execute it at all (a `knn.filter` matching nothing returns 1 hit there vs 0 on `_search`). The tests fire one body at `_search`, `_msearch`, `_search/template` and `_msearch/template` through the real router and require the verdicts to match; the field table is driven off `GuardedField::ALL`, so a new variant cannot ship without a fixture every entry point is checked against. Reverting only the `_msearch` call site to the whole-body walk fails three of them.
…rule exactly Follow-up to the guard-unification commit, closing three things verification found. No change to what any request answers. A COMPILE-TIME WIDTH ASSERTION. Adding a `GuardedField` variant was already a compile error in both exhaustive matches. Removing one was not, and that is the dangerous direction: every test over the guarded set iterates `ALL`, so they prove the guard covers whatever `ALL` currently holds rather than six specific fields. Measured — deleting `Self::Rescore` leaves the entire suite green while an 80,001-byte `rescore` script goes 400 -> 200 on all four entry points, which is exactly the bypass #111 closed. `const _: () = assert!(ALL.len() == 6)` now makes that a build failure; verified by mutation, it fails with "evaluation panicked: assertion failed: GuardedField::ALL.len() == 6". THE AGGS PRECEDENCE RULE WAS AMBIGUOUS. Three places said "first spelling present wins", which reads as document order. `serde_json` preserves key order workspace-wide, so document order is observable and it is NOT the rule: with `{"aggregations": <over-limit>, "aggs": <ordinary>}` all four endpoints answer 200 and execute the `aggs` value even though it is written second. The rule is key-name precedence — `aggs` wins whenever both are present — and the docs now say that. THE BLAST RADIUS WAS UNDER-NAMED. The behaviour-delta list named four unguarded fields plus the aggs pair. The actual change is the whole class: on `_msearch`, `_search/template` and `_msearch/template`, an over-limit script under ANY key other than the six guarded ones goes 400 -> 200. Measured on eight further top-level keys beyond those declared: post_filter, _source, fields, ext, stored_fields, pit, indices_boost, and a top-level inner_hits object. None is a script sink, so nothing executes and the direction is the ES-compatible one, but a reviewer reading the old list would have expected four fields. Two bodies now disagree between the typed and raw paths where main happened to agree: a `knn.filter` carrying a script, and `{"aggs": null, "aggregations": <over-limit>}`. Main's agreement was accidental — the whole-body walk 400'd everything — and no script executes on the raw paths in either case.
xerj-org
force-pushed
the
fix/script-guard-symmetry
branch
from
August 1, 2026 23:34
7441d9f to
bce7713
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #115 closed a real bypass —
/_msearchskipped the request-time script guard entirely — but it closed it with a different check than_searchuses, and that asymmetry is a live defect onmain.build_search_requestruns the guard on six specific fields (query,rescore,sort,script_fields,runtime_mappings,aggs). The_msearch,_search/templateand_msearch/templateguards ran it on the whole request body. So_msearchbecame stricter than_search: an over-limit script under any unrelated key returned 400 where the identical search returned 200.Measured with a 45-case router probe run against both commits, fingerprinting status,
hits.total, hit ids, agg keys and error type at all four entry points: 12 bodies onmainwhere_searchanswers 200 and the three raw paths answer 400.The fix
One definition.
enum GuardedFieldnames the six fields once;in_raw_bodyprojects it onto a raw JSON body andin_search_bodyonto the typedEsSearchBody. Both matches are exhaustive, so a field cannot be guarded on one path and skipped on the other.The
aggs/aggregationspair is a single value resolved the way the executor resolves it. This matters because an earlier attempt at this listed both spellings as independent keys, which produced a new asymmetry: a body with an ordinaryaggsand an over-limit script underaggregationsgot 200 from_searchand 400 from_msearch, even though neither endpoint executes that value —parse_requestalso doesobj.get("aggs").or_else(|| obj.get("aggregations")).The precedence rule is key-name, not document order.
serde_jsonpreserves order workspace-wide, so this is observable:{"aggregations": <over-limit>, "aggs": <ordinary>}answers 200 everywhere and executes theaggsvalue despite it being written second.Behaviour change
On
_msearch,_search/templateand_msearch/template, an over-limit script under any key other than the six guarded ones goes 400 → 200. Beyondhighlight,suggest,docvalue_fieldsandcollapse.inner_hits, that was measured onpost_filter,_source,fields,ext,stored_fields,pit,indices_boostand a top-levelinner_hitsobject.None of these is a script sink —
parse_requestreads onlyquery,sort,aggsandscript_fieldsamong them — so nothing executes and the direction is the ES-compatible one._searchitself is unchanged: every status in the probe table is identical before and after on the_searchcolumn.Two bodies now disagree between the typed and raw paths where
mainhappened to agree (aknn.filtercarrying a script, and{"aggs": null, "aggregations": <over-limit>}). Main's agreement was accidental — the whole-body walk 400'd everything — and no script executes on the raw paths in either case.The #115 bypass stays closed
Re-verified independently with a 240,531-byte three-hostile-item
_msearchbody: still rejected per-item.A gap verification found, now closed
Adding a
GuardedFieldvariant was already a compile error. Removing one was not, and that is the direction that reopens #115. Every test over the guarded set iteratesALL, so they prove the guard covers whateverALLholds, not six specific fields — measured, deletingSelf::Rescoreleft all 129 tests green while an 80,001-byterescorescript went 400 → 200 on every entry point.const _: () = assert!(GuardedField::ALL.len() == 6)makes that a build failure. Verified by mutation:evaluation panicked: assertion failed: GuardedField::ALL.len() == 6.Unrelated flake worth knowing about
es_compat::scripted_update_publication_tests::concurrent_scripted_update_by_query_preserves_every_incrementfails roughly 1 in 60 full-suite runs on both this branch andmain, and 0 in 60 targeted runs, so it only trips under suite concurrency. Introduced bydc24130; this diff does not touch it. Flagging so it is not misattributed to whichever PR is in flight when it eventually turns CI red.