fix(painless): surface script resource-limit trips instead of returning a wrong score - #116

… and pivot transforms (#126)

* fix(api): surface script resource-limit trips on _rank_eval, _explain and pivot transforms (#123)

PR #116 split Painless failures into "cannot evaluate" (degrade quietly, the
ES-compat contract) and "refused because a resource limit tripped" (must
surface), and wired the second class through ten request surfaces via the
task-local fault sink and `SearchResult::script_failure`. Three surfaces still
read that field's absence as success. This uses the same mechanism on all
three — no new machinery, only new call sites, plus the enum needed to carry a
fault out of the transform runner in the existing `script_exception` shape.

Captured verbatim from the pre-fix tree, all HTTP 200:

  _rank_eval, script_score past MAX_CALL_DEPTH:
    {"metric_score":0.5,"details":{"q1":{"metric_score":0.5, ... }},"failures":{}}
  _rank_eval, terms_set minimum_should_match_script past the same limit:
    {"metric_score":0.0,"details":{"q1":{"metric_score":0.0,"hits":[]}},"failures":{}}
  _explain, same query:
    {"matched":false, ... "description":"document [1] does not match the query"}
  POST /_transform/t1/_start over a source.query with that script:
    {"acknowledged":true}

_rank_eval is the sharpest case in the set and the reason the issue was filed
against it specifically: it exists to MEASURE relevance quality, so a degraded
score does not merely return a bad answer, it corrupts the number a caller
would use to notice bad answers. The 0.5 above is a precision figure computed
over a ranking whose scores no script produced.

_explain answers a yes/no question, and a fail-closed script answers it "no" —
a confident negative indistinguishable from a document that genuinely does not
match. The pivot transform is worse than an under-delete: it WRITES the
truncated summary into dest, where later queries read the wrong numbers as
fact. Both halves of a pivot can carry a user script (source.query and
pivot.aggregations), and the check sits before the writes. The runner is not
transactional, so buckets from earlier pages that completed cleanly stay
written, as with any mid-run failure; that is stated in the comment rather
than papered over.

run_rollup_job is deliberately left unchecked, and the reason is structural
rather than a judgement call: its request body is built entirely from the job
config's groups and metrics — composite sources over date_histogram / terms /
histogram with a field, plus named metric aggs over a field. There is no query
key and nowhere to put a script, so the search it issues carries none. The
other Index::search callers reached in this audit are unchanged for the same
kind of reason: terms_enum, execute_enrich_policy, clone_index_to and
open_index's timestamp probe all issue a literal match_all; eql_search and
sql_query parse their own query languages, neither of which admits Painless;
the sub-searches inside the _search handler are already covered, because
Index::search re-raises a consumed fault into the enclosing capture scope.

Second defect in the same issue. _delete_by_query and _update_by_query refused
correctly but shipped the refusal as 200 OK carrying {"error": ..., "status":
400}, so a client branching on the HTTP status saw a completed run and then
found no deleted / updated key. Both runners return a plain Value because the
detached wait_for_completion=false path stores it as a _tasks/{id} result,
which has no HTTP status of its own; the synchronous path handed that value to
Json(..), which is always 200. by_query_response now takes the status from the
body, which both error builders (script_limit_error_value and
ApiError::into_value) already populate correctly. A successful run has no
status key and stays 200 — pinned by its own test, since that mapping is now
load-bearing in both directions.

Regression coverage is at the wire in
tests/script_limits_remaining_surfaces_http.rs. Verified by reverting only
es_compat.rs and keeping the tests: six of the eight fail on the unfixed tree
with exactly the bodies quoted above, and pass with the fix. The two that pass
in both states are the guard rails — an out-of-subset script must keep scoring
neutrally on a 200 through _rank_eval and _explain, and a script-free
_delete_by_query must stay a 200 with its deleted count. Without them this
could be "fixed" by turning every unparseable script into a 400, which would
pass every loudness assertion and break the compat surface.

cargo test -p xerj-api: 122 lib tests, 6 script_limits_http, 8 new — all pass.
clippy --lib --tests -D warnings and fmt --check both clean.

Closes #123

* fix(api): report script-limit trips on _rank_eval and _explain without over-refusing

Follow-up to the first cut, which surfaced the faults but refused too much.

`_rank_eval` failed the WHOLE batch. A body with ten requests where one carries
a limit-tripping script 400'd all ten. ES provides `failures` for exactly this,
keyed by request id, and the response already carried an empty one. The fault
is now recorded there, the faulted request contributes nothing to
`metric_score` and does not appear in `details`, and its siblings are still
measured. That keeps the property the fix exists for — a `metric_score` is
never published over scores a refused script produced — without discarding
good measurements.

`_explain` returned 400 for a correct answer. Measured: a `function_score`
`script_score` past MAX_CALL_DEPTH used to return 200 `{"matched": true, ...}`,
which is right, because `matched` comes from the ids-filtered hit list and
scoring does not decide it. Refusing turned a correct response into an error
for precisely the person debugging the script.

`_explain` is a diagnostic endpoint, so the fault is now REPORTED rather than
refused: the verdict is still answered, and an extra `explanation.details` node
names the limit that tripped. Where the script IS load-bearing for matching,
`matched: false` would be a confident negative, which is why it cannot simply
be published silently either — disclosing it satisfies both.

The three tests that pinned the old behaviour are rewritten to pin this one,
and a fourth is added asserting a failure stays scoped to the offending request
while a healthy sibling is still measured. 9 passed, 0 failed.

…ape (#122)

Every limit the interpreter had bounds a structural property of a script: how
deep it nests (`MAX_PARSE_DEPTH`, `MAX_EVAL_DEPTH`), how many closure calls it
makes (`MAX_CALL_DEPTH`, `MAX_CALL_COUNT`), how many bytes of source it is
(`MAX_SCRIPT_LEN`). None of them bounds work, and shape and work are not the
same thing.

Measured on this branch's base in a release build, one core, `taskset`-pinned.
Every script below is flat, closure-free, nests nothing, calls nothing, and is
well inside the 64 KiB source cap — so every limit named above passes it:

    | script                                            | source   | per document |
    |---------------------------------------------------|---------:|-------------:|
    | 3,000 x `params.blob;`, params of 50,000 nodes     | 38.1 KiB |       9.03 s |
    | 3,000 x `params.blob;`, params of 200,000 nodes    | 38.1 KiB |      37.35 s |
    | 3,000 x `params['_source'];`, doc of 100,000 nodes | 55.7 KiB |      17.17 s |
    | 2,000 x `doc['rank'].value`, doc of 20,000 nodes   | 52.8 KiB |       1.13 s |

All four are O(size) behind an expression that looks O(1): `params.x` converts
a caller-supplied subtree into interpreter values, `params['_source']` clones
the document, and `get_doc_value` clones the whole document before walking into
it. None of that cost is visible in the statement count, which is why a step
counter alone would not have caught any of it. The doc-scan cooperative timeout
poll cannot help either: it only decides how often a document *boundary* is
checked, so a request `timeout` cannot interrupt a single expensive document at
all.

## The budget

A work counter inside `eval_painless`, charged for two things:

  * one unit per interpreter step (`eval_expr` / `exec_stmt` entry), and
  * one unit per node, plus one per 64 bytes, of every value a step produces.

The second half is the part that matters, and it has to be charged on *every*
path that produces a value — including the fast paths in `eval_access_chain`
that resolve `doc`, `params` and `Math` roots without a preceding value. Those
are exactly the paths the table above exploits.

`params` member reads are charged *before* the value is materialised
(`charged_params_read`), so the budget bounds how many such conversions an
evaluation may perform rather than performing one and asking afterwards.
`params['_source']` pays the document's weight through the same
`charge_document_read` that `doc['x'].value` uses.

`MAX_SCRIPT_OPS` = 5,000,000 is calibrated, not guessed: the largest *benign*
script the source cap admits (4,300 statements of `t = t + N.5;`, 59,773 bytes)
costs a measured 34,407 work units, so the ceiling keeps a 145x margin over the
worst legitimate script that can be submitted.
`painless::tests::the_budget_leaves_the_largest_benign_script_a_wide_margin`
pins that number so a change to the charging scheme reports how far the margin
moved.

## Integration with the request deadline

Every 1,024 work units the counter also reads the clock. The slice an
evaluation gets is the enclosing request's *remaining* time, clamped to
[100 ms, 500 ms], published by `with_script_deadline` from `Index::search` —
the same task-local mechanism as the fault sink.

That deadline is resolved in `PainlessCtx::new`, before any work is charged,
and this is load-bearing rather than tidy. Deriving it inside the first
`clock_check` costs one sampling window to *establish* the deadline and a
second one to be able to fail it, and a window is bounded in work units, not in
time — so "one window" is however long the largest single charge takes. It cost
one clock read per evaluation to fix; the measured overshoot against a 100 ms
slice is now 0.48 ms on a 10,000-node document, 1.17 ms at 40,000 and 7.11 ms
at 160,000, growing with the cost of one charged step exactly as that argument
says it should.

The 100 ms floor is what keeps the trip meaningful. The request deadline is
routinely already past when a script runs, because the doc scan only checks it
every 4,096 documents, so cutting evaluations off the moment it passes would
turn every ordinary `timeout` into a `script_exception` 400. 100 ms is ~360x
the measured 278 µs cost of the largest benign script that can be submitted.

Both trips classify as resource limits (`is_resource_limit_error`), so they
reach the caller as a 400 through the sink #116 built rather than degrading to
a plausible wrong score. Once any evaluation in a request has tripped, later
ones short-circuit on the recorded fault, which makes ONE budget the bound for
a whole request rather than one per document.

## Result

    | script                                            | before  | after  |
    |---------------------------------------------------|--------:|-------:|
    | 3,000 x `params.blob;`, params of 50,000 nodes     |  9.03 s |  97 ms |
    | 3,000 x `params.blob;`, params of 200,000 nodes    | 37.35 s | 109 ms |
    | 3,000 x `params['_source'];`, doc of 100,000 nodes | 17.17 s | 168 ms |
    | 2,000 x `doc['rank'].value`, doc of 20,000 nodes   |  1.13 s | 155 ms |

Flat in the size of the input, because the trip happens at a fixed work count.
Through `_search`, the first of those is now a 400 `script_exception` naming
the budget.

## Cost to ordinary scripts

Interleaved A/B: base and fixed binaries alternated inside each repetition and
pinned to the same idle core, 15 repetitions, each figure already the best of
40 trials. Reporting the best-of-repetitions, which is the least-interrupted
observation on each side:

    | script                             | base       | fixed      | delta   |
    |------------------------------------|-----------:|-----------:|--------:|
    | 42 B, reads `doc[...]` and `params`|   274.8 ns |   353.8 ns |  +28.7% |
    | 237 B arithmetic                   |  1124.7 ns |  1230.6 ns |   +9.4% |
    | 59,773 B arithmetic, 34,407 units  |   271.1 µs |   298.5 µs |  +10.1% |

These are larger than the numbers the first revision of this change quoted
(+6.9% / +0.46%), and the difference is not the round-2 fixes: measured on the
same run, the first revision alone is +19.5% / +0.9% / +5.3% against the same
base. A probe build carrying only the `eval_access_chain` charge lands within
1% of it on all three shapes, so the security fix itself is not what costs —
the accounting was always the price of the budget.

Of the remainder, the tiny script's +25 ns is the one clock read per evaluation
that anchoring the deadline requires, which is the right order for a vDSO
`clock_gettime`. The sub-5% gaps on the two arithmetic shapes are not
attributable: a probe that does strictly *less* work than the final build
measures slower than it on one shape and faster on the other, which is
build-to-build codegen variation of the same magnitude as the effect.

## Query width: `max_clause_count`

There was no clause limit anywhere, so a query could be arbitrarily *wide* as
well, and width is what multiplies per-document work. `MAX_CLAUSE_COUNT` =
1,024 matches Elasticsearch's documented
`indices.query.bool.max_clause_count` default and returns its
`too_many_clauses` error, so a query this refuses is one the system being
emulated already refused. Clauses are charged before they are parsed and summed
across the whole query, not per `bool`, and every clause-list shape counts
towards the same total: `bool`, `span_or`/`span_near` clauses, `dis_max.queries`
and an array `knn.filter`.

A cap is only worth having if tripping it is *visible*. `parse_span_or` and
`parse_span_near` collected clauses with `filter_map(|v| parse_query(v).ok())`,
discarding any clause the parser refused. That was survivable while every
refusable clause was malformed; with a width cap it becomes a correctness bug,
because a well-formed clause that is merely too wide now fails to parse. A
`span_or` over [an over-wide `bool`, a `span_term`] parsed to a `SpanOr`
containing only the `span_term` — 200 OK for a query nobody asked for — and a
`span_near` whose only clause was over-wide collapsed to `MatchNone`: 200 OK,
zero hits, no error anywhere. A silently different query is worse than an
unbounded one. Every clause error now propagates.

## Verification

  * Reverting only `painless.rs` and keeping the tests:
    `bare_params_reads_are_charged_for_what_they_materialise`,
    `subscripted_params_reads_are_charged_too` and
    `params_source_reads_are_charged_for_the_document_they_clone` all fail,
    each reporting that the script was charged 2,048 work units, and
    `script_cpu_budget_http` fails on the wall clock having to notice what the
    op ceiling should have. The work-count assertions are deliberate: an
    uncharged read is still stopped *eventually*, after ~15 s, so a test that
    only asserts "some resource limit tripped" passes on the broken tree.
  * Reverting only `parser.rs` and keeping `tests/span_clause_refusal.rs`: 6 of
    its 9 tests fail, reporting the dropped clause and the `MatchNone`
    collapse. That file lives outside `parser.rs` precisely so it survives the
    revert.
  * `the_time_budget_trips_on_its_own_inside_an_expired_request` exercises the
    wall-clock half on its own; every other test here trips the deterministic
    op ceiling first.
  * `cargo test --release -p xerj-query -p xerj-engine -p xerj-api` green.
  * `cargo fmt --check` and `cargo clippy --lib --tests -- -D warnings` clean on
    all three crates.

…ape (#122)

Every limit the interpreter had bounds a structural property of a script: how
deep it nests (`MAX_PARSE_DEPTH`, `MAX_EVAL_DEPTH`), how many closure calls it
makes (`MAX_CALL_DEPTH`, `MAX_CALL_COUNT`), how many bytes of source it is
(`MAX_SCRIPT_LEN`). None of them bounds work, and shape and work are not the
same thing.

Measured on this branch's base in a release build, one core, `taskset`-pinned.
Every script below is flat, closure-free, nests nothing, calls nothing, and is
well inside the 64 KiB source cap — so every limit named above passes it:

    | script                                            | source   | per document |
    |---------------------------------------------------|---------:|-------------:|
    | 3,000 x `params.blob;`, params of 50,000 nodes     | 38.1 KiB |       9.03 s |
    | 3,000 x `params.blob;`, params of 200,000 nodes    | 38.1 KiB |      37.35 s |
    | 3,000 x `params['_source'];`, doc of 100,000 nodes | 55.7 KiB |      17.17 s |
    | 2,000 x `doc['rank'].value`, doc of 20,000 nodes   | 52.8 KiB |       1.13 s |

All four are O(size) behind an expression that looks O(1): `params.x` converts
a caller-supplied subtree into interpreter values, `params['_source']` clones
the document, and `get_doc_value` clones the whole document before walking into
it. None of that cost is visible in the statement count, which is why a step
counter alone would not have caught any of it. The doc-scan cooperative timeout
poll cannot help either: it only decides how often a document *boundary* is
checked, so a request `timeout` cannot interrupt a single expensive document at
all.

## The budget

A work counter inside `eval_painless`, charged for two things:

  * one unit per interpreter step (`eval_expr` / `exec_stmt` entry), and
  * one unit per node, plus one per 64 bytes, of every value a step produces.

The second half is the part that matters, and it has to be charged on *every*
path that produces a value — including the fast paths in `eval_access_chain`
that resolve `doc`, `params` and `Math` roots without a preceding value. Those
are exactly the paths the table above exploits.

`params` member reads are charged *before* the value is materialised
(`charged_params_read`), so the budget bounds how many such conversions an
evaluation may perform rather than performing one and asking afterwards.
`params['_source']` pays the document's weight through the same
`charge_document_read` that `doc['x'].value` uses.

`MAX_SCRIPT_OPS` = 5,000,000 is calibrated, not guessed: the largest *benign*
script the source cap admits (4,300 statements of `t = t + N.5;`, 59,773 bytes)
costs a measured 34,407 work units, so the ceiling keeps a 145x margin over the
worst legitimate script that can be submitted.
`painless::tests::the_budget_leaves_the_largest_benign_script_a_wide_margin`
pins that number so a change to the charging scheme reports how far the margin
moved.

## Integration with the request deadline

Every 1,024 work units the counter also reads the clock. The slice an
evaluation gets is the enclosing request's *remaining* time, clamped to
[100 ms, 500 ms], published by `with_script_deadline` from `Index::search` —
the same task-local mechanism as the fault sink.

That deadline is resolved in `PainlessCtx::new`, before any work is charged,
and this is load-bearing rather than tidy. Deriving it inside the first
`clock_check` costs one sampling window to *establish* the deadline and a
second one to be able to fail it, and a window is bounded in work units, not in
time — so "one window" is however long the largest single charge takes. It cost
one clock read per evaluation to fix; the measured overshoot against a 100 ms
slice is now 0.48 ms on a 10,000-node document, 1.17 ms at 40,000 and 7.11 ms
at 160,000, growing with the cost of one charged step exactly as that argument
says it should.

The 100 ms floor is what keeps the trip meaningful. The request deadline is
routinely already past when a script runs, because the doc scan only checks it
every 4,096 documents, so cutting evaluations off the moment it passes would
turn every ordinary `timeout` into a `script_exception` 400. 100 ms is ~360x
the measured 278 µs cost of the largest benign script that can be submitted.

Both trips classify as resource limits (`is_resource_limit_error`), so they
reach the caller as a 400 through the sink #116 built rather than degrading to
a plausible wrong score. Once any evaluation in a request has tripped, later
ones short-circuit on the recorded fault, which makes ONE budget the bound for
a whole request rather than one per document.

## Result

    | script                                            | before  | after  |
    |---------------------------------------------------|--------:|-------:|
    | 3,000 x `params.blob;`, params of 50,000 nodes     |  9.03 s |  97 ms |
    | 3,000 x `params.blob;`, params of 200,000 nodes    | 37.35 s | 109 ms |
    | 3,000 x `params['_source'];`, doc of 100,000 nodes | 17.17 s | 168 ms |
    | 2,000 x `doc['rank'].value`, doc of 20,000 nodes   |  1.13 s | 155 ms |

Flat in the size of the input, because the trip happens at a fixed work count.
Through `_search`, the first of those is now a 400 `script_exception` naming
the budget.

## Cost to ordinary scripts

Interleaved A/B: base and fixed binaries alternated inside each repetition and
pinned to the same idle core, 15 repetitions, each figure already the best of
40 trials. Reporting the best-of-repetitions, which is the least-interrupted
observation on each side:

    | script                             | base       | fixed      | delta   |
    |------------------------------------|-----------:|-----------:|--------:|
    | 42 B, reads `doc[...]` and `params`|   274.8 ns |   353.8 ns |  +28.7% |
    | 237 B arithmetic                   |  1124.7 ns |  1230.6 ns |   +9.4% |
    | 59,773 B arithmetic, 34,407 units  |   271.1 µs |   298.5 µs |  +10.1% |

These are larger than the numbers the first revision of this change quoted
(+6.9% / +0.46%), and the difference is not the round-2 fixes: measured on the
same run, the first revision alone is +19.5% / +0.9% / +5.3% against the same
base. A probe build carrying only the `eval_access_chain` charge lands within
1% of it on all three shapes, so the security fix itself is not what costs —
the accounting was always the price of the budget.

Of the remainder, the tiny script's +25 ns is the one clock read per evaluation
that anchoring the deadline requires, which is the right order for a vDSO
`clock_gettime`. The sub-5% gaps on the two arithmetic shapes are not
attributable: a probe that does strictly *less* work than the final build
measures slower than it on one shape and faster on the other, which is
build-to-build codegen variation of the same magnitude as the effect.

## Query width: `max_clause_count`

There was no clause limit anywhere, so a query could be arbitrarily *wide* as
well, and width is what multiplies per-document work. `MAX_CLAUSE_COUNT` =
1,024 matches Elasticsearch's documented
`indices.query.bool.max_clause_count` default and returns its
`too_many_clauses` error, so a query this refuses is one the system being
emulated already refused. Clauses are charged before they are parsed and summed
across the whole query, not per `bool`, and every clause-list shape counts
towards the same total: `bool`, `span_or`/`span_near` clauses, `dis_max.queries`
and an array `knn.filter`.

A cap is only worth having if tripping it is *visible*. `parse_span_or` and
`parse_span_near` collected clauses with `filter_map(|v| parse_query(v).ok())`,
discarding any clause the parser refused. That was survivable while every
refusable clause was malformed; with a width cap it becomes a correctness bug,
because a well-formed clause that is merely too wide now fails to parse. A
`span_or` over [an over-wide `bool`, a `span_term`] parsed to a `SpanOr`
containing only the `span_term` — 200 OK for a query nobody asked for — and a
`span_near` whose only clause was over-wide collapsed to `MatchNone`: 200 OK,
zero hits, no error anywhere. A silently different query is worse than an
unbounded one. Every clause error now propagates.

## Verification

  * Reverting only `painless.rs` and keeping the tests:
    `bare_params_reads_are_charged_for_what_they_materialise`,
    `subscripted_params_reads_are_charged_too` and
    `params_source_reads_are_charged_for_the_document_they_clone` all fail,
    each reporting that the script was charged 2,048 work units, and
    `script_cpu_budget_http` fails on the wall clock having to notice what the
    op ceiling should have. The work-count assertions are deliberate: an
    uncharged read is still stopped *eventually*, after ~15 s, so a test that
    only asserts "some resource limit tripped" passes on the broken tree.
  * Reverting only `parser.rs` and keeping `tests/span_clause_refusal.rs`: 6 of
    its 9 tests fail, reporting the dropped clause and the `MatchNone`
    collapse. That file lives outside `parser.rs` precisely so it survives the
    revert.
  * `the_time_budget_trips_on_its_own_inside_an_expired_request` exercises the
    wall-clock half on its own; every other test here trips the deterministic
    op ceiling first.
  * `cargo test --release -p xerj-query -p xerj-engine -p xerj-api` green.
  * `cargo fmt --check` and `cargo clippy --lib --tests -- -D warnings` clean on
    all three crates.

…U DoS (#133)

* fix(painless): bound what a script DOES per document, not just its shape (#122)

Every limit the interpreter had bounds a structural property of a script: how
deep it nests (`MAX_PARSE_DEPTH`, `MAX_EVAL_DEPTH`), how many closure calls it
makes (`MAX_CALL_DEPTH`, `MAX_CALL_COUNT`), how many bytes of source it is
(`MAX_SCRIPT_LEN`). None of them bounds work, and shape and work are not the
same thing.

Measured on this branch's base in a release build, one core, `taskset`-pinned.
Every script below is flat, closure-free, nests nothing, calls nothing, and is
well inside the 64 KiB source cap — so every limit named above passes it:

    | script                                            | source   | per document |
    |---------------------------------------------------|---------:|-------------:|
    | 3,000 x `params.blob;`, params of 50,000 nodes     | 38.1 KiB |       9.03 s |
    | 3,000 x `params.blob;`, params of 200,000 nodes    | 38.1 KiB |      37.35 s |
    | 3,000 x `params['_source'];`, doc of 100,000 nodes | 55.7 KiB |      17.17 s |
    | 2,000 x `doc['rank'].value`, doc of 20,000 nodes   | 52.8 KiB |       1.13 s |

All four are O(size) behind an expression that looks O(1): `params.x` converts
a caller-supplied subtree into interpreter values, `params['_source']` clones
the document, and `get_doc_value` clones the whole document before walking into
it. None of that cost is visible in the statement count, which is why a step
counter alone would not have caught any of it. The doc-scan cooperative timeout
poll cannot help either: it only decides how often a document *boundary* is
checked, so a request `timeout` cannot interrupt a single expensive document at
all.

## The budget

A work counter inside `eval_painless`, charged for two things:

  * one unit per interpreter step (`eval_expr` / `exec_stmt` entry), and
  * one unit per node, plus one per 64 bytes, of every value a step produces.

The second half is the part that matters, and it has to be charged on *every*
path that produces a value — including the fast paths in `eval_access_chain`
that resolve `doc`, `params` and `Math` roots without a preceding value. Those
are exactly the paths the table above exploits.

`params` member reads are charged *before* the value is materialised
(`charged_params_read`), so the budget bounds how many such conversions an
evaluation may perform rather than performing one and asking afterwards.
`params['_source']` pays the document's weight through the same
`charge_document_read` that `doc['x'].value` uses.

`MAX_SCRIPT_OPS` = 5,000,000 is calibrated, not guessed: the largest *benign*
script the source cap admits (4,300 statements of `t = t + N.5;`, 59,773 bytes)
costs a measured 34,407 work units, so the ceiling keeps a 145x margin over the
worst legitimate script that can be submitted.
`painless::tests::the_budget_leaves_the_largest_benign_script_a_wide_margin`
pins that number so a change to the charging scheme reports how far the margin
moved.

## Integration with the request deadline

Every 1,024 work units the counter also reads the clock. The slice an
evaluation gets is the enclosing request's *remaining* time, clamped to
[100 ms, 500 ms], published by `with_script_deadline` from `Index::search` —
the same task-local mechanism as the fault sink.

That deadline is resolved in `PainlessCtx::new`, before any work is charged,
and this is load-bearing rather than tidy. Deriving it inside the first
`clock_check` costs one sampling window to *establish* the deadline and a
second one to be able to fail it, and a window is bounded in work units, not in
time — so "one window" is however long the largest single charge takes. It cost
one clock read per evaluation to fix; the measured overshoot against a 100 ms
slice is now 0.48 ms on a 10,000-node document, 1.17 ms at 40,000 and 7.11 ms
at 160,000, growing with the cost of one charged step exactly as that argument
says it should.

The 100 ms floor is what keeps the trip meaningful. The request deadline is
routinely already past when a script runs, because the doc scan only checks it
every 4,096 documents, so cutting evaluations off the moment it passes would
turn every ordinary `timeout` into a `script_exception` 400. 100 ms is ~360x
the measured 278 µs cost of the largest benign script that can be submitted.

Both trips classify as resource limits (`is_resource_limit_error`), so they
reach the caller as a 400 through the sink #116 built rather than degrading to
a plausible wrong score. Once any evaluation in a request has tripped, later
ones short-circuit on the recorded fault, which makes ONE budget the bound for
a whole request rather than one per document.

## Result

    | script                                            | before  | after  |
    |---------------------------------------------------|--------:|-------:|
    | 3,000 x `params.blob;`, params of 50,000 nodes     |  9.03 s |  97 ms |
    | 3,000 x `params.blob;`, params of 200,000 nodes    | 37.35 s | 109 ms |
    | 3,000 x `params['_source'];`, doc of 100,000 nodes | 17.17 s | 168 ms |
    | 2,000 x `doc['rank'].value`, doc of 20,000 nodes   |  1.13 s | 155 ms |

Flat in the size of the input, because the trip happens at a fixed work count.
Through `_search`, the first of those is now a 400 `script_exception` naming
the budget.

## Cost to ordinary scripts

Interleaved A/B: base and fixed binaries alternated inside each repetition and
pinned to the same idle core, 15 repetitions, each figure already the best of
40 trials. Reporting the best-of-repetitions, which is the least-interrupted
observation on each side:

    | script                             | base       | fixed      | delta   |
    |------------------------------------|-----------:|-----------:|--------:|
    | 42 B, reads `doc[...]` and `params`|   274.8 ns |   353.8 ns |  +28.7% |
    | 237 B arithmetic                   |  1124.7 ns |  1230.6 ns |   +9.4% |
    | 59,773 B arithmetic, 34,407 units  |   271.1 µs |   298.5 µs |  +10.1% |

These are larger than the numbers the first revision of this change quoted
(+6.9% / +0.46%), and the difference is not the round-2 fixes: measured on the
same run, the first revision alone is +19.5% / +0.9% / +5.3% against the same
base. A probe build carrying only the `eval_access_chain` charge lands within
1% of it on all three shapes, so the security fix itself is not what costs —
the accounting was always the price of the budget.

Of the remainder, the tiny script's +25 ns is the one clock read per evaluation
that anchoring the deadline requires, which is the right order for a vDSO
`clock_gettime`. The sub-5% gaps on the two arithmetic shapes are not
attributable: a probe that does strictly *less* work than the final build
measures slower than it on one shape and faster on the other, which is
build-to-build codegen variation of the same magnitude as the effect.

## Query width: `max_clause_count`

There was no clause limit anywhere, so a query could be arbitrarily *wide* as
well, and width is what multiplies per-document work. `MAX_CLAUSE_COUNT` =
1,024 matches Elasticsearch's documented
`indices.query.bool.max_clause_count` default and returns its
`too_many_clauses` error, so a query this refuses is one the system being
emulated already refused. Clauses are charged before they are parsed and summed
across the whole query, not per `bool`, and every clause-list shape counts
towards the same total: `bool`, `span_or`/`span_near` clauses, `dis_max.queries`
and an array `knn.filter`.

A cap is only worth having if tripping it is *visible*. `parse_span_or` and
`parse_span_near` collected clauses with `filter_map(|v| parse_query(v).ok())`,
discarding any clause the parser refused. That was survivable while every
refusable clause was malformed; with a width cap it becomes a correctness bug,
because a well-formed clause that is merely too wide now fails to parse. A
`span_or` over [an over-wide `bool`, a `span_term`] parsed to a `SpanOr`
containing only the `span_term` — 200 OK for a query nobody asked for — and a
`span_near` whose only clause was over-wide collapsed to `MatchNone`: 200 OK,
zero hits, no error anywhere. A silently different query is worse than an
unbounded one. Every clause error now propagates.

## Verification

  * Reverting only `painless.rs` and keeping the tests:
    `bare_params_reads_are_charged_for_what_they_materialise`,
    `subscripted_params_reads_are_charged_too` and
    `params_source_reads_are_charged_for_the_document_they_clone` all fail,
    each reporting that the script was charged 2,048 work units, and
    `script_cpu_budget_http` fails on the wall clock having to notice what the
    op ceiling should have. The work-count assertions are deliberate: an
    uncharged read is still stopped *eventually*, after ~15 s, so a test that
    only asserts "some resource limit tripped" passes on the broken tree.
  * Reverting only `parser.rs` and keeping `tests/span_clause_refusal.rs`: 6 of
    its 9 tests fail, reporting the dropped clause and the `MatchNone`
    collapse. That file lives outside `parser.rs` precisely so it survives the
    revert.
  * `the_time_budget_trips_on_its_own_inside_an_expired_request` exercises the
    wall-clock half on its own; every other test here trips the deterministic
    op ceiling first.
  * `cargo test --release -p xerj-query -p xerj-engine -p xerj-api` green.
  * `cargo fmt --check` and `cargo clippy --lib --tests -- -D warnings` clean on
    all three crates.

* fix(query): keep span-clause parsing lenient for unsupported types

The clause-cap fix made parse_span_clauses propagate EVERY clause parse error,
not just the MAX_CLAUSE_COUNT trip it was meant to surface. That regressed a
previously-passing ES-compat case: a `span_near` containing a `span_multi`
(a clause type this engine does not implement) went from silently dropping the
unsupported clause to 400-ing the whole search — conformance
search/190_index_prefix_search.

The cap must stay fatal — that was the #122 defect, an over-cap span quietly
becoming a different query. An unsupported clause type must not be. So
charge_clause() still propagates via `?`, while an ordinary parse error for an
unknown clause type is dropped as before.

Verified: full ES-compat suite 1365 passed / 0 failed (was 1 failed on the
span_multi case); the clause-cap refusal tests still pass.

* fix(query): propagate only the clause-cap error from span clauses, drop the rest

My first conformance fix dropped ALL span-clause parse errors, which silently
re-dropped a cap-refused clause: MAX_CLAUSE_COUNT trips INSIDE parse_query when
a clause is a wide `bool`, not in charge_clause, so `if let Ok` swallowed it and
a span with one over-cap bool collapsed to MatchNone again. That broke the
branch's own span_clause_refusal tests (a_refused_span_near_clause_does_not_
become_match_none, and the dis_max case).

Correct rule: the cap is fatal wherever it trips — propagate any error whose
message is `too_many_clauses`, from charge_clause OR from a nested bool. Every
other parse failure (an unimplemented `span_multi`, a malformed clause) is
dropped, which is the pre-#122 lenient behaviour ES-compat conformance depends
on (search/190_index_prefix_search).

Retargeted the one test that asserted a malformed clause is fatal: it now
asserts the unsupported clause DROPS and the supported one survives, matching
conformance. All 9 span_clause_refusal tests pass; the cap-fatal ones still
refuse, and full conformance was 1365/0 with unsupported clauses dropping.