Skip to content

feat(extension): experimental target readiness for Jev acts - #2994

Draft
miguelg719 wants to merge 9 commits into
jev/7-overlapfrom
jev/8-target-readiness
Draft

miguelg719 wants to merge 9 commits into
jev/7-overlapfrom
jev/8-target-readiness

Conversation

@miguelg719

@miguelg719 miguelg719 commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator

Stack

Stack: #2951#2952#2953#2954#2955#2988 WebMCP tools → #2993 settle overlap → this PR. Base: jev/7-overlap.

Objective

The least caller-visible wait at reasonable accuracy. After #2993 the DOM-settle wait (network quiet ≥ 500 ms, capped at 5 s) is more than half of a Jev act. This PR lets an act go ahead as soon as its target is there and hittable, and keeps the settle wait only as the upper bound.

Can Jev tell whether a page is loaded? (measured first, so it is not the design)

30 → 40 sites recorded every 200 ms from navigation start (1,569 samples); completeness = share of the page's final content already present.

signal fires at (median) page < 70% complete when it fires median wait past ≥ 90% complete
DOMContentLoaded 1.05 s 21 / 30 sites 0.34 s
load event 2.23 s 9 / 28 0.69 s
network quiet 500 ms (≈ the settle wait) 2.94 s 3 / 30 1.24 s
Jev over a content digest, four phrasings first sample, always 15 / 30 n/a

Content alone carries no "loadedness": a half-loaded page reads as a smaller finished one. With loading cues (in-flight requests, pending images, aria-busy, progress bars, spinner-like elements, interactive-element count) Jev does separate settled from loading pages (0.85–0.94 accuracy vs a 0.81–0.90 majority baseline); "cues quiet twice + Jev settled ≥ 0.6" let 15 of 40 pages go early with 2 premature exits, against 7 premature for the network-quiet heuristic itself. Wired live as a gate, it refused ("page busy") on exactly the pages readiness would have sped up and bought no accuracy, so it was removed (commit history has it). The objective is speed at reasonable accuracy; a page-level verdict trades the former for safety the evals did not show was needed.

What this PR does

Opt-in experimentalJevAct.targetReadiness (default off):

  • The first snapshot is captured while intent is asked; the ordinary pick runs on it.
  • Guard. If Jev accepts a target, one in-page call checks the selector still resolves to that node, it is connected and enabled, in the same place two animation frames later, and (pointer actions) a click at its centre would hit it, not an overlay (elementFromPoint; off-screen targets skip this). Refusals are traced with the reason and, when covered, what covers it.
  • Keeps looking. Until the target is there and hittable the act keeps looking for as long as the settle wait itself runs: a fresh snapshot each round, Jev re-asked only when the candidates it would see have changed. When the settle wait ends first, it decides as before. Click / hover / double-click and fill; other families wait as before.
  • The same guard runs before every pointer act once the settle wait is over: a consent or bot overlay that arrives after load covers the target the pick found (BBC: 3/3 clicks landed on the overlay; Target and Zillow: a px-captcha-modal iframe). The click is held while covered or moving, up to 1.5 s, then proceeds; the trace names the cover.
  • Two-step widgets wait in the page for a visible option (≤ 400 ms) instead of snapshot → 450 ms sleep → snapshot.

The guard and "wait for the effect, not the page" are borrowed from browser-use/jev-ultrafast.

Results (Browserbase, gemini-3.8-flash fallback)

Ordinary suites, 1 trial, caller-side time (settle included):

act (40) breadth (40, not in the stack)
pass, #2993 → this PR 39/40 → 39/40 37/40 → 38/40
caller-side median, Jev-only acts 1001 → 737 ms 1118 → 1379 ms*
acts that went early 35 / 49 29 / 48
guard refusals not found 19 · no selector 8 · covered 2 · other node 1 not found 5 · other node 3

* breadth median moved 889 → 1231 → 1379 ms across three runs of near-identical code on the same day; that spread is Browserbase/network variance, not the change. Earlier same-day run of this design: 785 ms for the acts that went early.

Load race, 15 tasks × 3 trials, act issued right after domcontentloaded on late-loading sites (not in the stack):

arm pass caller-side p50, all acts successful acts p50
LLM baseline, settle wait 18/43 7.5 s 8.2 s
Jev, settle wait 23/45 6.4 s 6.1 s
Jev + readiness, 3 polls then settle 21/45 5.3 s 5.3 s
Jev + readiness, keeps looking (this PR) 23/45 5.1 s 5.0 s

Six tasks fail in every arm for the same reasons: Airbnb, Target, Zillow put a modal or a PerimeterX captcha over the target (the guard reports covered: iframe: px-captcha-modal, waits its cap, then clicks — into the wall); CNN never gets past a "Loading…" placeholder within the 5 s cap; Spotify's sidebar button is present and hittable but the SPA has not attached handlers, so the click does nothing in the settle arm too; IKEA's "Products" opens a menu (task design). These are correctness gaps in reporting (a click that did nothing is returned as success), not readiness regressions; retryNoEffect is the existing opt-in that would catch them and is worth measuring on this suite.

What the floor is now. On heavy pages the first accessibility-tree capture is 3–6 s while the page is still churning (readiness "ready at attempt 1" ends at 2.5–8 s with intent ~150 ms and pick ~150 ms). The settle wait is no longer the binding constraint on those pages; snapshot capture is. That is the next lever (cheaper first snapshot, or an in-page DOM reader for the pointer view as jev-ultrafast does), and it is outside this PR.

Caveats: single trials on the ordinary suites; the load suite is small and noisy; eval pages rarely exercise a target that appears before it is usable, which is why this stays opt-in.

Testing

Pipeline tests: acts while settled never resolves once the guard says ok; keeps looking and acts when a late target lands, without a Jev request per unchanged snapshot; waits for settle while the guard reports moving / covered or another node; pre-act guard holds a covered click until it clears and clicks anyway past the cap; the settle-overlap ordering test from #2993. Full extension/protocol/sdk-ts/root suites, typecheck, lint, fmt, extensionpack --check pass locally; CI green.

@changeset-bot

changeset-bot Bot commented Sep 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 4298bb4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 20 packages
Name Type
@browserbasehq/stagehand-extension Patch
@browserbasehq/stagehand-go Patch
@browserbasehq/stagehand Patch
@browserbasehq/stagehand-python Patch
browse Patch
@browserbasehq/stagehand-integrations Patch
@browserbasehq/stagehand-integrations-example-eve-facade Patch
@browserbasehq/stagehand-integrations-example-pi-facade Patch
@browserbasehq/stagehand-integrations-claude-agent-sdk Patch
@browserbasehq/stagehand-integrations-example-claude-code-facade Patch
@browserbasehq/stagehand-integrations-codex-sdk Patch
@browserbasehq/stagehand-integrations-example-codex-facade Patch
@browserbasehq/stagehand-integrations-cursor-sdk Patch
@browserbasehq/stagehand-integrations-deepagents-sdk Patch
@browserbasehq/stagehand-integrations-eve-sdk Patch
@browserbasehq/stagehand-integrations-fx-sdk Patch
@browserbasehq/stagehand-integrations-mastra-sdk Patch
@browserbasehq/stagehand-integrations-example-mastra-facade Patch
@browserbasehq/stagehand-integrations-pi-sdk Patch
@browserbasehq/stagehand-integrations-example-vercel-ai-facade Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@miguelg719
miguelg719 force-pushed the jev/8-target-readiness branch from 00e84d7 to 4298bb4 Compare September 21, 2026 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant