Skip to content

ultraflow: two-stage classify, eager pauses, and context-menu/download operations to beat jev-ultrafast #617

Description

@LivioGama

Head-to-head evidence from 2026-10-04 (pixel ultraflow vs browser-use/jev-ultrafast on the same Google Drive task, run live):

  • pixel discover blocked in 36.7s: 64-label Ollaya budget truncated the action space, the model clicked a filter chip at p=0.21 then answered BLOCKED; the offered/truncated counts were invisible in the human log.
  • jev blocked too (4.0s): search executed fine, but the results page stalled it. Same perception failure class, ~5x faster where the loop works.
  • Verified on the real Drive results page with agent-browser: file rows ARE exposed (link "Notion Invoice January 12, 2025.pdf" [ref=e81]), the download path is a context menu (Download as an ordinary menuitem), and Advanced search is a deterministic form front door.

Work plan (each its own PR on top of main):

  1. Two-stage classify: operation first (7 options), then target among only that operation's slots - kills the 64-label ceiling while keeping every decision classified, never generated.
  2. Eager pause policy: poll url/changed_since instead of the fixed 3s open + 2s click sleeps in pixel-flow execute.
  3. contextmenu operation + download flow step with on-disk verification (Jev structurally cannot do either).
  4. Discovery honesty: print offered/truncated in render_trace, disclose the label-budget cut in the elements table, do not cancel remaining --repeat runs on a model-chosen BLOCKED.
  5. Perception completion: scrollIntoView-before-act rule; classify policy guidance for search front doors, filter-chip traps and menu-close behavior.

Activity

  1. navidemad commented on Oct 6, 2026

    @navidemad
    Collaborator

    Triage, as of main at d4f83f9, against the work plan:

    1. Two-stage classify: done (crates/pixel-ultraflow/src/twostage.rs, choose_two_stage).
    2. Eager pause policy: done in feat(flow): poll the URL after open and click instead of sleeping #627.
    3. contextmenu operation and download step with on-disk verification: not found in pixel-flow or pixel-ultraflow.
    4. Discovery honesty: partly done (offered, truncated and offered_operations in discover.rs). To check: whether render_trace prints them, and whether a model-chosen BLOCKED still cancels the remaining --repeat runs.
    5. Perception completion (scrollIntoView-before-act, classify policy guidance): not found.

    Proposed: narrow this issue to 3–5, or split them into their own tasks and close this one.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions