jevwright

Browser tests for business flows, written as the steps a user takes.

jevwright opens your app in Chromium and hands one step at a time to TypeSafe's Jev, which picks the control the step means on the live page. The path it finds is recorded, so later runs replay it without calling a model. The model never decides whether a test passed: your code does, through the requests you declare, your own checks against the database or API, invariants, and a monitor that watches every page for crashes and broken output.

import { act, check, defineTest, reload, verify } from '@hazymoon/jevwright';

export const profileSave = defineTest({
    id: 'profile-save',
    title: 'Edit the display name, save, and see it after a reload',
    risk: 'A saved profile change is lost after a reload',
    start: '/settings/profile',
    data: { name: 'Ada Lovelace' },
    steps: () => [
        act('Change Display name to {name}'),
        act('Save the profile', { expect: { write: { method: 'PATCH', path: '/api/profile' } } }),
        verify('the API returns the new name', async ({ page }) => (await (await page.request.get('/api/profile')).json()).name === 'Ada Lovelace'),
        reload(),
        check('The Display name field shows {name}'),
    ],
});

• Steps read like the test plan. No selectors or page objects. A redesign that keeps the words on the screen keeps the test working, and when a recorded path goes stale, the step is grounded again and the recording updated.
• The model finds controls; code decides the verdict. A step is done when the request you declared succeeds, not when a model says so. Business facts come from verify callbacks that read your database or API.
• Recorded once, replayed for free. The first run records each step as a semantic target (role, name, nearby text, row). Replays call no model: --mode replay is deterministic and costs nothing in CI.
• Every failure has a cause. Each failure is reported as product, agent, environment, model or timeout, with the evidence: the failing step, its requests, screenshots, Jev's decisions and a Playwright trace. Before blaming your app, jevwright checks whether each AI-driven step acted on the control it names.
• Implicit checks on every step. Uncaught exceptions, 5xx and unexpected 4xx responses, failed requests, console errors, hydration mismatches, and text such as undefined, NaN, Invalid Date or an untranslated key.

npm install -D @hazymoon/jevwright playwright
npx playwright install chromium

Requirements:

• Node.js 22 or newer (Bun works too).
• Playwright 1.63 or newer.
• For AI runs, an OpenRouter or Vercel AI Gateway key. Replay runs need no key.

1. Scaffold a config and an example test:

   npx jevwright init

   This creates jevwright.config.ts and jevwright/example.ts, and adds .jevwright/ (run reports) to .gitignore.

2. Point it at your app. Set baseURL to a running app, or give the config a setup function that starts a fresh one per run. Then rewrite the example for a real page, using the words your UI shows.

3. Author the test:

   export OPENROUTER_API_KEY=sk-or-...   # or VERCEL_AI_GATEWAY_API_KEY
   npx jevwright run --test profile-save --new

   --new runs the test twice in one go. The first pass lets the model ground every step and writes jevwright/recordings/profile-save.json. The second replays only that recording, with no model. When both pass, commit the test and its recording.

4. Run it from then on:

   npx jevwright run                 # auto: replay recordings, heal stale steps with AI
   npx jevwright run --mode replay   # recordings only: no key, no model calls (CI)
   npx jevwright serve               # open the latest report in your browser

A test is a list of steps. Code owns their order, and each step does one thing:

A test also declares:

• risk: the concrete business failure it guards against.
• data: the values its steps insert as {key}.
• fixture: the test's own accounts and seed data.
• invariants: what must never change, checked before the first step and after every step.

The guide to writing tests explains what to hand to jevwright and what to leave to lower-level tests. It also covers:

• how to word steps;
• how to declare expected requests and errors;
• how to inject faults;
• how to author a new test;
• how to read a failure.

Sign each test's user in from its fixture: add a session cookie, or post to your sign-in API with context.request, which shares cookies with the page. This is faster than signing in through the UI, calls no model, and keeps credentials out of model requests. See Signing in.

Each run writes a directory under .jevwright/runs/:

A failed test is retried once by default. A test that later passes is reported as flaky, not fixed. A test marked with knownIssue that fails on the product is reported as known, so a confirmed bug does not fail every run until it is fixed.

AI runs go through your gateway, so you pay its prices. One example, measured on a 55-test suite of a web app in September 2026 with the default models:

Your costs depend on your pages and steps. The limits:

• Each run stops at --max-cost (default $1).
• Each test attempt stops at 80 model calls.

Replay mode needs no key and no model, so it is the cheapest gate:

- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npx jevwright run --mode replay
- uses: actions/upload-artifact@v4
  if: failure()
  with: { name: jevwright-report, path: .jevwright/runs }

Replay rechecks the direct page evidence recorded by passing check steps. Checks without usable evidence are unverified and make the CLI exit 1; --allow-unverified overrides that exit policy while retaining the report count. A step without a recording fails, so a test committed without its recording cannot pass CI unnoticed. To heal stale recordings and run checks in CI, run the default auto mode and provide OPENROUTER_API_KEY as a secret. CLI replay leaves recording files unchanged. A failed auto attempt retains an existing complete recording; without one, it can save a verified partial prefix. Secret checks still apply before writing a replacement.

• Writing tests: what to test with jevwright, step wording, expectations, faults, signing in, authoring, failures.
• Configuration and CLI: every config field and flag, setup and env, the programmatic API.
• How it works: observation, the decision loop, recordings, failure attribution, safety.

A helper output generation or parsing failure is reported as model. Review the saved model error and retry the run. Delayed autosave receives the declared expectation timeout even after the last allowed action.

Anything else, or an engine that gets a well-worded step wrong: open an issue with the run's report.md.

Unmarked legacy end states retain their half-anchor rule. New strict end states require every durable anchor and reject newly introduced errors that were absent when recording. Informational role=alert content does not qualify as an error. Strict replay fails when required end states do not match. Legacy replay reports end-state drift and leaves the test verdict to later checks and code verification. The report lists missing anchors. Confirm the cause with an auto run; a mismatch alone does not establish a product defect.

• The engine can:

  • click, type into fields and contenteditable editors, press keyboard shortcuts, select exact editable text, and choose native or ARIA options;
  • upload declared files, including a chosen group in one multiple input, and verify declared downloads;
  • follow popups and return after they close;
  • scroll the page or a container, search for instruction text while scrolling, and bring static text into view;
  • wait for transient loading signals and relocate stale targets after DOM replacement;
  • hover to reveal controls;
  • choose right-click, an 800 ms long press, double-click, drag between two semantic targets, and browser back;
  • reach script-created closed shadow roots without opening them to application code.

• It cannot:

  • draw on a canvas;
  • use the clipboard.

  When one of these is only a precondition, do it in a fixture or run step. When it is the behavior under test, use a scripted Playwright test.

• Declarative closed shadow roots are not captured. The engine wraps attachShadow and keeps references to script-created roots; the roots retain their original mode.

• Dragging covers HTML drag-and-drop and pointer gestures. Drawing still needs scripted Playwright steps. Keyboard shortcuts and exact editable text selection can drive rich-text formatting. Computed format ranges are available only when text-node offsets match the complete unprotected rendered value; use code verification for the required formatting.

• Scroll searches use a container-sized budget, share at most 120 seconds per step, and stop after 500 viewports or five unchanged positions. Missing goals fail explicitly.

• Desktop defaults to 1280×900. Use device: "mobile" for touch and a mobile user agent, or provide a custom device.

• Page values can be entered only when their exact text appears in the current observation. Replay re-reads them from recorded surrounding text; changed or ambiguous sources need an auto run. Declared secrets remain available only through their keys.

• check waits up to 15 additional seconds per observation for visible loading. It then checks the visible content; missing evidence while loading is a timeout. check requires the asserted content to be visible. A count or saved-button state cannot prove what another tab contains.

• Failed checks receive a second look. Visible contradictions, or missing expected content in an open visible region, are product; a collapsed or unopened region is agent. Uncertain region evidence receives adjudication.

• Compound factual checks can record several fields or exact page quotes. Replay rechecks every selected piece in its recorded region. Missing direct evidence and recognized absence or negative clauses remain unverified.

• A positive adjudication can retain proof from a check that already leaned positive. The proof must match its observation; adjudication creates no proof.

• Quoted subjects and unquoted visible labels can use a unique local text excerpt that retains surrounding context. Replay requires the whole excerpt; changing dates, generated ids, clipped text and live content cannot supply this proof.

• Check proof excludes element names containing generated ids or dates. Labeled read-only cards retain rendered paragraph boundaries when their visible text agrees.

• Proof containing Unicode … remains unverified, including a literal ellipsis. Use verify to assert that exact text.

• check is a model judgment in auto/AI mode. Replay verifies recorded direct evidence; checks without it remain unverified. Exact values (money, multilingual text, line breaks) and negative claims belong in verify.

• New recordings bind app routes to the run’s baseURL, so changing the host or port alone does not invalidate the route check. Path, sorted query and recorded effects must still match. Routes to other origins remain literal. Unmarked legacy routes retain path-only checks.

• Chromium only.

MIT

A single click can end at its newly opened dialog when the next act explicitly chooses a dialog control. This requires causal evidence and an independent target audit. Additional clauses and repeated clicks retain normal recovery. Later checks still verify the resulting behavior.