Verist

Replay + diff for AI decisions — see what changes before you ship.

A deterministic workflow kernel that captures AI outputs as artifacts, replays them exactly, and shows reviewable diffs when prompts or models change. Human corrections survive recomputation by design.

Ask Verist Architect in ChatGPT or Gemini about replay semantics, diffs, and kernel invariants.

You tweak a prompt, upgrade a model, or change extraction logic. AI decisions silently change. A missing field breaks downstream code, a reclassified ticket confuses support, a weaker extraction goes unnoticed — until customers report it.

You can't answer the question that matters: "What decisions would change?"

• Logs show what happened, not what would change
• Testing on 3 examples doesn't catch the 2% regression in production
• When you recompute, human corrections get overwritten

So you don't ship. Or you ship and hope.

Verist gives you a different workflow:

change prompt → recompute → see diff → approve → ship

• Diff before shipping — See what would change after a prompt or model upgrade
• Replay any decision — Reproduce past AI decisions exactly from stored artifacts
• Human authority preserved — Manual overrides survive recomputation by design

Verist is not logging or observability — it is deterministic replay with reviewable diffs.

npm install verist @verist/cli zod

Get started with no API keys — verist init scaffolds a deterministic step using regex extraction:

npx verist init
npx verist capture --step parse-contact --input "verist/inputs/*.json"
npx verist test --step parse-contact

Once you have API keys, define an LLM-powered step and see what changes when you modify prompts:

OPENAI_API_KEY=sk-... bun examples/prompt-diff/quickstart.ts

✓ Baseline: 4 claims (specific numbers, names, dates)
✓ Recompute: 3 claims (vague summaries)

--- Diff ---
  claims[0]: "Acme Corp reported $4.2M in Q3 revenue" → "Acme Corp had strong Q3 revenue growth"
- claims[3]: "Plans to expand engineering team from 45 to 60 by March 2025"

Full example: examples/prompt-diff/

Verist captures AI outputs as artifacts. When you change something, replay against stored inputs and see a diff.

import { diff, formatDiff } from "verist";

const before = { claims: ["Revenue: $5M", "Headcount: 45"] };
const after = { claims: ["Revenue was strong"] };

console.log(formatDiff(diff(before, after)));
//   claims[0]: "Revenue: $5M" → "Revenue was strong"
// - claims[1]: "Headcount: 45"

import { z } from "zod";
import { extract } from "@verist/llm";
import { defineStep, run, unwrap, recompute, formatDiff, fail } from "verist";

const ClaimsSchema = z.object({ claims: z.array(z.string()) });

// Define a step with typed input/output
const extractClaims = defineStep({
  name: "extract-claims",
  input: z.object({ text: z.string() }),
  output: ClaimsSchema,
  run: async (input, ctx) => {
    const result = await extract(
      ctx,
      {
        model: "gpt-4o",
        messages: [{ role: "user", content: input.text }],
        responseFormat: "json",
      },
      ClaimsSchema,
    );
    if (!result.ok) return fail(result.error);
    return { output: result.value.data };
  },
});

// Run with artifact capture
const baseline = unwrap(
  await run(extractClaims, { text }, { adapters: { llm } }),
);

// Later: change prompt, recompute from snapshot
const recomputeResult = unwrap(
  await recompute(baseline, extractClaims, { adapters: { llm: newLlm } }),
);
console.log(formatDiff(recomputeResult.outputDiff));

For pure extraction steps, use defineExtractionStep() from @verist/llm to avoid boilerplate.

Expected failures should return fail(...) so runners can act on code and retryable. Throw only for programmer errors or invariant violations.

Use --format json or --format markdown for machine-readable output:

verist test --step extract-claims --format json    # structured JSON for scripts
verist test --step extract-claims --format markdown # PR comment summary

See CI Integration Guide for GitHub Actions examples.

• Prompt iteration — Test changes against production history before deploying
• Model upgrades — Quantify impact of switching models (GPT-4 → Claude, etc.)
• Safe recompute — Rerun AI logic without overwriting human corrections
• Decision audit — Replay any past decision exactly, with full provenance

Verist is not a chat framework or agent runtime. It's the trust layer that makes AI decisions reproducible and reviewable.

• Getting Started
• CI Integration Guide
• Replay and Diff Guide
• Step Execution Spec (fail(), defineExtractionStep)

Apache-2.0