Free · No signup · Bring your own OpenAI key

Test prompt changes before they ship

Run your system prompt against real test cases on several OpenAI models at once. Compare the answers side by side and get per-metric scores, with reasons, from an LLM judge. It all runs in your browser.

OpenAI bills API usage to your own key. Litmiq never sees your key.

REPLAYlitmiq.com/app · recorded run · 13 Nov 2026 · judge gpt-oss-120b
Real outputs and scores from Cloudflare Workers AI at temperature 0, replayed in your browser (not generated live). Watch the full run, or run your own with your OpenAI or Anthropic key.
Works with these OpenAI models today
GPT-6 Astra
GPT-6.1 Sol
GPT-6.1 Terra
GPT-6 Luna
GPT-5.4 Mini
8Test cases

In the recorded support-bot run

3Models

Compared side by side, in your browser

4Judge scores

Accuracy, tone, helpfulness, conciseness

Try it in 30 seconds, no key needed

Watch a real Litmiq run: one support-bot prompt, 8 test cases, 3 models, scored by an LLM judge. These are recorded outputs from Llama 3.3 70B, Gemma 4 26B, Llama 3.1 8B on Cloudflare Workers AI (13 Nov 2026), replayed in your browser, not generated live.

Suite: Compare OpenAI models on one prompt (Harbor Lane Books support, fictional) · recorded run (JSON)

Your turn: run the same suite on GPT or Claude. Paste your own OpenAI or Anthropic key in the app; it stays in your browser. Open this suite in the app →

What Litmiq does today

A focused browser tool for checking a prompt change against real cases before you ship it.

Test cases with expected behavior
Write the user inputs that matter and, optionally, what a good answer should do. Re-run the same cases whenever the prompt changes.
OpenAI and Anthropic models side by side
Pick one or more OpenAI models. Every test case runs on every selected model, and the outputs line up for comparison.
Per-metric LLM-judge scores
A judge model scores each output, guided by your criteria, and explains why. If its reply can't be read, the result is marked unscored — never given a default number.
Your criteria as the rubric
Describe what good looks like in plain language. The judge uses it as its rubric and reports four fixed scores: accuracy, tone, helpfulness and conciseness.
Bring your own key
Your OpenAI key stays in your browser's localStorage and is sent only to OpenAI's API. No account, no Litmiq backend.
Autosave and JSON export
Your setup autosaves in your browser. After a run, download every output, score and reason as JSON.

From prompt edit to a clear comparison

1
Paste your system prompt
Open the app — no signup. Add your OpenAI key once; it stays in your browser.
2
Add test cases
Real user inputs, each with an optional expected behavior.
3
Pick models and criteria
Select the OpenAI models to compare and describe how outputs should be scored.
4
Run, compare, export
Read outputs side by side with per-metric scores and reasons, then export the run as JSON. Judge scores are estimates — read the cases that matter.
litmiq-eval-….json · export shape
// Shape of the JSON export. Values are filled in by a real run.
{
  "allResults": [{
    "testCase": { "idx", "input", "expected" },
    "models": [{
      "model": "gpt-6.1-terra",
      "output": /* model's answer */,
      "judgment": {
        "scores": { /* one per metric, 0–10 */ },
        "overall",
        "reasoning"
      },
      "error": /* set if the call or judge failed */
    }]
  }],
  "timestamp"
}

Start from a playbook, not a blank page

Each playbook explains what to test and why, with an illustrative case set you can open in the app in one click.

8 illustrative cases that target changed numbers, dropped hedges, lost negations, invented owners and instructions hidden in the source.
12 illustrative single-turn injection cases with benign controls and a canary-string method for spotting system prompt leaks.
Test extraction prompts in three layers (parse, shape, values) with 10 illustrative cases and an offline checker script.
A Must / Must not / Okay template for expected behavior that a person or an LLM judge can check, with 8 rewrites and a 6-case suite.
16 illustrative single-turn cases for a support system prompt, grouped by how support bots go wrong, plus a scoring rubric.

All templates · All guides

How does Litmiq compare?

Fair, sourced comparisons with the tools people use for prompt testing, including what to use after OpenAI Evals shuts down on 30 November 2026.

A free browser app for quick prompt checks vs an open-source CLI for CI evals and red teaming: when to use each.
No-signup browser prompt tests vs a hosted platform for tracing, datasets and online evaluation.
Designing one prompt vs testing it on many cases, and what to use after OpenAI Evals shuts down on 30 Nov 2026.
Seven prompt testing and LLM eval tools grouped by job: browser checks, CI frameworks and hosted platforms.
A free browser app for quick prompt tests vs a hosted platform for tracing, playgrounds and experiments.
No-code browser prompt tests vs an open-source framework that runs LLM evals like unit tests.
A free, no-signup browser app for quick prompt tests vs open-source LLM tracing, prompt management and evals.

LLM evaluation glossary · Prompt testing checklist · About Litmiq

The app is free. Pro adds CI, history and alerts.

Use everything Litmiq does today in your browser, with no account. OpenAI bills API usage to your key. Pro ($19/month) and Team ($49/month) add saved suites, CI on a schedule and regression alerts: see the pricing page.

Product updates optional
Not needed to use the app. Leave your email if you'd like occasional Litmiq news — for example when new features ship.

  • Litmiq product updates only
  • Unsubscribe anytime

See our Privacy Policy.

Check your next prompt change before it ships

Free · No signup · OpenAI API usage is billed to your own key.

FAQ

What is Litmiq?

Litmiq is a free, browser-based prompt evaluation tool. You paste a system prompt, add test cases with optional expected behavior, run them on several OpenAI and Anthropic models side by side, and an LLM judge scores every output for accuracy, tone, helpfulness and conciseness (0-10) with reasons. No signup: you bring your own OpenAI or Anthropic API key, which stays in your browser. See About Litmiq.

Does Litmiq require an account?

No. The app runs in your browser. You provide your own OpenAI API key, which stays in localStorage.

What does Litmiq cost?

The app you use today is free, and stays free. OpenAI bills you for API usage when you run evals with your own key. Pro ($19/month) and Team ($49/month) add saved suites, CI on a schedule and regression alerts; see the pricing page.

How is Litmiq different from Promptfoo or DeepEval?

Promptfoo and DeepEval are strong for CLI and CI workflows. Litmiq is a browser UI for quick side-by-side prompt tests and LLM-judge scores without installing anything. Read the prompt evaluation tools comparison.

Which models does Litmiq support?

OpenAI chat models shown in the app's model picker. Requests go from your browser to the OpenAI Chat Completions API.

What can I use now that OpenAI Evals is shutting down?

OpenAI's deprecations page lists the Evals dashboard and API as scheduled to shut down on 30 November 2026, and OpenAI recommends Promptfoo for code-based evals. For quick, manual checks of a system prompt on several OpenAI models, Litmiq runs in your browser with no signup. Step by step: how to export and migrate your OpenAI Evals, or see Litmiq vs OpenAI Playground.

Can I start from a ready-made test suite?

Yes. The templates page has free, illustrative suites from our guides. Each opens in the app with its system prompt, scoring criteria and test cases filled in. If you've edited your setup, the app asks before replacing it and offers an undo.

Does Litmiq see my prompts or test cases?

No. There is no Litmiq backend for evals. Your setup is saved in your browser's localStorage, and when you run an eval your browser sends the prompt, cases and outputs directly to OpenAI's API with your key.

Litmiq Pro

Make every prompt change prove itself

Pro saves your suites and run history, exports a GitHub workflow that re-runs them on every pull request and on a schedule, and flags a regression when scores drop. $19 a month, cancel any time. Your API keys stay in your browser and your CI. Migrating off OpenAI Evals? The in-browser converter is free.

Free: OpenAI Evals migration checklist

The step-by-step list we use: export, convert, re-check scores, add a CI gate. One follow-up email, no newsletter.