Test prompt changes before they ship
Run your system prompt against real test cases on several OpenAI models at once. Compare the answers side by side and get per-metric scores, with reasons, from an LLM judge. It all runs in your browser.
OpenAI bills API usage to your own key. Litmiq never sees your key.
In the recorded support-bot run
Compared side by side, in your browser
Accuracy, tone, helpfulness, conciseness
Try it in 30 seconds, no key needed
Watch a real Litmiq run: one support-bot prompt, 8 test cases, 3 models, scored by an LLM judge. These are recorded outputs from Llama 3.3 70B, Gemma 4 26B, Llama 3.1 8B on Cloudflare Workers AI (13 Nov 2026), replayed in your browser, not generated live.
Suite: Compare OpenAI models on one prompt (Harbor Lane Books support, fictional) · recorded run (JSON)
What Litmiq does today
A focused browser tool for checking a prompt change against real cases before you ship it.
From prompt edit to a clear comparison
// Shape of the JSON export. Values are filled in by a real run. { "allResults": [{ "testCase": { "idx", "input", "expected" }, "models": [{ "model": "gpt-6.1-terra", "output": /* model's answer */, "judgment": { "scores": { /* one per metric, 0–10 */ }, "overall", "reasoning" }, "error": /* set if the call or judge failed */ }] }], "timestamp" }
Start from a playbook, not a blank page
Each playbook explains what to test and why, with an illustrative case set you can open in the app in one click.
Guides and comparisons teams start with
Practical pages on regression testing, LLM judges, tool choice, and moving off OpenAI Evals before 30 November 2026.
How does Litmiq compare?
Fair, sourced comparisons with the tools people use for prompt testing, including what to use after OpenAI Evals shuts down on 30 November 2026.
LLM evaluation glossary · Prompt testing checklist · About Litmiq
The app is free. Pro adds CI, history and alerts.
Use everything Litmiq does today in your browser, with no account. OpenAI bills API usage to your key. Pro ($19/month) and Team ($49/month) add saved suites, CI on a schedule and regression alerts: see the pricing page.
- No signup or account
- OpenAI models compared side by side
- Test cases with expected behavior
- Per-metric LLM-judge scores and reasons
- Setup autosaved in your browser
- JSON export after each run
- Litmiq product updates only
- Unsubscribe anytime
See our Privacy Policy.
Check your next prompt change before it ships
Free · No signup · OpenAI API usage is billed to your own key.
FAQ
What is Litmiq?
Litmiq is a free, browser-based prompt evaluation tool. You paste a system prompt, add test cases with optional expected behavior, run them on several OpenAI and Anthropic models side by side, and an LLM judge scores every output for accuracy, tone, helpfulness and conciseness (0-10) with reasons. No signup: you bring your own OpenAI or Anthropic API key, which stays in your browser. See About Litmiq.
Does Litmiq require an account?
No. The app runs in your browser. You provide your own OpenAI API key, which stays in localStorage.
What does Litmiq cost?
The app you use today is free, and stays free. OpenAI bills you for API usage when you run evals with your own key. Pro ($19/month) and Team ($49/month) add saved suites, CI on a schedule and regression alerts; see the pricing page.
How is Litmiq different from Promptfoo or DeepEval?
Promptfoo and DeepEval are strong for CLI and CI workflows. Litmiq is a browser UI for quick side-by-side prompt tests and LLM-judge scores without installing anything. Read the prompt evaluation tools comparison.
Which models does Litmiq support?
OpenAI chat models shown in the app's model picker. Requests go from your browser to the OpenAI Chat Completions API.
What can I use now that OpenAI Evals is shutting down?
OpenAI's deprecations page lists the Evals dashboard and API as scheduled to shut down on 30 November 2026, and OpenAI recommends Promptfoo for code-based evals. For quick, manual checks of a system prompt on several OpenAI models, Litmiq runs in your browser with no signup. Step by step: how to export and migrate your OpenAI Evals, or see Litmiq vs OpenAI Playground.
Can I start from a ready-made test suite?
Yes. The templates page has free, illustrative suites from our guides. Each opens in the app with its system prompt, scoring criteria and test cases filled in. If you've edited your setup, the app asks before replacing it and offers an undo.
Does Litmiq see my prompts or test cases?
No. There is no Litmiq backend for evals. Your setup is saved in your browser's localStorage, and when you run an eval your browser sends the prompt, cases and outputs directly to OpenAI's API with your key.
Litmiq Pro
Make every prompt change prove itself
Pro saves your suites and run history, exports a GitHub workflow that re-runs them on every pull request and on a schedule, and flags a regression when scores drop. $19 a month, cancel any time. Your API keys stay in your browser and your CI. Migrating off OpenAI Evals? The in-browser converter is free.
Free: OpenAI Evals migration checklist
The step-by-step list we use: export, convert, re-check scores, add a CI gate. One follow-up email, no newsletter.