TESTING FOR AI-BUILT WEBSITES AND APPS
Your AI agent says it works.
Make it prove it.
Testsy independently tests what your agent builds: your app on iPhone, iPad or Android, on the OS version you pick, or your site in a browser. Then it tells your agent what broke and how to fix it, and you can watch the replay.
https://app.testsy.dev/mcpBuilt it in Lovable, Bolt, Replit or v0? Test it here, free.
01 · PLAIN WORDS
Say what should happen. That's the test.
No code, no scripts. Tell your agent, or type it yourself, the way you'd explain it to a friend: what to do, and what must be true at the end. Testsy turns it into steps and checks you can read before anything runs.
- An app build, or the address of a deployed site (a preview link works)
- Sign-in accounts stored once; the password never goes in a prompt
- Files for uploads and imports, attached to the test
- Written in your own language, not just English
- No coding agent? Paste the address of your Lovable, Bolt, Replit or v0 app
Check that switching to Fahrenheit shows on the home screen of my iOS app.
testsy · plan_test (platform: "ios", appId: "com.skyline.weather", projectId: "p_skyline", description: "Add Sydney, switch the units to Fahrenheit, then check the home screen shows the temperature in °F")
5 steps · 3 checks · no questions · passed the planning review
testsy · create_test (projectId: "p_skyline", title: "Temperature shown in °F after switching units", platform: "ios", appId: "com.skyline.weather", plan: …)
t_w4t7 · saved with the approved checklist
Saved. It adds Sydney, switches the units to Fahrenheit and checks the home screen temperature ends in °F. Want me to run it?
02 · A SECOND OPINION
Your agent wrote it. Someone else should check it.
An agent that tests its own work tends to agree with itself. Testsy doesn't read your code. It only uses the app, the way your customers will, and reports what happened on the screen.
- A separate tester, not the same AI marking its own work
- Four honest outcomes: passed, failed, inconclusive or blocked
- Blocked is never a run you pay for
Sign-up is fixed. Everything works now.
Prove it.
testsy · run_test (testId: "t_k3m9")
j_c91m7 started · watchUrl to follow it live
testsy · wait_run (runId: "j_c91m7")
final · failed · 1 of 3 checks failed
The dashboard greets the user by name: expected "Hi, Sam", saw "Hi, undefined" (step 6)
The confirmation email arrives · The confirmation link opens the dashboard
I was wrong. The name is missing right after sign-up. Fixing it now.
The dashboard greeted the new user as “undefined”.
Web test in Chrome · started by AI agent (MCP)
The dashboard greets the user by name
Sign-up finished and the confirmation link opened the dashboard, but the greeting had no name in it.
2 checks passed
03 · EVERY CHANGE
Fix one thing without breaking another.
Agents fix one thing and quietly break another. Your saved tests run again after every change, on a schedule or after each deploy, so you find it before your first customer does.
- Saved tests your agent can run again with one call
- Schedules, with an email when one fails
- From CI when you have a pipeline
I changed the pricing page. Did anything else break?
testsy · run_project (projectId: "p_myapp")
8 runs started · none skipped
testsy · list_runs (projectId: "p_myapp", limit: 8)
7 passed
1 failed: "Coupon applies at checkout"
testsy · get_run (runId: "j_4h7c2")
expected "$39.00" · saw "$49.00"
The pricing change broke the coupon: that test passed yesterday. I'll fix it and run all 8 again.
MyApp
Test sign-up on iPhone, with the SMS code.
testsy · run_test (testId: "t_p7q2", device: "phone", osVersion: "iOS 26")
j_m2x8 started · iPhone · iOS 26
testsy · wait_run (runId: "j_m2x8")
final · passed · all 4 checks
The SMS code is accepted: saw "482913" (step 3)
The confirmation email arrives: saw "Confirm your account" (step 5)
The verification call is answered: pressed 1 (step 7)
A new user signed up with the SMS code. The SMS, the email and the call are in the run's Messages tab on the dashboard.
A new user signed up with the SMS code.
iOS test on iPhone · iOS 26 · started by AI agent (MCP)
This run's number and inbox: +1 555 010 0142, run-k3m9@inbox-test-engine.testsy.dev. They stopped accepting messages when the run ended.
- SMS received2:03:14 pm · step 3From MyApp482913
Your MyApp code is 482 913. It expires in 10 minutes.
- Email received2:03:41 pm · step 5Confirm your accountFrom hello@myapp.examplemyapp.example/confirm?token=…
Hi Sam, tap the link below to confirm your email address and finish setting up your account.
- Call received2:04:20 pm · step 7From +1 555 010 0199The test answered the call and pressed 1 to confirm.
04 · BEYOND YOUR LAPTOP
It tests what you can't test from your laptop.
Each test can have its own mailbox and its own phone number, so it signs up like a new customer: it opens the confirmation email, reads the SMS code, even takes a call. On iPhone, iPad and Android, on the OS version you pick.
- Its own mailbox on every plan, the free trial too
- Its own phone number for SMS codes and calls, on paid plans
- Phone or tablet, upright or on its side
- Websites in Chrome at desktop, tablet and phone sizes
05 · WATCH, DON'T READ
You don't need to read the code. Watch the replay.
Every run comes back as something you can watch: each step replayed, with the spot on the screen each value was read from. A green tick is a claim. A value is proof.
- See the proof: the exact screen region each value was read from, outlined in the replay
- Copy fix prompt: the failure, values and steps as one prompt for Cursor, Claude Code or Codex
- Five tabs on every result: Replay, Video, Values seen, Console and Network (plus the device log for apps)
- Share a read-only link, copy it as an issue, or fetch the same result as JSON and JUnit
testsy · wait_run (runId: "j_w4t7")
final · passed · all 3 checks
Sydney appears in the list of places: saw "Sydney, NSW" (step 2)
Fahrenheit is selected under Units: saw "Fahrenheit ✓" (step 4)
The home screen temperature ends in °F: saw "71°F" (step 5)
testsy · create_watch_link (runId: "j_w4t7")
Sydney was added and the home screen showed 71°F after switching to Fahrenheit. Here is the link to watch the replay yourself, no sign-in needed.
Four honest outcomes. Blocked is never a run you pay for.
These are the verdicts exactly as they appear at the top of a result.
Each check confirmed against a value the tester actually saw, with where it was read.
Counts as a runExpected versus seen, the step it was judged at, steps to reproduce and the moment in the replay.
Counts as a runA bot wall or sign-in got in the way. The result says where, instead of calling it your bug.
Not billedThe evidence wasn't on screen to decide. The result says what was missing; nothing is rounded up to a pass.
Counts as a run06 · THE FIX
You don't have to describe the bug. Copy the fix prompt.
When a check fails, Testsy writes the bug report for you: what was expected, what it saw, the step it happened on and how to reproduce it, as one prompt. Paste it to your agent, or let your agent fetch it itself.
- Expected versus seen, in plain words
- For Claude Code, Cursor, Codex or any agent you can paste into
- Run the same test again to confirm the fix
testsy · get_run (runId: "j_8f2k4")
failed · 1 of 2 checks: The saved project title matches after reload
expected "Launch notes" · saw "Untitled" · step 6, after the reload
The title is kept in the browser but never saved to the server. Fixing the save, then deploying.
testsy · run_test (testId: "t_r8n2")
testsy · wait_run (runId: "j_9d3k1")
final · passed · both checks
Fixed. "Project title survives a reload" now passes.
The saved project title reverted to Untitled.
Web test in Chrome
Testsy test "Project title survives a reload" did not pass on launchpad.example. Failed check: The saved project title matches after reload What the tester saw: The project title became "Untitled" after reload Values read: saved title = Launch notes; title after reload = Untitled Steps: Open launchpad.example → Create a new project → Name it "Launch notes" → Save → Reload → Check the title Proof: step 6 of the replay, app.testsy.dev/o/…/runs/j_8f2k4 Find the cause in the code and fix it. Explain what was wrong.
CONNECT ONCE
One connection. Then your agent tests its own work.
Add Testsy's MCP server to Claude Code, Cursor, Codex or another MCP client and approve it in your browser. When you have a pipeline, the same tests run from CI after every deploy.
- Browser sign-in or a scoped API key; your password is never shared with the agent
- Ready-made packages for Claude Code, Cursor, Codex and Gemini CLI
- Tools to plan, save, run and read tests
- GitHub Actions, GitLab CI or any CI through a one-file CLI
Read the docs: CI, MCP and the API →
name: Testsy
on:
deployment_status:
workflow_dispatch:
jobs:
qa:
if: github.event_name == 'workflow_dispatch' || github.event.deployment_status.state == 'success'
runs-on: ubuntu-latest
steps:
- name: Run Testsy tests
env:
TESTSY_AI_API_KEY: ${{ secrets.TESTSY_AI_API_KEY }}
TESTSY_AI_URL: https://app.testsy.dev
run: |
curl -fsSL "$TESTSY_AI_URL/cli/testsy-ai.mjs" -o testsy-ai.mjs
node testsy-ai.mjs run --project p_workspace --junit testsy-ai-junit.xml
- uses: actions/upload-artifact@v4
if: always()
with: { name: testsy-ai-junit, path: testsy-ai-junit.xml }2 passed, 1 failed · exit code 1
claude mcp add --transport http testsy-ai https://app.testsy.dev/mcp --header "Authorization: Bearer $TESTSY_AI_API_KEY"
⎿ 5 steps · 3 checks · sign-in account "customer" · no questions
⎿ t_k3m9 · saved with the approved checklist
⎿ j_c91m7 started · https://app.testsy.dev/o/launchpad/runs/j_c91m7
⎿ final · passed · "All 3 checks passed." · title after reload "Release notes" · 2 minutes 18 seconds
PRICING
Pay per run. No seats, no minutes.
One run is one test on one platform. Web, Android and iOS share the same allowance. Every run stops at 20 minutes and never costs more than one run.
Free trial
Fits your numbersTry it on your own site or app.
- Runs10, once
- Web runs at once1
- App runs at once1
- Projects1
- Extra 50 runs—
Starter
Fits your numbers$0.49 a run
- Runs100 / month
- Web runs at once5
- App runs at once1
- Projects2
- Extra 50 runs$30
Team
Fits your numbers$0.30 a run
- Runs500 / month
- Web runs at once10
- App runs at once3
- Projects5
- Extra 50 runs$20
Growth
Fits your numbers$0.25 a run
- Runs1,200 / month
- Web runs at once20
- App runs at once10
- Projects15
- Extra 50 runs$15
Need more than Growth?
More than 1,200 runs a month, more runs at the same time, invoicing, a security review or a dedicated region — we set it up for you.
Questions
What counts as a run?
One test executed on one platform. Running the same test on the web and on Android is two runs. Web, Android and iOS runs come out of the same allowance. A run the site blocked is not billed.
What happens if a run takes long?
Every run stops at 20 minutes. There are no duration extensions and no charges for time: a run is one run however long it takes.
Do I need to write code or selectors?
No. You write what should happen in plain English and review the steps and checks Testsy drafts. Developers can still drive everything from CI, the CLI, the REST API or MCP.
How is this different from other AI testing tools?
Every verdict shows the value it was decided on, where it was read and on which step, with the screenshot and the moment in the replay. And a site that blocks the test is reported as blocked, not as a bug in your app.
Can it test pages behind a sign-in?
Yes. Add a sign-in account to the project once and name it in the test. The password never goes in the test text, and never appears in results.
How do I test my mobile app?
Upload an Android .apk or a zipped iOS .app from the dashboard, the CLI or CI, then write the test against that build. Choose iPhone or iPad, an Android phone or tablet, and the OS version: iOS 27, 26, 18 or 17; Android 16 to 10. A new test runs on a phone with the latest version, held upright; it can start on its side instead, and a test can turn the device during the run. Every run starts on a device that has been fully reset.
What does the free trial include?
10 runs, once, on one project with one run at a time. Web, Android and iOS are all included. No card needed.
Don't ship on “it should work now”.
Connect your agent, pick the flow that would hurt most if it broke, and let it show you the proof.