AI agents (MCP)
Connect Claude Code, Cursor, Codex or another MCP client to a Testsy workspace with OAuth or an API key. Draft release checks, read evidence and rerun after fixes.
Choose a connection#
Testsy's remote MCP server is at https://app.testsy.dev/mcp. It uses Streamable HTTP with JSON answers and supports MCP protocol versions 2025-11-25, 2025-06-18, 2025-03-26 and 2024-11-05.
Use browser authorization when your MCP client supports Testsy's OAuth discovery with dynamic client registration or a Client ID Metadata Document. API keys remain available for direct client setup, CI and clients without that authorization flow. Both connect to one Testsy workspace; runs use its allowance and can perform the actions in the test.
Connecting and listing the tools needs no sign-in: a client, or anyone choosing an MCP server, can call initialize and tools/list without a credential to see what Testsy offers. Every tool call needs a connection. Without one, the server answers 401 Unauthorized with a WWW-Authenticate challenge that points to the OAuth metadata, so a client with browser authorization signs in at that moment and retries the call. A key or token that is sent but invalid or revoked is refused on every request.
Choose the least access your workflow needs. Reading results needs read access; running saved tests needs run access; drafting, creating or editing tests needs write access. Your workspace role still applies to OAuth operations, including changes reserved for administrators.
Connect with browser authorization#
In a compatible MCP client, add the remote server address https://app.testsy.dev/mcp and start its authorization flow. The client discovers Testsy's authorization server and registers its callback. It must support authorization code flow with S256 PKCE and the Testsy MCP resource audience.
Testsy opens its Connect to Testsy screen at https://app.testsy.dev/connect. Sign in, review the client name and callback, choose the workspace and check the requested permissions before approving. The client name is self-registered and unverified; it is not evidence of a trusted publisher. Declining leaves the client without access.
| Permission | Allows |
|---|---|
testsy:read | Read workspace projects, tests and results. |
testsy:run | Read, run and stop saved tests, upload app builds and create report links. |
testsy:write | Read, run, draft, create, edit and delete tests and other permitted workspace resources. |
A client with insufficient permission must request a new authorization with the needed scope; a refresh cannot add permissions. Remove a connection in Account settings → Connected apps. Revocation, membership removal, relevant role changes and credential changes are checked on future use. If a connection expires, authorize it again.
For client implementers, authorization-server metadata is at https://app.testsy.dev/.well-known/oauth-authorization-server; protected-resource metadata is at https://app.testsy.dev/.well-known/oauth-protected-resource/mcp. Metadata advertises dynamic client registration at /oauth/register, authorization at /oauth/authorize, token exchange at /oauth/token and revocation at /oauth/revoke. Use the discovered addresses and exact registered callback. A client can also identify itself with a Client ID Metadata Document instead of registering: its client_id is the https address of its JSON document (a public client; exact https redirects, and loopback redirects matched without their port).
Connect with an API key#
Create a key in Settings → API keys. Run tests and read results is sufficient for saved-test execution and investigation. Choose the option that can also create and edit tests when the agent will draft and save new tests.
Store the key securely as TESTSY_AI_API_KEY, then use your client's setup:
claude mcp add --transport http testsy https://app.testsy.dev/mcp --header "Authorization: Bearer $TESTSY_AI_API_KEY"codex mcp add testsy --url https://app.testsy.dev/mcp --bearer-token-env-var TESTSY_AI_API_KEY{
"mcpServers": {
"testsy": {
"url": "https://app.testsy.dev/mcp",
"headers": { "Authorization": "Bearer qa_…" }
}
}
}The generic JSON shows the bearer-header shape; replace the example using your client's supported secret store or environment mechanism. Do not commit a real key, paste it into a prompt or share it in a screenshot. Review how your client stores command-line headers before using the Claude Code command. The MCP server accepts the key as a bearer token; it does not need your account password.
Installation packages and directory status#
Testsy's integration package prepares the same three workflows for Codex, Claude Code, Cursor, Gemini CLI and portable Agent Skills hosts: testsy-check-release, testsy-investigate-run and testsy-regression-from-bug. Skills guide the agent to preserve expected behavior, inspect evidence and distinguish a failure from an incomplete test.
The packages are published as public repositories (MIT) and can be installed by address today:
- Claude Code:
/plugin marketplace add testsy-ai/testsy-claude-plugins, then/plugin install testsy@testsy. The plugin brings the MCP connection and the three skills. There is no key to set: the first time Claude calls a Testsy tool you sign in in the browser and choose the workspace (/mcpshows the connection). For CI or another headless session, add the connection with an API key instead (the Claude Code tab above). - Claude desktop app, Cowork and claude.ai: add
testsy-ai/testsy-claude-pluginsas a marketplace under Customize and install Testsy from it, or addhttps://app.testsy.dev/mcpas a custom connector under Customize → Connectors and sign in. The three skills can also be uploaded one by one under Customize → Skills from the skill downloads. - Gemini CLI:
gemini extensions install https://github.com/testsy-ai/testsy-gemini-extension; supply the API key setting when asked. - Cursor: testsy-ai/testsy-cursor holds the plugin (skills +
mcp.json). Or add the MCP server from the tab above and copy theskills/folders into your project's.cursor/skills/. - Codex:
codex mcp add testsy --url https://app.testsy.dev/mcp --bearer-token-env-var TESTSY_AI_API_KEY; the skills are in testsy-ai/testsy-integrations underskills/. Codex asks before it starts a run or changes a test. For unattended use (codex execwith approvals off), adddefault_tools_approval_mode = "approve"under[mcp_servers.testsy]in~/.codex/config.toml; without it Codex answers "MCP tool call requires approval". - MCP Registry: listed as
dev.testsy/testsy(remote server, bearer header). Clients that read the registry can add Testsy from there. - GitHub Action:
uses: testsy-ai/testsy-action@v1— see CI.
Source and downloads: testsy-ai/testsy-integrations and integration downloads. Directory listings (Cursor, Claude, ChatGPT, GitHub Marketplace) are submitted separately; this documentation does not imply that a directory has approved a listing. Use the direct MCP setup above when a supported public package installation is not available.
Check a SaaS release#
Open an editable release smoke test, or give the connected agent a specific journey, such as creating a synthetic task and checking that its saved title survives reload. The Claude Code walkthrough covers the complete workflow.
list_projects, thenplan_testwith what to test: the assistant drafts the steps and checks.- Review the plan. To revise it, call
plan_testagain with the samedescription, theplanand achange; answer its questions withanswers. Describe the test in any language: the plan and the results come back in the test's language; evidence such as page text and screenshots stays as it is. - Save the confirmed plan with
create_test, then start it withrun_test. Give the requester the returnedwatchUrl: it opens the run live, and its result afterwards, with no sign-in (watch links). - Call
wait_rununtilfinalis true — one call waits up to 25 seconds (passseconds, at most 55) and returns the run the moment it finishes, so there is no need to pollget_runevery few seconds — then inspect each check's result, reason and evidence. - Deploy a supported fix and rerun the same test. Preserve the original result. Use
create_share_linkonly after reviewing what you will share.
Tests can also get files (add_test_file), sign-in accounts (add_credential), endpoints into your system (add_endpoint) and schedules (create_schedule). Runs may create accounts, change records or call allowed endpoints: review the plan and use authorized synthetic data.
add_credential pass through your agent. They are encrypted when stored and do not reach the tester's AI, but prefer adding passwords in the web app.Upload a new app build and run it#
An agent can upload an Android .apk or a zipped iOS Simulator .app, then run existing tests against it. Run permission allows build uploads. Check which builds are accepted.
A build never travels through tool arguments. The agent asks for signed links and hands the bytes to a shell command:
find_build {projectId, sha256?, fingerprint?}: if the project already holds this exact file (its sha256) or this native build (a fingerprint stored with an earlier upload), it returns that build; skip to step 4. When the build sits behind an https link (an EAS, Bitrise, S3 or GitHub artifact link),import_build {projectId, url, headers?}fetches and checks it server-side with nothing to upload; skip to step 4.create_build_upload {projectId, fileName, size, label?, fingerprint?}returns{id, partBytes, parts, expiresAt, uploadUrl, partUrlTemplate, completeUrl, command}.sizeis the file size in bytes.- Run
commandin a shell: it downloads the command-line tool and uploads the file to the links, then completes the upload and prints the build id. The links are valid for a few hours and only for this upload; no API key or token is needed in the shell. The shell needs curl and Node 18 or later. Run the command in the folder that holds the file (it uses thefileNameyou gave), and give the file's exact size: a different size is refused. Zip an iOS.appbefore you ask for the links. Without the tool,PUTthe raw bytes of eachpartBytes-sized part topartUrlTemplate(with{partNumber}from 1), thenPOST {parts: [{partNumber, etag}]}tocompleteUrl, or callcomplete_build_upload {buildId, parts}. An unsupported build is refused with the reason. run_test {testId, buildId}runs one test, orrun_project {projectId, buildIds: [buildId]}runs the project's tests with this build for the matching app. Wait for each returned run withwait_run.- To run app tests on another phone, tablet or OS version this time, add
device(phone or tablet),osVersion(latest,"iOS 26","Android 14") and/ororientation(default,portrait,landscape: how the screen is turned at the start) torun_testorrun_project;list_deviceslists them. The test keeps its own setting;create_testandupdate_testchange it. The run'sdevicesays what it ran on.
The server receives the bytes from the shell; it does not read a path on your computer. Uploading alone does not start a run. Build overrides are optional: omit buildId / buildIds to use each test's configured build selection.
Every tool#
Workspace
| Tool | What it does |
|---|---|
get_workspace | The workspace this key belongs to: its id, name and plan, and this period's usage. no arguments |
get_usage | Plan, runs used this billing period, included runs, extra-run packs and runs left. no arguments |
get_network | How our browsers reach your sites: the static addresses to allow-list, the country they browse from, and the countries available. no arguments |
set_browse_from changes | Choose the country our browsers and devices reach your sites and app servers from (web and app tests), or automatic: the site's own country (an app test then uses our default country).country |
Projects
| Tool | What it does |
|---|---|
list_projects | The projects in this workspace (a project groups the tests of one site or app). no arguments |
get_project | One project.projectId |
create_project changes | Create a project, for example one per site or app.name description? |
update_project changes | Rename a project or change its description.projectId name? description? |
delete_project changes | Archive a project: its tests stop running; past runs stay in the run history.projectId |
Tests
| Tool | What it does |
|---|---|
list_tests | The tests of a project, each with its latest run.projectId |
get_test | One test: its start address, requirement, plan, options and latest run.testId |
create_test changes | Create a test of a website (url) or of an app (platform android or ios, and appId; device phone or tablet and osVersion, default a phone on the latest OS).projectId title url? requirement? plan? platform? appId? buildId? device? osVersion? orientation? credentials? options? |
update_test changes | Change a test: its title, start address, requirement, plan or options.testId title? url? requirement? plan? device? osVersion? orientation? options? |
duplicate_test changes | Copy a test under a new title: the test's info is copied over (start address, requirement, plan, options and files). Its run history and schedules stay with the original.testId title? projectId? |
delete_test changes | Archive a test: it stops running; past runs stay in the run history.testId |
Test plans
| Tool | What it does |
|---|---|
plan_test changes | Have the assistant draft a test from a plain-language description, or revise a plan, without saving anything.description url? projectId? platform? appId? plan? change? answers? |
Test files
| Tool | What it does |
|---|---|
list_test_files | Files a test can upload during its steps (for example an import file).testId |
add_test_file changes | Add a file the tester can upload during the test: up to 25 MB, up to 10 files per test.testId name contentBase64? contentType? |
delete_test_file changes | Remove one of a test's files.testId name |
Sign-in accounts
| Tool | What it does |
|---|---|
list_credentials | The sign-in accounts stored for a project: names, usernames and sites. Passwords are never returned.projectId |
add_credential changes | Store a sign-in account the tester can use. The password is encrypted, never returned, and never sent to the tester's AI: the tester types it.projectId name username password site? |
delete_credential changes | Delete a sign-in account. Tests that sign in with it stop running until it is added again or they are edited.credentialId |
Endpoints
| Tool | What it does |
|---|---|
list_endpoints | Calls into your own systems the tester may make during a run (for example "create a voucher"). Headers are never returned.projectId |
add_endpoint changes | Save the definition of one call into your own system that the tester may make during a run (for example "create a voucher"). This only stores the definition: it sends no request itself. The tester sees only the name, the description and what the call returned.projectId name description method? url headers? bodyTemplate? |
delete_endpoint changes | Delete an endpoint.endpointId |
Runs
| Tool | What it does |
|---|---|
run_test runs | Start a run of one test. Runs take minutes: poll get_run until final is true. watchUrl opens the run live without signing in.testId mode? buildId? device? osVersion? orientation? url? |
run_project runs | Start a run of every test in a project (a suite). Each started run has a watchUrl to watch it live without signing in.projectId mode? buildIds? buildId? device? osVersion? orientation? url? |
list_runs | Recent runs, newest first, optionally for one test or project or with one status; or the runs with the given ids.ids? testId? projectId? status? limit? |
get_run | A run: its status, and once final the verdict, the headline, each check's result with the reason, the values read, and links to the video, screenshots and the result page.runId |
wait_run | Wait up to seconds (default 25, at most 55) for a run to finish, then return it: one call instead of polling get_run every few seconds. Call it again while final is false.runId seconds? |
stop_run runs | Stop a queued or running run. A run you stop is not counted.runId |
Share links
| Tool | What it does |
|---|---|
create_watch_link runs | A view-only link to watch a run live and read its result, with no sign-in, valid while it runs and for some hours after. Runs started here get one automatically; this makes another, or one for a run started from the web app.runId |
revoke_watch_link runs | Turn off every watch link of a run straight away.runId |
list_share_links | The share links made for a run, with when they expire and how often they were opened.runId |
create_share_link runs | A read-only link to a run's result, replay and video that anyone can open without signing in, until it expires.runId days? |
delete_share_link runs | Turn off a share link straight away.shareId |
Schedules
| Tool | What it does |
|---|---|
list_schedules | When tests run by themselves: every schedule in the workspace, or those of one test or project.testId? projectId? |
create_schedule changes | Run a test by itself: on chosen weekdays, or monthly on a day, at a time in a time zone. Each scheduled run counts like any other run.testId days? dayOfMonth? time timezone? |
update_schedule changes | Pause or resume a schedule.scheduleId enabled |
delete_schedule changes | Delete a schedule.scheduleId |
App builds
| Tool | What it does |
|---|---|
list_devices | The phones and tablets app tests can run on, and their OS versions (latest first). Pass device and osVersion to create_test, update_test, run_test or run_project. no arguments |
list_builds | The app builds uploaded to a project (Android .apk, iOS Simulator .app zipped), newest first, with the largest size and part size for uploads.projectId |
get_build | One build: its app id, version, status and any warnings.buildId |
find_build | Is this build already uploaded? Looks up the project's newest ready build by the file's sha256, else by a native fingerprint; build is null when neither matches.projectId sha256? fingerprint? |
create_build_upload runs | Start a build upload and get signed links that upload the file without an API key: run the returned command in a shell, or PUT each part and POST complete to the links. The build is then checked and ready to run.projectId fileName size label? fingerprint? |
import_build runs | Fetch a build from an https link (an EAS, Bitrise, S3 or GitHub artifact link, up to the build size limit) into the project; it is checked and ready to run. Nothing to upload from your side.projectId url headers? label? fingerprint? |
complete_build_upload runs | Finish the upload with every part's etag. The build is checked (app id, version, whether our devices can run it) and returned.buildId parts |
delete_build changes | Delete a build. Runs that used it keep their results; a build a test pins cannot be deleted.buildId |
Each tool is also a REST endpoint with the same parameters: see the API reference. For connection problems or an incomplete result, see Help and support.