Agent shopping sessions, observed.
UCP Playground runs real AI models against real UCP-verified stores and logs every JSON-RPC tool call. The aggregate across 26 models from 8 vendors and 531 stores tells us what works in agentic shopping — and what doesn't.
Pick a model. Pick a UCP-verified store. The agent connects to the store's MCP endpoint, browses, carts, and tries to check out — using only the tools the store advertises in its /.well-known/ucp profile.
Every JSON-RPC request and response, every retry, every failed schema match — captured per session. Postman for agentic commerce: inspect what an agent actually did, not what it claimed to do.
Models from Anthropic, Google, OpenAI, Meta, DeepSeek, xAI, Alibaba, Moonshot run side-by-side against the same stores. Vendors can't credibly benchmark themselves; the platform layer has the same problem one level down.
Roughly 1 in 7 sessions. Fast, decisive tool-call rhythm — not deliberation — is what wins at agentic shopping.
DeepSeek R1, QwQ, Grok 3 Mini consistently burn tokens on chain-of-thought and miss the next tool call. Agentic shopping rewards speed, not introspection.
A well-typed profile and tight tool responses move the conversion needle more than swapping models. Most "the agent is broken" tickets are actually store-side.
| Model | Avg tokens | Avg duration | Session share |
|---|---|---|---|
| Gemini 3.6 Flash | 74,954 | 22.2s | 14.1% |
| Gemini 3 Flash | 76,507 | 22.5s | 13.2% |
| Claude Sonnet 4.5 | 69,836 | 33.8s | 12% |
| Claude Opus 4.6 | 61,037 | 30.5s | 7.9% |
| Gemini 2.5 Flash | 44,976 | 13.9s | 6.8% |
| GPT-5.2 | 52,119 | 33.2s | 6.7% |
| Gemini 3.1 Pro | 61,504 | 42.4s | 5.3% |
| GPT-4o | 34,077 | 16.4s | 5.1% |
| Gemini 2.5 Pro | 47,160 | 37.4s | 5.1% |
| DeepSeek v3.2 | 68,697 | 47.2s | 3.1% |
| o4-mini | 70,285 | 39.6s | 2.6% |
| Grok 4 | 35,601 | 45.4s | 2.6% |
| Claude Sonnet 5 | 261,470 | 47.2s | 2.5% |
| Llama 3.3 70B | 58,192 | 40.4s | 2.4% |
| DeepSeek R1 | 23,023 | 53.7s | 1.7% |
| DeepSeek v4 | 115,018 | 67.1s | 1.6% |
| Claude Opus 5 | 194,451 | 54.1s | 1.6% |
| Grok 3 Mini | 35,632 | 40.6s | 1% |
| QwQ 32B | 14,586 | 36.7s | 0.9% |
| Llama 4 Maverick | 43,535 | 15.1s | 0.9% |
| DeepSeek v4 Flash | 124,257 | 49.9s | 0.7% |
| Qwen3.7 Max | 72,412 | 27.4s | 0.7% |
| Grok 4.5 | 110,646 | 41.1s | 0.7% |
| Kimi K3 | 77,433 | 84.3s | 0.6% |
| Gemini 3.8 Flash | 120,674 | 60.7s | 0.2% |
| GPT-6 Luna | 103,889 | 21s | 0.1% |
| Tool | Call share | Avg latency | Error rate |
|---|---|---|---|
| search_catalog | 35.1% | 725ms | 21.9% |
| update_cart | 20.1% | 609ms | 32.5% |
| search_shop_catalog | 15.5% | 483ms | 26.9% |
| search_global_products | 7% | 347ms | 0% |
| get_product_details | 5.8% | 266ms | 25.1% |
| get_cart | 3.9% | 279ms | 43% |
| create_cart | 3.4% | 916ms | 36% |
| POST /catalog/search | 3.4% | 4,543ms | 37.4% |
| search_products | 3.1% | 1,229ms | 14.3% |
| get_product | 2.7% | 1,043ms | 35.4% |
Common questions
What is UCP Playground?
How is UCP Playground different from UCP Checker?
Is UCP Playground a UCP demo?
demo-travel.ucp.dev, sandbox subdomains) so you can watch UCP in action without affecting production inventory.Which AI model performs best at agentic shopping?
Why do reasoning-tuned models underperform on shopping tasks?
How is the checkout completion rate calculated?
How many stores has UCP Playground tested?
Does my store implementation matter more than the model I pick?
Is UCP Playground free to use?
How do I run a UCP test on my own store?
/.well-known/ucp profile and reachable MCP or REST endpoints, agent sessions will run against it directly. Stores that grade well on UCP Score typically also pass UCP tests in Playground, but a Playground test surfaces runtime issues that static checks cannot — schema drift, slow tool responses, partial cart state, missing variant data.Run your own agent session
Pick a model, pick a store, watch the agent shop. Every JSON-RPC tool call is logged and every session is replayable — so you can see what an agent actually did, not just what it claimed.
Open UCP PlaygroundBuilding your own agent? Start at /agents · Reading the spec? /protocol
Get the agentic commerce digest every Monday
Real adoption data, ecosystem trends, new spec versions, and the stores that broke or recovered this week. Read by founders and engineers building the next generation of commerce.