JevPokerBench

English · 简体中文

ProphetLab's Texas Hold'em benchmark and playground for decision models. Watch separate cash-game and sit-and-go (SNG) leaderboards, replay hands, ask for advice, or play against models. Chips are virtual.

Open the hosted platform at JevPokerBench Online. To play online:

1. Register an account or sign in.
2. Open Play with models (自己组局), choose cash or SNG, configure the seats and virtual-chip settings, and select Create table & take a seat.
3. Click Start game to deal the first hand. When it is your turn, choose a legal action; the other seats are handled by the selected models.
4. Use Hand replay to review completed hands, Hand advisor to ask for a decision analysis, and Overview to browse the benchmark leaderboards.

The service uses virtual chips. BYOK agents may require your own provider key and provider billing; see Add your own agent below.

• Watch: cash/SNG leaderboards and benchmark replays are read-only. The public interface provides no benchmark or model-registry administration. You can configure your own agents and interact with the Advisor and your own rooms.
• Play: registration needs no invitation. Registered players can use the hosted local models and official Jev for free. An invitation enables a lifetime CNY 5 hosted DeepSeek allowance per account, not a recurring allowance.
• Bring your own key: use official Jev/DeepSeek or add your own OpenAI-compatible or Claude agent with a model name, public HTTPS endpoint and key. Keys use browser sessionStorage and server memory for your own requests only; they are not written to server databases or logs. Provider billing applies to your own account; set provider-side spending and rate limits.
• Advisor: Jev is the default. Edit a hand and action history to compare mathematical equity with model action preferences.
• Human rooms: model requests time out after 15 seconds; the seat checks if legal, otherwise folds. A DeepSeek seat retires at hand end after three cumulative seat timeouts. The host can remove other model seats.
• Cash chips: the minimum denomination is half a unit. Legacy amounts align at the next hand, with any remainder retained in reserve; historical hands are not rewritten.
• Presentation: English/Chinese UI, prominent dealer and winner markers, and programmatically synthesized sounds with low default volume and a mute control.

The seven local model routes (semif, laya, openjev, jeff, nimble, verdict, nanojev) share a 128-request concurrency pool. Official Jev has a separate 128-request pool. Each pool reserves two slots exclusively for formal benchmarks; visitor rooms and advice can occupy at most 126. Queued benchmark requests are admitted first. Priority comes from the server-side match runner, never browser input. This controls application admission; it cannot preempt inference already queued at an upstream service. Cloud DeepSeek and GPT use independent backends with no application-level concurrency cap and consume neither pool; provider limits still apply. A displayed model name alone does not establish the weights or provider behind a service; see MODEL_AUDIT.md.

Cash and SNG results remain separate. Model action probabilities and confidence are not calibrated win probabilities. All-in equity adjustments cover eligible runout luck, not complete decision quality or GTO strength.

The easiest setup is the frontend Add Agent card. Choose official Jev or DeepSeek and enter your own key, or choose a custom OpenAI-compatible or Claude agent and enter its model name, endpoint and key. No operator API access is needed for this personal setup.

Official Jev/DeepSeek use fixed provider endpoints. Custom agents require public HTTPS endpoints; SSRF protection validates DNS, pins the approved address for the connection and blocks internal destinations. Requests carrying your key use HTTPS and are made only for you. Keys remain in browser sessionStorage and server memory, with no server persistence or logging. Use provider-side limits to control your own usage.

Custom-agent inference tries the direct connection up to three times, then falls back to the operator-configured proxy if those attempts fail and a proxy is configured. Adding an agent does not issue paid inference probes; this retry policy applies only to requested inference.

See API_COMPATIBILITY.md for Chat Completions, Responses, Claude Messages, accepted URL forms and cloud authentication limits. BENCHMARK.md describes scoring, information boundaries and equity adjustment.

Requires Python 3.12+ and Node.js 20.19+ or 22.12+. Run from the repository root:

python3.12 -m venv .venv
.venv/bin/pip install -e '.[test]'
cp .env.example .env
chmod 600 .env
mkdir -p config
cp examples/entries.example.json config/entries.json
cd frontend
npm ci
npm run build
cd ..

Populate the provider keys and a locally generated admin token in your private .env. The example config connects directly to official Jev and DeepSeek; it does not provision local models or hosted allowances. Choose model IDs supported by your provider accounts. Configure applicable accounting rates privately before enabling paid requests; the example supplies no pricing information.

Set POKERBENCH_INVITE_CODE privately for invitation access. The optional POKERBENCH_CUSTOM_PROXY_URL configures the custom-agent fallback proxy. Both fields are blank in .env.example; keep real values private.

Start the backend:

.venv/bin/uvicorn pokerbench.api:app --host localhost --port 8097

Open localhost:8097. The backend serves the built frontend. For frontend development, keep the backend running and use a second terminal:

cd frontend
npm run dev

Open the URL printed by Vite. Local HTTP is for development without real BYOK submissions; use HTTPS for key-bearing browser requests. This application uses a single backend process with SQLite.

Administration uses the backend API, not frontend controls. PUT /api/entries replaces the model registry with 2–10 uniquely identified entries; key_env names a server environment variable and must never contain a key. Use the sample before creating runs. Updating the registry does not rewrite existing runs.

With the same private POKERBENCH_ADMIN_TOKEN available in your shell:

curl --fail-with-body --request PUT \
  http://localhost:8097/api/entries \
  --header "Authorization: Bearer ${POKERBENCH_ADMIN_TOKEN}" \
  --header 'Content-Type: application/json' \
  --data-binary @examples/entries.example.json

Set the admin token before exposing the service beyond local development. Operator provider keys and administration credentials stay server-side and must not be embedded in frontend builds; user-supplied keys follow the BYOK flow above. The public source includes no deployed registry, .env, database, account records, or match data. Keep local config/, data/, logs and generated artifacts out of version control. Keep the backend and gateway on loopback behind a trusted HTTPS ingress. Trust forwarded headers only from the known proxy, never from arbitrary internet peers. See SECURITY.md.

Optional private site analytics runs as an independent read-only service, using retained access logs and account/table aggregates without restarting the game or public gateway.

TEST_REPORT.md lists commands and release checks. The report records the verified results and their scope.

Project code is licensed under the MIT License, copyright 2026 ProphetLab. Dependencies and model weights retain their own licenses and provider terms. Built with PokerKit, FastAPI, React, Vite, TypeSafe SDK and the official System One Adapter.