LiveBench hosted-evaluation request

Recipient: livebenchai@gmail.com (the address published in LiveBench's
contribution instructions)

Subject: Hosted LiveBench 2026-06-25 evaluation request for four Ruzzler modes

Message

Hello LiveBench team,

We would like to request a hosted evaluation of four provider-neutral Ruzzler
routing modes on the exact LiveBench 2026-06-25 release:

• ruzzler-auto
• ruzzler-bolt
• ruzzler-power
• ruzzler-mosaic

Ruzzler exposes an OpenAI-compatible Chat Completions endpoint:

https://www.ruzzler.com/v1/chat/completions

Each mode is selected by sending its identifier in the request's model
field. We will provide a dedicated revocable benchmark API key through the
secure channel you prefer. The key will be restricted by a hard spend budget
and will be revoked after evaluation.

To avoid spending on near-ceiling tasks, please run only the preregistered
LiveBench Hard-7 slice: logic_with_navigation, code_generation,
typescript, integrals_with_game, tablejoin, plot_unscrambling, and
simplify. Please use the exact same task IDs, prompts, tool contract, maximum
output, temperature, timeouts, and scoring implementation used for the
published 2026-06-25 leaderboard. Please retain endpoint failures as
availability outcomes rather than silently substituting models or prompts.

For reproducibility, could you return or publish:

1. the LiveBench release and harness commit identity;
2. exact parameter settings and task counts;
3. per-subtask scores and aggregate scores;
4. question-level status without private prompt or ground-truth content;
5. request counts, token counts, latency, and reported endpoint cost when
   available;
6. any exclusions, retries, or infrastructure failures.

We will publish the four Ruzzler rows beside the already-published results for
the preregistered comparison roster. We will not claim universal superiority
from this single benchmark.

Thank you.

Send gate

Do not attach or transmit an API key until its exact key/project/tenant budget
is configured and independently verified. Do not send this request without
the account owner's approval for external communication.