KeyPulse

Collecting free AI inference keys is easy. Groq, Cerebras, SambaNova, Together, Mistral, Gemini, OpenRouter, and a new one every other week on Hacker News. Living with them is the hard part.

Free tiers are unstable by design. A key that answered in 300 ms this morning is rate-limited by lunch, silently degraded by dinner, and revoked on Friday. So you end up doing the same three things by hand, forever:

• Switching keys. Comment out one base_url, paste a different key, restart the agent, hope. Repeat when that one dies.
• Guessing. Which provider is actually fastest for you, on your connection, at this hour? Nobody's public benchmark answers that. Yours would, if you had one.
• Finding out too late. A new provider launches with a generous free tier and you hear about it three weeks after the thread scrolled off the front page.

KeyPulse turns all three into background processes.

It probes every key on a schedule with one small chat completion, and from that single call records time-to-first-token, total latency, tokens per second, and rolling jitter. It scores each key against the others. Then it exposes http://127.0.0.1:8787/v1 — plain OpenAI API — and sends each request to the best candidate, moving to the next one on a 429, an auth failure, a timeout or a 5xx, with exponential backoff that honours Retry-After. Your agent, your editor plugin, your script: one base URL, changed once, and never again.

A second tab watches GitHub, Hacker News and any RSS feed you pin for new free and trial inference offers, so the next Groq shows up as a card with an Add to my providers button instead of a thread you missed.

Nothing leaves your machine. Keys are sealed with AES-256-GCM under a master key in your OS keychain. State is a SQLite file in your app data directory. There is no KeyPulse account, no hosted control plane, and no telemetry — the only network calls are to the providers you added and the discovery sources you enabled.

http://127.0.0.1:8787/v1 speaks the OpenAI API, so any SDK, CLI or editor plugin can point at it unchanged.

curl http://127.0.0.1:8787/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"keypulse-auto","messages":[{"role":"user","content":"hi"}]}'

keypulse-auto (or auto, or an empty model) means you choose — KeyPulse picks per the current strategy and each key's own default model. Name a model explicitly and routing narrows to the keys that list it. Non-OpenAI providers are translated in both directions, streaming included: a chat stream is re-encoded frame by frame, not buffered.

A background scheduler probes every enabled key on your interval with one small chat completion. From that single call it records time-to-first-token, total latency, tokens/sec, and rolling jitter (standard deviation over the recent window) — deliberately not a large-payload download test, which measures your network rather than the provider.

Failures are classified as rate limit, auth, timeout, server, network, bad request or bad response. A 429 puts the key in cooldown so the gateway stops choosing it. Quota comes from the provider's rate-limit headers when it sends them and from figures you enter when it doesn't, with a time-to-exhaustion estimate based on your observed usage.

Best composite score · lowest latency · fastest TTFT · highest throughput · lowest error rate · round robin · weighted round robin · strict priority · random.

The composite score weights success rate .4, latency .3, throughput .2 and jitter .1 by default — all four adjustable. Whatever you pick, Settings shows the exact candidate order the strategy produces right now, before a single request depends on it.

The Benchmark card is separate and manual. Run on every key, or the gauge button on one row, fires several requests at that key on demand — the same small prompt, repeated. Use it to answer "which of these is fastest for me?" without waiting for the probe schedule.

Discovery runs on a schedule or on demand across:

• GitHub — the documented search API, over rotating queries about free LLM API lists.
• Hacker News — the Algolia search API.
• Pinned sources — RSS/Atom feeds, GitHub markdown lists and plain pages you add. Five community feeds are seeded on first run and can be deleted.
• Web search — optional, only if you store a Brave or Tavily key.

Findings pass a relevance gate (it has to look like an AI/LLM API and like a free or trial offer) and become cards: name, website, docs, base URL, whether it looks OpenAI-compatible, free-tier details, pricing notes, models, context length, capabilities, first-seen, last-verified, source link. Cards are de-duplicated against each other and against the keys you already have. Each one has a one-click Add to my providers → that opens the Tab 1 dialog pre-filled — you only paste the key.

On politeness: only official APIs, public feeds and documented endpoints. A pinned page is fetched only after its host's robots.txt says a generic agent may; if it says no, the source is reported as skipped rather than worked around. One source failing never fails a run. Findings are leads, not facts — verify a free tier on the provider's own pricing page before you rely on it.

There are excellent tools in this space. KeyPulse overlaps with them and is not trying to replace all of them.

LiteLLM is the more complete production gateway — more providers, more routing knobs, team/spend management, Kubernetes-native. If you are putting a gateway in front of a product, use LiteLLM. Helicone is an observability platform first: it is the better answer to "what did my app send last Tuesday." OpenRouter is the least work of all — one key, hundreds of models, someone else's problem.

KeyPulse is for the case none of them target: you, on your own laptop, with a drawer full of free-tier keys and no infrastructure. Three things follow from that.

1. Zero infrastructure. No Docker, no Postgres, no Redis, no YAML, no account. Install the app, paste a key, done. State is one SQLite file.

2. Your keys never leave the machine. They are sealed under your OS keychain and used only for calls you initiated. There is no vendor in the request path because there is no vendor.

3. It tells you about providers you don't have yet. Every other tool manages the keys you already own. Provider Radar is how you get the next one.

And because the probes run from your connection, the numbers are about your actual situation — not a status page in another region.

Checked against the linked official docs on 2026-07-30. Corrections by PR are welcome; if something below is wrong or out of date, open an issue and it gets fixed.

— means we could not find it documented on the official pages linked in the header row. It may well exist; absence here is absence of evidence, not a claim about the product.

One distinction the table can't show: OpenRouter's latency and throughput percentiles are measured across its own fleet, and LiteLLM's health checks run from wherever you deployed it. KeyPulse's numbers are measured from the machine you are sitting at, per key, which is the only vantage point that answers "will this be fast for me right now."

Under two minutes from clone to a running gateway.

• Rust 1.82 or newer (rustup recommended)
• Node.js 20.19+ or 22.12+ and npm
• Platform build dependencies for Tauri v2:
  • Windows — WebView2 runtime (preinstalled on Windows 11) and the MSVC build tools
  • macOS — Xcode Command Line Tools (xcode-select --install)
  • Linux — webkit2gtk-4.1, libayatana-appindicator3, librsvg2, patchelf (see the Tauri prerequisites for your distribution's exact package names)

git clone https://github.com/Matinj031/KeyPulse.git
cd KeyPulse
npm install
npm run app:dev      # Vite dev server + the Rust core, with hot reload

The first launch creates the database and a master key. On a machine with no usable keychain (a bare Linux container, say) KeyPulse falls back to a file-backed master key with restricted permissions and tells you so in Settings.

Prebuilt installers are not published yet — npm run app:build produces them into src-tauri/target/release/bundle/ if you want one now. Watch releases for the first tagged build.

1. Add key, top right of Tab 1.
2. Name it something you'll recognise — Groq — personal, not key 1. Several keys may share a name; they are still scored separately.
3. Base URL: the API root, e.g. https://api.groq.com/openai/v1. Pasting the full .../v1/chat/completions also works.
4. Wire format: OpenAI-compatible covers Groq, Together, OpenRouter, Cerebras, SambaNova, DeepInfra, Fireworks, Mistral and most others. Pick Anthropic or Gemini for those two.
5. Authentication: Bearer for almost everything; Custom header for providers that want x-api-key or similar.
6. API key: pasted once, encrypted before it touches disk. On a later edit, leaving the field blank keeps the stored key.
7. Custom headers (optional): Name: value, one per line — sent on probes, model listing and every proxied request for this key. For gateways that want a tenant id, a beta flag, or a particular User-Agent. ⚠️ These are stored and exported in the clear, so the key itself belongs in the field above.
8. Models: one per line, or hit Fetch from provider once the key is saved to pull them from the provider's own /models. The probe model is the one health checks call — pick the cheapest, fastest one.
9. Capabilities: the gateway refuses to route a request a key cannot serve, so tick only what the key actually supports.
10. Save. The first probe runs on the next cycle, or immediately via Probe all now.

Enable the proxy in Settings, then change one line:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key="unused")
client.chat.completions.create(
    model="keypulse-auto",                      # or a specific model id
    messages=[{"role": "user", "content": "hi"}],
)

The listener binds to loopback only, so it is not reachable from the network. That is not sufficient on a shared or multi-user machine — any local process can reach a loopback port — so Require a token in Settings adds a bearer token the client must present. Turn it on if you are not the only user of the box.

src/                        Vue 3 + TypeScript frontend
  views/                    GatewayView · RadarView · SettingsView
  components/               provider table, charts, dialogs, ui/ primitives
  stores/                   Pinia: health · discovery · settings · toast
  lib/                      typed invoke() wrappers, shared types, formatters

src-tauri/src/              Rust core
  commands.rs               the Tauri command surface the frontend calls
  proxy.rs                  local OpenAI-compatible endpoint + failover loop
  health.rs                 probe scheduler, error classification, quota parsing
  discovery.rs              the radar's sources, relevance gate and de-duplication
  scoring.rs                one key's samples → score, status, series
  routing.rs                a health snapshot → an ordered candidate list
  adapters/                 per-provider wire formats (openai · anthropic · gemini)
  db.rs / crypto.rs         SQLite storage and secret sealing
  state.rs                  shared app state, event emission, proxy handle

Data flows one way. The Rust core owns all state; the frontend reads snapshots through invoke and receives pushes on three events (keypulse://health, keypulse://failover, keypulse://discovery), so both tabs stay live without polling.

scoring and routing are pure functions over plain structs — no clock, no network, no database. That is what makes the decision logic testable without mocks, and it is why the routing preview in Settings can show you the real candidate order instead of an approximation of it.

The adapter is the one abstraction worth investing in. Everything above it speaks the OpenAI chat-completions shape; an adapter is the only thing that knows a provider's real wire format. Supporting a new non-OpenAI API means one new file implementing eight small methods, plus a variant in AdapterKind. There is no repository layer, no DI container and no DDD scaffolding — db.rs is a thin typed wrapper over SQL, and the modules that need it take a &Db.

Everything is in Settings inside the app; there is no config file to edit. Values are clamped by the core, so a typo cannot put the app in a bad state.

• API keys are sealed with AES-256-GCM under a master key held in the OS keychain. Where no keychain is available, the master key falls back to a file with restricted permissions and the app says so in Settings.
• Keys are masked to their last four characters everywhere in the UI. The plaintext is decrypted only for a request you asked for, and revealing one is an explicit action.
• Raw keys are never written to logs. Provider error messages sometimes echo a key back; those strings are redacted before they are stored or logged.
• The gateway binds to 127.0.0.1 only. Enable the bearer token if other people or untrusted processes share the machine.
• Custom request headers are stored and exported in the clear — only api_key is encrypted. Put secrets in the key field, not in a header.
• Exported bundles contain no plaintext keys unless you ask for keys and supply a passphrase.

Found something? Open a regular issue for anything non-sensitive. For a suspected vulnerability, please report it privately via GitHub security advisories rather than a public issue.

KeyPulse is built to be a good citizen of the free tiers it watches, and it is your responsibility to keep it that way.

• Probing costs the provider something. Each health check is one small chat completion, but multiplied by your key count and divided by your interval it adds up. The interval has a hard floor of 30 seconds and warns below 60; on a free tier, minutes are usually more appropriate than seconds. Providers with tight daily caps should get a longer interval, and you can stop probing a key by disabling it.
• Read the terms of the keys you add. Some free tiers prohibit sharing a key across users, automated load distribution, or use in a service that resells access. Pointing the local gateway at your own tools is not the same thing as pooling keys for other people, and only you can tell which side of a given provider's terms you are on.
• Rate limits are signals, not obstacles. A 429 puts a key in cooldown; that is the intended behaviour, not something to tune away. KeyPulse never retries a key faster than the Retry-After it was given.
• The radar does not crawl. Documented APIs, public feeds and pages you pinned yourself. It checks robots.txt before fetching a pinned page and reports a source as skipped when it is disallowed. Please don't add sources whose terms forbid automated access.

npm install                     # frontend deps
npm run app:dev                 # the real thing: Rust core + Vue, hot-reloaded
npm run dev                     # frontend only — Tauri APIs are absent, so most of the app is inert

Tests. 120 tests, green at the time of writing, over scoring, routing, probe error classification, crypto round-trips, the command surface, discovery parsing and de-duplication, all three adapters, and the proxy's failover behaviour — retryable classification, attempt clamping, backoff doubling and its cap, Retry-After, and SSE frames that arrive split across chunks. Because scoring and routing are pure functions, those tests are plain tables of input and expected output with no mocks in sight.

There is no CI yet — see the roadmap. Until there is, please run cargo test, cargo clippy --all-targets and npm run build before opening a pull request.

Nothing below is a dated promise. It is the order in which things would land, and an honest list of what today's 0.1.0 does not do.

In 0.1.0

• Local OpenAI-compatible gateway with 9 routing strategies and failover
• Scheduled health probing — TTFT, latency, throughput, jitter, error taxonomy
• Quota tracking from rate-limit headers, with time-to-exhaustion estimates
• On-demand benchmarking and historical charts
• Provider Radar — GitHub, Hacker News, pinned feeds, optional search APIs
• openai, anthropic and gemini wire formats
• Keychain-sealed storage, per-provider custom headers, passphrase-sealed backups

Next

• CI on push — fmt, clippy, cargo test, vue-tsc
• Tagged releases with prebuilt installers for Windows, macOS and Linux, so running KeyPulse does not require a Rust toolchain
• Screenshots, GIFs and an architecture diagram — every image slot in this README is still a placeholder
• A native Anthropic /v1/messages endpoint on the gateway, so clients that speak that shape rather than the OpenAI one can route through it too
• More non-OpenAI wire formats — AWS Bedrock and Vertex AI are the obvious next adapters; anything with an OpenAI-compatible endpoint already works today
• Tool-call and vision passthrough for the non-OpenAI adapters. Today the Anthropic and Gemini adapters translate text only, so a request carrying tools or an image is routed only to keys whose wire format already speaks it. The gateway detects the need and filters the candidate pool rather than silently dropping the field — but the translation itself is not written yet
• Rate-limit-aware pacing — spread gateway traffic to stay under a published limit, instead of reacting after the 429 has already been spent
• Headless mode — run the gateway without the window open

Ideas, not yet designed

• Per-model scoring instead of per-key, for providers whose models fail independently
• Price-aware routing for paid keys, alongside the latency-aware strategies
• A metrics endpoint for people who already run Prometheus or Grafana

Deliberately not planned: a hosted version, telemetry of any kind, and any feature whose purpose is sharing one key between several people.

Contributions are welcome, including the unglamorous ones. The repository is young — one commit, no releases — which is the best possible time to shape it.

Why it is a pleasant codebase to work in

• The decision logic is pure. scoring.rs and routing.rs take plain structs and return plain structs. No clock, no network, no database, so a change you make is a test you can write in a minute.
• One abstraction, on purpose. No repository layer, no DI container, no DDD. If you are looking for where something happens, it is in the file named after it.
• The tests are real. 120 of them, including a socket-level test that pins header precedence, because a provider once rejected requests over a User-Agent.
• MIT, no CLA. Your commit is yours.

Good first contributions

There is no issue tracker with labels on it yet, so here is the honest list of self-contained gaps:

MIT — Copyright (c) 2026 KeyPulse contributors.

Use it, fork it, ship it. There is no hosted tier to upsell you to and no server to switch off: your keys and your probe history are files on your own disk.