The AI me

One real run, timed: “How do you stop double charges in billing?”, asked on the home page. The guards, the kept-answer check, both breakers and the day’s tokens share one round trip to the store. The five best passages are found in 8 ms. Nothing in the question asks to see a card, so the model is asked. Its first request calls for the IABTM card, on screen at 1.28 s. The second writes the answer and cites the billing section. First words at 2.35 s, done at 2.73 s: two requests, 10,260 tokens in, 95 out.

The core is one function, runTurn, that yields events: passages, words, a card to show, an action to take, a trace, done. The chat, voice and the evals all run it. The route streams the events one JSON object a line, so a card lands before the first word: NDJSON over a streamed fetch, rather than EventSource, which can’t send a session header. The loop is capped at 3 requests and 20 s. A card goes out the moment its call is whole; a lookup goes back to the model; the last pass is offered no tools, so it has to answer. Across 20 timed questions a turn took 1.5 requests on average.

The fastest answers never reach the model. A plain command (“make it night”) is done in under 0.1 s. An explicit ask to see something puts its card up before the model starts, from a page of patterns that plans the obvious calls. Follow-ups keep their thread: a question carries the last 6 turns with notes of what each showed and did, the last 4 kept whole and fitted before any passage, so “tell me more about that” knows what “that” is. Over 20 live questions on a production build, something was on screen within 0.3 s for 14; first words came at 1.85 s median (2.3 s at p90), the whole answer at 2.15 s (2.85 s at p90).

No agent framework and no vendor SDK. The model is an address, a name and keys in the environment, so any endpoint that speaks chat completions and streams server-sent events can run it. I read the stream by hand: tool calls assembled from their pieces by index or id, any thinking dropped, tokens estimated where an endpoint doesn’t report them. That is a few hundred lines, and owning them is what lets a card arrive mid-answer and a backup take over mid-turn.

Failover happens inside the turn. Keys rotate, and a refused key rests 1 minute (a rate limit) or 1 hour (spent, revoked, or a 400 that names the key) while the next is tried in the same request. Behind the first provider are 3 backups, each with its own endpoint, model, keys, quota and breaker (5 failed questions in 2 minutes open it for 3). Backups are offered fewer tools, because their tokens a minute are often fewer. A question counts once against a provider, however many of its keys were tried.

It is proven against the real endpoints. Before a deploy, a fire drill breaks each provider’s keys in turn, to see the next provider answer, a refused key rest and not be asked again, and the end of the line say so plainly. It catches what a unit test can’t: on its first run, two suspended keys and a backup whose model was no longer free. Each fault it finds is fixed, with a test. When every provider fails, my phone hears at once.

Every answer is built from what this site says, in two parts. The facts go in whole: a fact sheet of about 7,300 characters (every job, project and date) is in every prompt, so a date never depends on retrieval. The rest is searched: the site’s own markdown copies (this page is /projects/ai-me.md), cut at headings into about 190 passages of 400 to 900 characters from 17 pages, each linked to its section.

Passages are found by their words, with BM25 I wrote by hand: titles counted twice, a light stemmer, a small synonym table, a 1.6 lift for a project the question names. I also built search by meaning and measured it against the words: vectors made at build for changed passages only, in MongoDB Atlas Vector Search, fused by reciprocal rank, capped at 700 ms. On the first run it put the right page first 5 more times. Run again from Atlas, it found the page in the top five for 30 questions against the words’ 31, and cost 0.4 s a question. At this size that doesn’t pay, so it is off and the words answer alone.

Citations are checked, not trusted: the model may cite only the passages it was given, by number, and any other citation is dropped as it streams. It says it’s an AI when asked and never commits me to a date, a rate or a meeting. When the site doesn’t say something, it says so and offers to send the question to the real me. The fit check is one JSON request, checked before it shows: its shape against a schema, every link against the site’s pages, and a gap never gets a link.

A tool is one entry in a registry: a name, a description, a JSON schema and a function. 5 look things up, 18 show one of the site’s own components (a project card, my calendar, a drafted note), and 8 act on the page (scroll, highlight, tour, navigate, day or night, weather, undo). Arguments are checked against the same schema the model was shown. A bad call (an unknown tool, broken JSON, a value out of range) goes back to the model as an error to read, never a throw. Only 17 are offered by default, because every tool offered is read before the first word.

The server only proposes an action. The chat carries it out from its own short list, checks the arguments again, finds its target by the label the page gives it (never a selector or a script from the model), and leaves a receipt with an Undo. Commands work misspelled (“drak mode”, by edit distance) but never fire on a question: “what’s summer like?” changes nothing, and a matrix of more than 100 phrasings, from each state of the page, holds it to that. If an answer claims an action that never ran, a line after it says so.

Prompt injection is handled by capability more than by wording. No tool sends, pays or writes, so a drafted note goes only when you press Send. The visitor’s words, the passages and a pasted job description are fenced as data, and obvious tries (“ignore your previous instructions”) get a line back in under 0.1 s, with no model. Output is filtered as it streams: no HTML is ever drawn, links go only to https or this site, and there are no images to carry data out. A tool call written out as words (a bold name, tags, a JSON fence) is held back, and the card it asked for is shown for real.

A chat that calls a paid model from a public page is a bill anyone can run up. So every limit is counted in the store, and it fails closed: if the limits can’t be counted, the model isn’t asked. A question needs the site’s origin, a browser and a session token: an HMAC over its issue time and the address, good for 2 hours, compared in constant time, never stored. Visitors are counted under a keyed hash of the address that changes daily, so no address is kept. Answers in flight are entries stamped with their start that age out after 90 s, not a counter that drifts when a function dies.

Tokens go only where a model adds something. Commands, off-topic questions and tries at its rules never reach one. The suggested questions are answered ahead of time and ship with the site, each kept only if a second reading by a model finds every claim on the site. A fresh answer is kept a day and replayed in about 25 ms for no tokens. Its key is the question, the page and its parts, the time and weather showing, and a hash of the prompt and content. An answer that acted on the page is never kept: a kept “I’ve made it night” would replay without the night.

What a question costs, measured: an answered one averaged about 8,000 tokens, so the daily budget carries about 185, before kept answers and commands, which cost none. On a free tier the provider’s tokens a minute bind sooner. Asked back to back, it refused after 8 questions; spaced to 5 a minute, 25 in a row went through.

A prompt change is tested like a code change. The prompt has a version that goes into every trace and into the kept-answer key, so a new prompt never serves an old answer. The unit tests run with no model: the stream parser cut at every point, tool calls assembled across chunks and providers, key rotation, schema checks, every leak shape, the guards, and retrieval, which fails CI if hit@5 drops under 90%.

Answers are scored against the real model. The eval set is 44 questions, each with where its answer lives and the words a right answer must say, plus follow-ups, persona checks and tries at its rules. A script runs them before a prompt change ships. A nightly workflow asks 11 of them and tells my phone when answers fall under the bar: 9 in 10 true to the site on a second reading by a model, in the first person, with no invented number. It waits only on the model’s keys in CI.

Every answer carries its own trace, open under it: each step as a span, when anything first showed, tokens in and out, the provider and the prompt’s version. A trace holds timings and names, never words, and folds into daily counts kept 35 days.

What I keep is counts, not transcripts. Your conversation stays in your browser. Questions are kept 30 days with emails and phone numbers blacked out, on a review desk only my key opens, and the ones it couldn’t answer tell me which pages to write next. My phone hears about each fit check, a provider failing or the budget running low, and gets the day at 8 pm.

Voice is a transport, not a second brain. Press the mic above the tab, or v, and a realtime speech-to-speech model runs over a socket the browser opens with a single-use token my server mints. The brief, facts, tools and voice are sealed into that token, so a modified page can’t change them. Its tool calls run on my server through the same registry and guards, into the same conversation. The voice is a stock one, never a clone, and it says it’s an AI.

Voice runs on its own keys, so talking can’t starve typing, and adds minutes to the chat’s guards: 3 minutes a session, a few sessions a visitor a day, a cap on sessions at once and a daily budget. When realtime is full, it falls back to the chat’s own engine, spoken a sentence at a time. Each sentence is sealed to the chat’s answer, so the site’s voice can’t be made to say anything else. About 80 spoken questions are checked against the real model before a change ships.

The day’s token budget fills first, and that is a choice. Next is the provider’s tokens a minute, which isn’t; the fix is a paid tier, with the backups on their own quotas. Then the store: an answered question is 46 to 53 store commands, 12 for the limits alone. One script per question that counts every window, plus one write of the day’s counts a turn, would cut that by more than half. Concurrency comes last: the in-flight cap, at about 2 s an answer, is several hundred a minute, far above the 60 a minute allowed.

Under real load I’d add four things. Rate limits and a firewall rule at the edge, so a flood never wakes a function. A semantic cache, so “what’s your stack” and “what do you build with” share an answer. The prompt’s fixed 4,000 tokens put first, for a provider’s prefix cache. And a short queue that gives a place in line instead of a refusal.

Routing is two layers today: patterns for commands and the obvious cards, then one model with tools picked per question. Next is a learned router over 5 paths (command, precomputed, retrieval only, full agent, fit check) and a small model for the simple questions, each route judged by its precision and recall on real questions from the review desk. Many agents wait until the evals show one agent plateauing. Until then they only add latency, calls and evals that are harder to trust.

Then the engine ships as a package any site can drop in, after 10xAnswers, my chat component on npm. The core is ready for it: it already knows nothing of HTTP, the model is an address, and tools are registry entries.

Most of the speed comes from not asking the model. A small model is unreliable in small ways, so what it writes is a stream to parse and check, not text to trust. And the limits are the product: an assistant on a public page is only as good as how it behaves when it’s abused, rate-limited, out of tokens or talking to a broken endpoint.