A voice-first desktop assistant that lives in a floating bubble, stays on top of whatever you're doing, and can see your screen when you ask it to.
Built for the Gemini Live Agent Challenge.
- Talks — live bidirectional audio through the Gemini Live API, not record-then-send
- Sees — screen capture on request, so "what's this error?" works without copy-paste
- Reads — drag a file onto the bubble and ask about it
- Stays out of the way — always-on-top, frameless, collapses to a bubble when idle
Voice is a different interaction model, not a different input field. Latency and interruption handling matter more than transcription accuracy — an assistant you have to wait for is one you stop using. The audio path is streamed in both directions rather than batched.
Separate failure points. Transcription, intent, response generation, and screen capture each fail differently. They're handled as distinct stages so one degrading doesn't take the session down.
Screen access is explicit. It captures when asked, never ambiently.
Electron · React · Vite · Tailwind v4 · Gemini Live API (audio + vision)
npm install
npm run devRequires a GEMINI_API_KEY in .env.