ABxAG is a private 3D AI desktop companion for Windows. She's not just a chatbot — she's a real desktop companion that:
| Feature | Description | |
|---|---|---|
| 🎙 | Voice Conversation | Talk naturally with Gemini Live. "Hey AB" wake word, reminders & alarms, quiet hours |
| 👁 | Screen Vision | With your permission, sees and understands your screen — OCR, vision, screen analysis |
| 🖐 | Real PC Control | 96 local tools (apps, files, mouse & keyboard, browser, clipboard) with confirmations & emergency stop |
| 🧠 | Memory & Mind | Episodic memory, goals, learned skills, curiosity & presence — all processed locally |
| 🧍 | Body | 3D character engine — import any .pmx / .vrm / .glb / .fbx / .obj model, .zip or folder |
| 🖱 | Reliable Hands | Drives your physical mouse with exact text labels, pauses when you touch the PC |
| 🔄 | Auto-Update | Hybrid updater: differential installs, version switching, mirror-feed fallback |
| Layer | Technology |
|---|---|
| UI | React 19 · Vite 6 · Tailwind 4 · Three.js (MMD/PMX) |
| Backend | Node + Express + WebSocket · Google GenAI Live bridge |
| Shell | Electron 43 · Tray · Crash recovery · Auto-update |
| Agent | Python FastAPI desktop tool server (UI Automation, input, files, apps) |
| Brain | Local event-driven cognition runtime (memory, goals, skills, presence) |
| Site | Static pages → Cloudflare Pages |
| Platform | Build | Status |
|---|---|---|
| Windows 10/11 64-bit | Setup (installer + auto-updates) | ✅ Recommended |
| Windows 10/11 64-bit | Portable (single file) | ✅ Available |
| Linux x64 | AppImage (experimental, no PC control) | 🧪 Experimental |
| macOS | — | 🔜 Planned |
- 🔐 Your Gemini key is encrypted with Windows DPAPI — never reaches the web page
- 🏠 All AI calls go through your local backend
- ✅ High-risk actions ask for confirmation
- 🛡️ Websites, documents & screen text are treated as untrusted data
- 🚨 Emergency stop shortcut (default
Ctrl+Alt+Shift+S) - 📦 No API key, no character model, no personal data shipped in the repo
6 eras shipped: Voice → Eyes → Hands → Memory & Mind → Body → Co-pilot (dual-cursor + HUD overlay)
Next: Full-isolation mode · Voice v2 · Skill sharing · Richer characters Exploring: Phone remote · Beyond Windows
- Work on the
DEVbranch (never commit straight to it — open a PR) - Branch naming:
feat/<x>,fix/<x>orchore/<x>; merge only when CI is green - Definition of done:
npm run lintclean, relevant tests green, website changes verified - Bigger features get a PRD in
PRD/+ a conductor track inconductor/tracks/<id>/
| Metric | Value |
|---|---|
| PC tools built in | 96 |
| Your mouse stays yours | 100% |
| Cloud accounts needed | 0 |
| Subscriptions required | 0 |
🌐 Website · ⬇ Download · 🗺 Roadmap · 📦 Releases · 👤 Creator
Built with ❤️ by @ABsUP · ABxAG is a private 3D AI desktop companion powered by your own Gemini API key.