glimpsh

A glimpse into a potential future where you navigate multiple terminals with your eyes.

This is a fun experimental project exploring eye-tracked terminal focus. Look at a pane to focus it. Combine with speech-to-text for hands-free coding.

Glimpsh is a prototype exploring what becomes possible when lightweight gaze tracking is cheap and ubiquitous. Today's webcam-based eye tracking is imperfect, but it's improving fast.

Gaze won't replace the mouse - it's a complementary input that adds a new dimension to human-computer interaction. Where it shines is managing many things at scale. Some power users are fast with keybinds, but gaze adds an intuitive HCI primitive to terminals: your eyes are already looking where you want to act.

Pair gaze with real-time voice transcription and the interaction model shifts: look at an agent, speak a command. As brain interfaces mature, the same patterns will translate directly.

This project exists to explore that future now.

git clone https://github.com/dchrty/glimpsh
cd glimpsh
pip install uv
uv sync --extra glimpsh-eyetrax
uv run glimpsh --claude  # or --codex for OpenAI Codex

On first run, follow the on-screen calibration (look at dots until they disappear).

To recalibrate or fine-tune the eye tracking model:

uv run glimpsh --recalibrate

Or for hands-free AI coding with Claude Code and voice (requires a speech-to-text tool that copies transcriptions to the clipboard — see Voice Input):

uv run glimpsh --claude --dangerously-skip-permissions --voice

We've started with glimpsh-eyetrax, based on Chenkai Zhang's EyeTrax, for webcam-based gaze tracking. Adapting other gaze servers is relatively easy - just implement the websocket protocol.

uv run glimpsh --gaze test

Use Ctrl+Arrow keys to simulate eye movement.

The --voice flag monitors the clipboard for changes and auto-types them into the focused pane. It's designed to work with any speech-to-text tool that puts transcribed text on the clipboard — all glimpsh cares about is the clipboard changing.

uv run glimpsh --voice

Look at a pane to focus it, then speak. Your dictation tool copies to clipboard, glimpsh types it in.

Tools like Wispr, Talon, Nerd Dictation, or open-weight models such as Whisper / whisper.cpp and Vosk could all serve as the transcription source. Some of these copy to the clipboard out of the box; others may need a minimal wrapper script to pipe their output there.

Run Claude Code in each pane:

uv run glimpsh --claude

For fully autonomous operation (Claude can run commands without confirmation):

uv run glimpsh --claude --dangerously-skip-permissions

Combine with voice for hands-free AI coding - look at a pane, speak your request:

uv run glimpsh --claude --dangerously-skip-permissions --voice

Run Codex CLI in each pane:

uv run glimpsh --codex

MIT