ghost_pad

Turn the desk around your laptop into tap-buttons, using nothing but the built-in MacBook mic. Tap the desk to open apps. No hardware, no wiring.

Status: failed experiment. It technically runs, but the core idea doesn't hold up. Kept here as a writeup of why.

One microphone. Tap different spots on your desk, and each spot opens a different thing — chess in one corner, Slack in another, a book, a browser. The desk becomes a silent macro pad you never have to look at.

Two decoding strategies were built:

1. Position (train / run classifier path) — record ~40 taps at each "zone" (e.g. left corner, right corner), extract a spectral fingerprint per tap, and train a tiny k-NN classifier. At run time, classify each tap to a zone.
2. Count (the run burst loop) — count taps in a quick burst (1/2/3/4 taps) and fire the matching action. This is the path the final run() actually uses.

The mic gives you exactly two honest signals per tap: when it happened and how loud/what-shaped it was. Everything else is inferred from those.

• Detection — stream audio in ~23 ms blocks, keep a rolling 80 ms ring buffer, and trigger when RMS crosses NOISE_FLOOR × TRIGGER_MULT. calibrate measures your room's noise floor so the threshold isn't guessed.
• Rejecting non-taps — a tap is a transient: it's loud for under ~25 ms. Speech syllables sustain much longer, so is_transient() filters most talking out by measuring how long the envelope stays above 25% of its peak.
• Position features — for the classifier, each 80 ms window becomes: log-scaled FFT magnitude of the loudest slice, peak amplitude, RMS energy, and spectral centroid ("brightness"). The theory: distance to the mic and the desk's own resonances make each spot sound acoustically distinct and, crucially, repeatable.
• Count features — much simpler: just debounce and time the gaps between taps (TAP_GAP = 0.45s).

Commands: calibrate (noise floor), meter (live level bar), devices (list inputs), train (record zones), tune (compare tap vs speech metrics), run (listen + fire).

The fundamental problem: a single mic doesn't give you position. It gives you duration and amplitude, and I was trying to back out a 2D location from that. It doesn't invert cleanly.

• The position path is too fragile. The whole trick depends on each zone's sound being repeatable — which requires the laptop, the desk, and the mic to never move, and every tap to land on the exact same spot with the same finger, same force, same angle. In practice a tap 3 cm away, or a slightly softer knuckle, blurs the fingerprint and the k-NN gets it wrong. Corners far apart work okay; anything finer collapses. It's a party trick, not a UI.
• Amplitude ≈ distance is a lie. Loudness depends on how hard you tapped far more than where you tapped, so the one signal that should encode position is dominated by tap force, which you can't hold constant.
• So it fell back to counting taps — which is reliable, but at that point you've thrown away the whole "spots on the desk" idea. Counting 1–4 taps is just Morse-lite; it has nothing to do with the desk being a surface. A keyboard shortcut does the same job with zero false triggers.
• False triggers everywhere. Typing, a coffee mug, a door — all transient, all loud. The 25 ms duration filter helps but doesn't close the gap. An always-listening mic that opens apps on stray desk noise is worse than no shortcut at all.

The honest conclusion: to do position sensing acoustically you need multiple mics and time-difference-of-arrival, or a different sensor (piezo/accelerometer on the desk). With one mic you only ever get "something was tapped" and a rough "how hard" — enough to count, not enough to locate. Counting works but doesn't need any of this.

pip install sounddevice numpy scikit-learn
python desk_buttons.py calibrate   # measure noise floor, edit the two constants
python desk_buttons.py meter       # sanity-check the mic is reaching Python
python desk_buttons.py run         # 1=chess 2=slack 3=book 4=brave

macOS only (uses open -a). Needs mic permission for your terminal in System Settings → Privacy & Security → Microphone.