Camtrol

A browser-based webcam tracker for movement, hand gestures, and body pose, powered by TensorFlow.js. All processing runs locally in the browser — video never leaves your device.

• Motion detection — frame-difference analysis (no ML model, very lightweight)
• Hand tracking — MediaPipe Hands via @tensorflow-models/hand-pose-detection
• Hand gestures — poses (thumbs, peace, fist, etc.), touch-style (tap, swipe, scroll, long press, rotate), pinch zoom, wave
• Face expressions — smile, grin, frown, surprise, grimace, squint, kiss, brow raises, and more
• Gaze estimation — MediaPipe iris zones by default; optional WebGazer.js screen calibration (experimental, GPLv3)
• Body pose — MoveNet (single-person) via @tensorflow-models/pose-detection
• Event log — timestamps, details, and console output for every detection
• Live overlay — skeleton lines on the camera preview

GitHub Pages serves the site over HTTPS, which is required for camera access. For local development, use any static server with HTTPS or localhost:

npx --yes serve . -p 3000
# Open http://localhost:3000

1. Push this repository to GitHub.
2. Go to Settings → Pages.
3. Under Build and deployment, set Source to GitHub Actions.
4. Merge to main — the included workflow (.github/workflows/pages.yml) deploys the site root automatically.

Alternatively, set Source to Deploy from a branch and choose main with folder / (root).

1. Open the site (HTTPS or localhost).
2. Wait for models to load.
3. Click Start camera and allow webcam access.
4. Move, show gestures, or shift your body — events appear in the sidebar log.
5. Toggle detectors or adjust Motion sensitivity as needed.

MediaPipe (default) — uses iris landmarks from Face Mesh (refineLandmarks: true). A yellow crosshair on the video shows the estimated look point; the event log updates when your gaze moves to another 3×3 zone on the video frame.

WebGazer (experimental) — choose Gaze engine → WebGazer in the sidebar (page reloads). Uses the WebGazer.js library (GPL-3.0): it learns from your clicks and mouse movement while you look at the cursor, then maps gaze to the same zones on the camera preview. Face expressions are disabled in this mode to avoid loading two face models. Same camera stream as Camtrol — no second webcam.

Neither mode is calibrated lab eye-tracking — useful for demos and rough “where are you looking?” feedback.

index.html          # App shell
css/main.css        # Styles
js/app.js           # Camera loop & orchestration
js/event-log.js     # UI event log
js/overlay.js       # Canvas skeleton overlay
js/detectors/
  motion.js         # Pixel-diff motion
  hands.js          # Hand + gesture detection
  gestures.js         # Static pose classification
  gesture-motion.js   # Wave & pinch zoom (temporal)
  touch-gestures.js   # Tap, swipe, scroll, etc.
  face.js             # Face mesh detector
  face-expressions.js # Grimaces & expressions
  gaze.js             # Iris-based gaze point & zones
  gaze-webgazer.js    # WebGazer integration (optional engine)
  gaze-viewport.js    # Screen → video frame coordinate mapping
  gaze-engine.js      # Saved gaze engine preference
  gesture-stabilizer.js
  pose.js           # MoveNet body pose

Apache-2.0 (TensorFlow.js models use their respective licenses).