Skip to content

feat(gestures): Add mouse gesture support with scroll-wheel combos#922

Open
landn172 wants to merge 5 commits into
Caldis:masterfrom
landn172:feat/mouse-gesture
Open

feat(gestures): Add mouse gesture support with scroll-wheel combos#922
landn172 wants to merge 5 commits into
Caldis:masterfrom
landn172:feat/mouse-gesture

Conversation

@landn172
Copy link
Copy Markdown

@landn172 landn172 commented Apr 16, 2026

Summary

This PR adds a full mouse gesture system to Mos, allowing users to bind custom actions to directional mouse movements combined with button holds and scroll-wheel inputs.

Features added:

  • Mouse gesture recording: Hold a mouse button and move in a direction (Up/Down/Left/Right) to trigger bound actions
  • Scroll-wheel gesture combos: Hold a mouse button and scroll up/down to trigger additional gesture bindings
  • Gesture configuration UI: New gesture table in the Buttons preferences tab for adding, editing, and deleting gesture bindings
  • Independent configs: Movement-based gestures and scroll-based gestures have separate sensitivity/threshold settings
  • Conflict resolution: Gestures coexist with smooth scrolling — scroll gestures re-fire correctly while a button is held
  • Localization: All new strings are localized (12 languages supported)
  • Unit tests: GestureProcessorTests covers core gesture detection logic

Key files:

  • Mos/InputEvent/GestureBinding.swift — Gesture data model and binding definitions
  • Mos/InputEvent/GestureProcessor.swift — Core gesture detection and direction classification
  • Mos/Windows/PreferencesWindow/ButtonsView/GestureTableCellView.swift — UI for gesture table rows
  • Mos/Options/Options.swift — New gesture-related config keys
  • MosTests/GestureProcessorTests.swift — Unit tests

How to test

  1. Build and run the app
  2. Open Preferences → Buttons
  3. Add a gesture binding (e.g., hold Right Button + move Left → Mission Control)
  4. Hold the mouse button and swipe in the configured direction
  5. Verify the action fires; verify smooth scrolling still works normally

ScreenShot

image

Related issues

#917

landn172 and others added 5 commits April 15, 2026 12:02
Hold a configurable trigger button (e.g. middle click) and move the
mouse to execute system actions by direction. Supports up/down/left/
right with independent action bindings per gesture.

- GestureBinding data model: trigger + 4 directional SystemShortcut actions, threshold, enabled flag
- GestureProcessor state machine: idle → pending → active, dominant-axis direction detection with 1.5× diagonal rejection, click replay on threshold miss
- MouseInteractionSessionController: gestureMotionHandler callback + setGestureTracking() keep-alive flag
- ButtonCore: intercept matching button events before InputProcessor; clearState on disable
- Options/Constants: gesture bindings persistence via UserDefaults
- PreferencesButtonsViewController: segmented Bindings/Gestures switcher reusing KeyRecorder
- GestureTableCellView: programmatic cell with KeyPreview + 4 direction popups
- Localizable.xcstrings: bindings, gestures, gestureNone, direction keys (11 languages)
- GestureProcessorTests: state machine, direction resolution, Codable round-trip, motion tap hooks

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add GestureInputMode enum (.mouseMovement / .scrollWheel) to GestureBinding with backward-compatible Codable (old JSON without key defaults to .mouseMovement)
- Add handleScrollEvent() to GestureProcessor: consumes scroll events in both .pending (accumulate delta, resolve direction) and .active (prevent smooth-scrolling pipeline leak) states
- Guard handleMotionEvent and startGestureTracking to only run for .mouseMovement bindings
- Hook GestureProcessor.handleScrollEvent into ScrollCore.scrollEventCallBack before trackpad check
- Fix footer layout: deactivate conflicting storyboard constraints and repin +/- buttons to trailing edge so segmented control doesn't overlap them
- Add right-click context menu delete to GestureTableCellView
- Add input mode segmented control (Movement / Scroll) to GestureTableCellView with onInputModeChanged callback
- Wire updateGestureInputMode() in PreferencesButtonsViewController
- Add gestureMouseMovement, gestureScrollWheel, delete localization keys (en/zh-Hans/zh-Hant/ja)
- Add 11 new unit tests covering scroll gesture state machine, backward-compatible Codable, and active-state scroll leak prevention

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…irection only

Movement (↑↓←→) and Scroll (↑↓) are now separate, simultaneously active configs
on the same gesture binding — remove the single-mode toggle entirely.

- GestureBinding: remove GestureInputMode enum and inputMode field; add
  scrollUpAction/scrollDownAction/scrollThreshold fields; manual encode(to:)
  skips legacy inputMode key; backward-compatible decoder ignores old inputMode
- GestureProcessor: independent pendingScrollDY accumulator (separate from motion
  DX/DY); handleScrollEvent uses scrollAction(for:) and scrollThreshold;
  handleMotionEvent guards on hasAnyMovementAction; motion tap only started when
  binding has movement actions; pendingScrollDY reset on all pending→idle transitions
- GestureTableCellView: remove inputMode segmented control; show two labeled
  sections side-by-side (Movement 4-dir, Scroll 2-dir); separate popup/action
  selectors and callbacks for each section
- PreferencesButtonsViewController: replace updateGestureInputMode with
  updateGestureScrollAction; wire onScrollActionChanged callback; row height 150
- Localizable.xcstrings: rename gestureMouseMovement→gestureMovement,
  gestureScrollWheel→gestureScroll (section header labels)
- GestureProcessorTests: update helpers, replace inputMode tests with
  threshold/scroll-action tests, add accumulator-independence test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two root causes:

1. Silent event consumption: when scrollDownAction is nil but scrollUpAction
   is set, hasAnyScrollAction=true caused all scroll-down events to be consumed
   (returning true) without any action firing. User saw no scrolling and no action
   — perceived as "scroll conflict".

   Fix: direction-aware consume. Check binding.scrollAction(for: eventDirection)
   before accumulating. If nil for that direction, immediately return false
   (let scroll pass through normally). Reset accumulator on direction change.

2. Smooth-scroll mouse delta: reading scrollWheelEventDeltaAxis1 (integer) gives 0
   for continuous/smooth scroll devices. No accumulation → threshold never reached.

   Fix: prefer scrollWheelEventFixedPtDeltaAxis1 (float fixedPt field, same
   priority logic as ScrollEvent.initEvent), fall back to integer field.

Both pending and active states updated to use the same direction-aware logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…hold

Previously, after a scroll gesture fired the state became .active and all
subsequent scroll events in configured directions were consumed without firing.
This caused a 5+ second apparent freeze: the user had to fully release the
trigger button before the other direction would work again.

Fix: in .active state, scroll gestures now re-fire on every threshold crossing.
Hold button + scroll up → fires, scroll down → fires, scroll up → fires again.
Directions with no configured action still pass through immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant