Skip to content

Native key input: per-platform status (v1.7 attempt-first /key) #3

Description

@slabbdev

v1.7 shipped POST /key as attempt-first, honest mode: real OS-level events when possible (isTrusted: true), automatic fallback to the in-page synthetic dispatch, and the response always reports which mode ran. The README links here for where each platform stands.

Status per platform

  • Windows (wry / WebView2) — verified end-to-end. SendInput + SetForegroundWindow, targeting the WebView2 child HWND via EnumChildWindows. Native events land in the page; this is the reference implementation.
  • Linux (wry / WebKitGTK) — implemented, X11 sessions. x11rb XTEST fake_input with keycode resolution from the live keymap. The ghost window is brought on-screen first (X11 refuses focus to unviewable windows), and deliberately no gtk::main_iteration() runs on the HTTP thread (GTK is main-thread-only; pumping from a worker deadlocked the loop — that was CI exit 52). Headless CI has no focus target → falls back cleanly.
  • macOS — parked, deliberately. CGEvents can be posted, but routing them into a headless WKWebView reliably needs an app-bundle activation story the Window Server only grants to real foreground apps; the experimental on-screen approach (git history) still left document.hasFocus() == 0. Until that's solved, macOS reports "native unavailable" and falls back to synthetic.

Open work

  • macOS activation story (real app bundle, or session_show-assisted native typing on the visible window)
  • Wayland: XTEST is X11-only — fine under XWayland, needs a different path on pure Wayland
  • Keymap coverage: grow the x11_keysym table; special keys/modifiers on all three paths

Activity

  1. slabbdev commented on Oct 8, 2026

    @slabbdev
    OwnerAuthor

    macOS: RESOLVED in b3c24a0 — focus-gated CGEvents, verified in-page

    The activation story that unblocked it: show_window() is the whole recipe — it materializes the ghost's window, converts it to a titled window (the borderless ghost mask can never become key), centers it, activates the app, and makes it key. Only when isKeyWindow confirms does anything get posted — posting into the HID tap without focus would type into someone else's app.

    Layout independence: single characters carry CGEventKeyboardSetUnicodeString — keycode 0x00 is 'a' on QWERTY but 'q' on AZERTY; the attached unicode string wins, so the agent's character is what lands ('Z' arrives uppercase with no shift needed).

    Honesty preserved: a document-level once-listener must observe {trusted:true} before Ok is reported — otherwise Err and the synthetic fallback runs with the mode saying so.

    Live-tested on AZERTY hardware: a, Enter, Z, Tab, ArrowLeft → mode: native, TRUSTED:<key> on all five, zero fallbacks.

    Still open here: Wayland (XTEST is X11-only; fine under XWayland, needs libei/uinput on pure Wayland) and keymap coverage growth (the mac table covers letters/digits/punctuation/named keys; the X11 keysym table can grow the same way).

  2. slabbdev commented on Oct 9, 2026

    @slabbdev
    OwnerAuthor

    Closing with the scoreboard: macOS — RESOLVED and live-verified (v1.9.0, AZERTY hardware 5/5 TRUSTED, show_window focus gate + CGEventKeyboardSetUnicodeString, b3c24a0). Windows — verified end-to-end since v1.7 (SendInput). Remaining: Wayland (XTEST is X11-only; needs libei/uinput) and keymap table growth — demoted to backlog, this issue is the record; reopen when Wayland lands on the roadmap. The synthetic fallback carries every gap in the meantime, with the mode honestly reported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions