Plugin based kittest inspector#8136
Draft
lucasmerlin wants to merge 37 commits into
Draft
Conversation
|
Preview available at https://egui-pr-preview.github.io/pr/8136-lucaskittest-inspector-plugin View snapshot changes at kitdiff |
eab06d9 to
8350402
Compare
6ac83f8 to
5e7c70b
Compare
2634e78 to
f4d60f0
Compare
* Part of #8180 So far, you've been able to move any `egui::Window` by dragging anywhere on it. This makes sense on touch screens with thick fingers, but less so on non-touch-screens. With this PR, you can now control it with a new enum `WindowDrag`
## Related * #5851 * #7988 ## What We want to make it easier to understand the lineage of a `Id` (which is the parent `Id`, and the parent of that, etc?) As a first step of that, we want to clarify the different between a globally unique `Id`, and an `IdSalt` I also introduced the `AsId` and `AsIdSalt` traits, which are implemented of anything that implements `Hash` and `Debug`. The `Debug` half of that is unused here, but will later be used in `Debug` builds to produce a proper tree.
* Closes #8029 --------- Co-authored-by: lucasmerlin <hi@lucasmerlin.me>
## What Adds a way for apps to push an RGBA bitmap as the OS cursor — the missing companion to `Context::set_cursor_icon`. The integration translates it into a real `winit::CustomCursor`, so the cursor is drawn by the compositor and can extend past the egui window edge like any native cursor. ## Why Apps with custom-shaped windows (Winamp-style skins, themed launchers, kiosk apps) currently have no clean way to display a custom cursor: - `CursorIcon` is limited to the standard system enum. - Painting the cursor sprite via `egui::Painter` works inside the canvas but gets clipped at the window edge — the bottom/right of the cursor disappears the moment the pointer is near the boundary, and there's no way to render onto the desktop area exposed by a transparent/region-shaped window. `winit` 0.30+ already supports `CustomCursor::from_rgba` + `ActiveEventLoop::create_custom_cursor`, but `egui-winit` doesn't surface it. This PR exposes it through egui. ### Visual demonstration Driving use case: a Winamp WSZ skin player ([all3f0r1/oneamp](https://github.com/all3f0r1/oneamp)) with a transparent + region-shaped window where the skin ships its own `.cur` files. The bottom-right corner of the playlist exposes the resize cursor — notice how it gets clipped at the window edge in the painter-based approach. | Before (cursor painted via `egui::Painter`) | After (cursor pushed via `set_cursor_image`) | | --- | --- | |  |  | ## API ```rust // new in egui::data::output pub struct CustomCursorImage { pub rgba: std::sync::Arc<[u8]>, pub size: [u16; 2], // matches winit's u16 to avoid lossy casts pub hotspot: [u16; 2], } // new field on PlatformOutput (skipped from serde — ephemeral) pub cursor_image: Option<CustomCursorImage>, // new method on Context ctx.set_cursor_image(Some(image)); // overrides cursor_icon for this frame ctx.set_cursor_image(None); // revert to cursor_icon ``` `Arc<[u8]>` is intentional: the integration dedupes by `Arc::as_ptr`, so reusing the same Arc across frames means the bitmap is only uploaded to the OS once per skin, not once per frame. ## Integration changes - `egui_winit::State::handle_platform_output_with_event_loop(window, Option<&ActiveEventLoop>, ...)` is a new method that threads the active event loop so it can call `event_loop.create_custom_cursor(...)`. - The legacy `handle_platform_output(window, ...)` delegates with `None` and silently drops `cursor_image`. **No existing callers break.** - The icon and bitmap paths are unified in a private `apply_cursor`. The no-flicker dedupe of the old `set_cursor_icon` is preserved on both paths. - If `CustomCursor::from_rgba` rejects the bitmap (bad dimensions, hotspot OOB, etc.), we log a warning and fall back to the icon path. - eframe's wgpu + glow integrations thread `&ActiveEventLoop` through `run_ui_and_paint` (glow already had it; wgpu needed one extra parameter) and call the new method. - Immediate viewports keep the old path because they're invoked from a `Context` callback that doesn't have an event loop reference. Custom cursors are a no-op in immediate viewports — acceptable since they're a niche path. ## Fallback semantics | backend / context | what happens | |--------------------------------|-------------------------------| | eframe wgpu/glow main viewport | bitmap displayed via OS | | eframe immediate viewport | falls back to `cursor_icon` | | eframe web | falls back to `cursor_icon` | | custom integrations not opted in | falls back to `cursor_icon` | | `from_rgba` returns `BadImage` | warning + falls back to icon | ## Verification - `cargo fmt --all -- --check` ✅ - `cargo clippy -p egui -p egui-winit -p eframe --all-targets --all-features -- -D warnings` ✅ - `cargo doc --lib --no-deps -p egui -p egui-winit -p eframe --all-features` ✅ - `cargo check -p egui --no-default-features --features serde` ✅ (validates the `serde(skip)` on `cursor_image`) - Interactive validation on Linux/Wayland with the OneAmp WSZ skin player — see screenshots above. I haven't run the full snapshot test suite (`scripts/check.sh`) because we're on Linux and the snapshots are macOS-rendered — happy to run it if you'd like. ## Notes Drafted per the contributing guide ("open a draft PR, you may get helpful feedback early"). Open to design feedback on: 1. Whether `CustomCursorImage` should live in `egui::viewport` rather than `egui::data::output`. 2. Whether the legacy `handle_platform_output` should grow `event_loop` directly (breaking) instead of getting a sibling method (non-breaking, what I did). 3. Whether to also wire it through eframe-web (probably not — `wasm-bindgen-cursor` would need its own path). --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Closes #7254 You can now drag-to-close a panel. Also drag-to-expand panels. This is a breaking change: the animated panel functions now take a `open: &mut bool` instead of `open: bool`. This is only enabled for resizable panels
The three methods for showing a `Panel` are now: * `panel.show`: always show the panel. * `panel.show_collapsible`: show or hide the panel, with a slide animation in between. * `Panel::show_switched`: animate between two different panels: a thin/collapsed one and a thick/expanded one.
Double-clicking the edge of a resizable panel will now toggle it collapsed/expanded
Current behavior fails when translating file uris that contain windows UNC paths. This commit attempts to fix that behavior by looking at the hostname attribute of the uri and changing behavior if the hostname is present. * Closes <#8161> * [x] I have followed the instructions in the PR template
* Closes #8179
This changes the monitor selection used when restoring a persisted window position. Currently, `egui-winit` picks a monitor by checking whether the saved window position fits inside a loose monitor range. This can choose the wrong monitor when a saved window rectangle slightly overlaps another monitor. My failure case was on Windows with two monitors: - primary monitor on the right - secondary monitor on the left - window maximized on the primary monitor - persisted outer position was slightly negative, e.g. `x = -8`, because of the invisible window border That position matched both monitor ranges, so the restored maximized window could open on the secondary monitor instead of the primary one. This PR picks the monitor with the largest overlap with the saved window rectangle instead. Related note: probably the best solution would be to save the normal window position when maximized, so that when unmaximizing, the window would get restored to the previous state. It's mentioned in this comment #3494 (comment). I tried doing that in [fix-windows-maximized-restore-placement](https://github.com/YelovSK/egui/tree/fix-windows-maximized-restore-placement), and it works, but it requires adding windows-sys as a dependency to call a relevant winapi, so that's probably not the right solution. Winit doesn't seem to provide an API that would return this information. * [x] I have followed the instructions in the PR template --------- Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
`ab_glyph` would output coverage values, but `vello` outputs RGBA. So the old name was a misnomer. I also suspect our default values are wrong, but I need to investigate that more properly in a separate PR.
* Follows #8199 This makes the animation of the collapsing panel a bit smoother, by taking into account the spacing between the header and the body.
New `egui_inspection` crate ships: - `protocol` (default): wire types + length-prefixed msgpack framing for the inspector ↔ egui-peer connection. Transport-neutral (stdio / unix socket / TCP). - `plugin`: `InspectionPlugin`, an `egui::Plugin` that dials a unix socket from `EGUI_INSPECTION_SOCKET`, streams frames + accesskit tree updates, and applies inbound `InspectorCommand`s back into the running `egui::Context`. eframe gains an `inspection` feature that auto-attaches the plugin during native startup (glow + wgpu integrations) when the env var is set. Connection failures log via `log::warn!` and do not abort startup. Lives in its own crate (rather than `egui_kittest`) so eframe can pull the protocol in without picking up the test harness, and so external tools can depend on it directly.
f4d60f0 to
4a5daa6
Compare
Replace std::os::unix::net::UnixStream in the InspectionPlugin with the interprocess crate's local_socket::Stream, so the transport works on Windows (named pipe) as well as unix/macOS (unix domain socket). - New transport module (transport feature) with socket_name() and generate_socket_target() — one shared, platform-split place to build/allocate local-socket names, used by both ends of the connection. - Drop the cfg(unix) gates on the plugin module; gate on the plugin feature only. - attach() now takes a socket name string and connects via interprocess; the stream is split with Stream::split() instead of UnixStream::try_clone().
4a5daa6 to
026a859
Compare
…l feature - FrameScreenshot now carries PNG bytes instead of raw RGBA (PROTOCOL_VERSION 1→2); add a shared `encode_png` helper behind a new `png` feature so the live plugin and the kittest harness encode frames identically. - Make the protocol module unconditional: drop the `protocol` feature flag and the optional serde/serde_bytes/rmp-serde deps it gated. - plugin.rs: re-stamp screenshot-bearing frames with the current step (so inspectors waiting for step > prev don't reject them) and pump a tail-side repaint while awaiting the GPU readback.
026a859 to
39978b7
Compare
Add transport::connect (dial + split) and a sync transport::Listener (bind + accept) so both ends of the inspection connection build streams identically without depending on interprocess directly. Plugin now dials via transport::connect. These back the kittest harness moving onto the same local socket as the live plugin.
Replaces the separate on_accesskit_update hook with a `&TreeUpdate` parameter on `after_step` — one hook per step, tree delivered inline. `step_no_side_effects` now returns the TreeUpdate so plugins driving the harness from within their own hook (where nested dispatches are suppressed) can still see it. Also adds `#[track_caller]` to the internal `_step` / `_step_no_side_effects` / `_try_run` so `Location::caller()` inside `step()` walks up to the user's original call site when reached via `run()`.
Drop the redundant `plugin_` prefix on the dispatch helper (it's already a method on `Harness`). Make `TestResult` `Copy` so it can be passed by value inside the dispatch closure, removing the need for the `fail_ref` re-borrow helper. Also drop the empty-plugins early-return microopt in `dispatch` — the `mem::take`/`extend` dance is cheap on empty vecs.
The wrapping Option only existed so Drop could consume SnapshotResults before plugins fire. mem::take swaps in a fresh default instead, dropping the original — same outcome with no Option-juggling at every call site (no more .as_mut().expect(...), and take_snapshot_results stops needing .replace()). The default SnapshotResults has handled = true so dropping the placeholder is a no-op.
New `InspectorPlugin` (gated behind `inspector` feature) launches a
`kittest_inspector` child process and streams the harness's frames + accesskit
tree updates to it over framed MessagePack on stdin/stdout. The inspector
drives the harness by sending `InspectorCommand`s back; supported commands
include `Step` / `Run` / `Play` / `Pause` (deterministic stepping),
`Handle { events }` (event injection), `Resize`, and `Screenshot`.
Auto-attaches when the `KITTEST_INSPECTOR` env var is truthy — the inspector
binary path can be overridden via `KITTEST_INSPECTOR_PATH`. Uses the new
`egui_inspection::protocol` types and starts every connection with a
`HarnessMessage::Hello { peer_kind: Kittest, capabilities: KITTEST }` so the
inspector can render the right controls.
Also re-exports `egui_inspection` as `egui_kittest::inspector_api` for crates
that only depend on kittest.
…tion::encode_png - Encode harness frames with egui_inspection::encode_png instead of sending raw RGBA, so the inspector socket carries compressed bytes. - Features: pull egui_inspection/png + image/png for the encoder; drop the now-removed egui_inspection `protocol` feature.
The harness inspector now speaks the wire protocol over the same interprocess local socket as the live egui_inspection plugin, in two modes: - connect: EGUI_INSPECTION_SOCKET set -> dial the listening socket (e.g. the kittest MCP bridge). - spawn: KITTEST_INSPECTOR truthy, no socket -> bind a socket, spawn the kittest_inspector binary pointed at it, accept. env_enabled() now also auto-enables when the socket var is set. Pulls egui_inspection/transport into the inspector_api feature.
39978b7 to
e3bdddf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Alternative implementation of the inspector from #8119, based on the plugin api from #8135