Skip to content

feat: support resizing WKWebView browser viewports - #8072

Merged
austinywang merged 52 commits into
mainfrom
issue-8056-wkwebview-viewport-resize
Jul 15, 2026
Merged

austinywang merged 52 commits into
mainfrom
issue-8056-wkwebview-viewport-resize

Conversation

@austinywang

@austinywang austinywang commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • implement a per-surface logical viewport for WKWebView-backed browser surfaces across the exact 1...4096 CSS-pixel range
  • aspect-fit the requested viewport inside the existing pane without resizing the pane, moving other surfaces, or changing focus
  • apply the same viewport source of truth across inline, portal, preload, zoom, and offscreen screenshot hosts
  • add cmux browser viewport reset plus structured mode, logical/display size, scale, presentation, exactness, and pane_resized: false results
  • return localized invalid_state guidance when an attached browser inspector owns the split layout
  • document the CLI/RPC semantics and add responsive-layout, screenshot, zoom, focus, reset, and temporary-host restoration coverage

Review fixes

  • recompute viewport geometry from the current host after screenshot reattachment instead of restoring stale pane dimensions
  • preserve exact CSS and PNG dimensions across non-default page zoom
  • keep a newer portal/inline/preload host authoritative if host reconciliation moves the WKWebView while an asynchronous capture is running
  • localize all new CLI and socket-visible strings in English and Japanese and use generic browser-inspector wording
  • extract viewport CLI parsing/help from the oversized CLI file without changing either budget TSV
  • cap full-page stitching at 128 lazily addressed tiles so tiny emulated viewports cannot allocate or capture millions of tiles
  • reconcile the attached-inspector transition with the live WKWebView bounds and normalize zoomed full-page snapshot coordinates
  • suspend and restore emulated viewport ownership around element fullscreen, and normalize native-zoom tiles to exact CSS-pixel output
  • reset viewport emulation whenever an attached DevTools inspector takes ownership of the browser split, across both inspector detection and redock paths

Validation

  • arch -arm64 swift test --package-path Packages/macOS/CmuxBrowser --filter BrowserViewport - 17 tests passed
  • regression provenance: test-only commit af32e630f5 fails locally because the bounded tile plan and attached-inspector reset are absent; fix commit 854d59506c passes. GitHub Actions is disabled for this repository, so the red commit could not be recorded in CI.
  • full CmuxBrowser package suite - 146 tests passed earlier in this branch
  • syntax-parsed every touched app/CLI Swift file with xcrun swiftc -frontend -parse
  • python3 -m py_compile tests_v2/test_browser_cli_agent_port.py tests_v2/test_browser_api_unsupported_matrix.py
  • python3 scripts/swift_file_length_budget.py --base-ref origin/main - blocked by the current origin/main baseline: Sources/CmuxEventBus.swift is 619 lines and has no budget entry; this PR does not modify that file or either budget TSV
  • ./scripts/check-pbxproj.sh
  • ./scripts/lint-pbxproj-test-wiring.sh
  • python3 scripts/check-workspace-package-groups.py --check
  • python3 scripts/check-package-resolved-policy.py
  • localization catalog JSON parse plus English/Japanese coverage audit for all 11 viewport keys
  • git diff --check

Neither .github/swift-file-length-budget.tsv nor .github/swift-warning-budget.tsv is changed.

Fixes #8056

Summary by CodeRabbit

  • New Features
    • Added WKWebView browser viewport emulation with exact logical sizes (1×1–4096×4096), aspect-fit rendering, and browser viewport <width> <height> plus browser viewport reset.
    • Added structured render-limit handling and consistent screenshot sizing for emulated viewports.
  • Bug Fixes
    • Improved viewport re-application during page zoom changes and temporary webview rehosting.
  • Documentation
    • Updated CLI, skill, and contract docs for new viewport commands, inspector requirements, and limit/error behavior.
  • Tests
    • Added coverage for viewport layout, rendering/tiling planning, metrics, and end-to-end CLI viewport emulation + reset.

@vercel

vercel Bot commented Jul 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Canceled Canceled Jul 15, 2026 12:12pm
cmux-staging Building Building Preview, Comment Jul 15, 2026 12:12pm

@coderabbitai

coderabbitai Bot commented Jul 14, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds exact logical viewport emulation for WKWebView browser surfaces, including native reset, aspect-fit layout, CLI/API handling, hosting integration, screenshot support, documentation, and end-to-end tests.

Changes

WKWebView viewport emulation

Layer / File(s) Summary
Viewport contracts and layout model
Packages/macOS/CmuxBrowser/Sources/CmuxBrowser/BrowserViewport*.swift, Packages/macOS/CmuxBrowser/Tests/CmuxBrowserTests/BrowserViewportLayoutTests.swift
Defines bounded logical dimensions, native/emulated layout calculations, render limits, observable state, restoration policy, snapshot planning, and unit tests.
WKWebView layout and hosting integration
Sources/Panels/WKWebView+BrowserViewport.swift, Sources/Panels/BrowserPanel*.swift, Sources/BrowserWindowPortal.swift, Sources/Panels/CmuxWebView.swift
Propagates viewport state, applies aspect-fit frames and logical bounds, updates hosting and preload paths, and restores viewport-aware geometry during reparenting.
Viewport-aware screenshot pipeline
Sources/Panels/BrowserScreenshotSnapshotter.swift, Sources/Panels/BrowserScreenshot*.swift
Applies viewport geometry during offscreen rendering, derives content metrics, normalizes screenshots, and plans tiled capture geometry.
Browser API and CLI workflow
Sources/TerminalController*.swift, CLI/*.swift, Resources/Localizable.xcstrings, cmux.xcodeproj/project.pbxproj
Enables WKWebView viewport requests and reset handling, centralizes CLI parsing and help text, adds localized validation messages, and registers new sources.
Parity validation and support documentation
tests_v2/*browser*, docs/*, skills/cmux-browser/*
Updates the support contract and documentation, removes the unsupported assertion, and adds end-to-end checks for dimensions, screenshots, focus, zoom, and reset behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant TerminalController
  participant BrowserPanel
  participant WKWebView
  CLI->>TerminalController: Set or reset browser viewport
  TerminalController->>BrowserPanel: Apply requested viewport
  BrowserPanel->>WKWebView: Compute and apply layout
  WKWebView-->>BrowserPanel: Return applied layout
  BrowserPanel-->>TerminalController: Return mode and dimensions
  TerminalController-->>CLI: Output viewport result
Loading

Possibly related PRs

Suggested reviewers: lawrencecchen


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (2 errors, 1 warning)

Check name Status Explanation Resolution
Cmux User-Facing Error Privacy ❌ Error The viewport overflow API error says "WKWebView render limits," exposing an upstream framework name in user-facing error text. Rewrite the localized error to generic browser wording (e.g. "browser render limits") and keep WKWebView references out of user-visible bodies.
Cmux Full Internationalization ❌ Error Fail: the new viewport/CLI localization keys only have en/ja, but the catalog already supports 18 more locales. Add translated entries for every existing locale in Resources/Localizable.xcstrings for each new browser viewport/help/error key.
Docstring Coverage ⚠️ Warning Docstring coverage is 6.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (22 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes implement WKWebView viewport resizing support, reset behavior, docs, and tests requested by issue #8056.
Out of Scope Changes check ✅ Passed The added code, tests, docs, and localization all support the viewport feature and do not show clear unrelated scope creep.
Cmux Swift Actor Isolation ✅ Passed The new viewport model is explicitly @MainActor, and all new accesses occur in MainActor/UI code; no shared Sendable mutable refs or background UI-store access were introduced.
Cmux Swift Blocking Runtime ✅ Passed No new semaphores, sleeps, syncs, or locks were added in the production diff; the only timer usage was preexisting and unchanged.
Cmux Browser Automation Off-Main ✅ Passed viewport set is main-actor geometry only; wait-based browser eval/screenshot/cookies stay in worker router and are policy-covered.
Cmux Expensive Synchronous Load ✅ Passed PASS: The diff is viewport/layout-only; scans found no new RestorableAgentSessionIndex.load(), agent-history parsing, or other expensive sync loaders on main/interactive paths.
Cmux Cache Substitution Correctness ✅ Passed Snapshot sizing reads the event-driven BrowserViewportModel as source of truth, with nil fallback; no stale/cold cache substitution in persisted/history paths found.
Cmux No Hacky Sleeps ✅ Passed The only new waits are time.sleep(0.2) retries in a Python test helper; no covered non-Swift production/runtime files add fixed delays or polling.
Cmux Algorithmic Complexity ✅ Passed New loops are fixed-size or tile-bound; no new scalable rescans/sorts on workspace-like collections, and the subview scan targets a tiny view tree.
Cmux Swift Concurrency ✅ Passed No new background queues, Combine app state, or fire-and-forget Tasks were introduced; the remaining completion handlers wrap WKWebView callback boundaries.
Cmux Swift @Concurrent ✅ Passed No changed code adds misused @concurrent or nonisolated async work; new async screenshot/view helpers are all @MainActor/UI-bound.
Cmux Swift File And Package Boundaries ✅ Passed Viewport math/state moved into small CmuxBrowser types; app-target edits are AppKit/CLI/socket glue with only small deltas in existing large files.
Cmux Swiftpm Lockfiles ✅ Passed Diff only adds source files to cmux.xcodeproj; no SwiftPM package refs, Package.swift, Package.resolved, or .gitignore lockfile changes are present.
Cmux Swift Logging ✅ Passed No added or changed print/NSLog/Logger in production Swift diff; remaining prints are CLI/test output and pre-existing BrowserPanel NSLog sites are untouched.
Cmux Swiftui State Layout ✅ Passed The viewport state uses @Observable BrowserViewportModel; changed SwiftUI code is confined to AppKit bridge hosting, with no new ObservableObject/@published or layout-mutating GeometryReader.
Cmux Architecture Rethink ✅ Passed Clear single source of truth (BrowserViewportModel) with value-layout helpers; no new timing/polling/lock or split-owner repair paths in touched viewport code.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed PASS: The only new NSPanel is an offscreen, non-key render host (cmux.browserVisualAutomationRender), and scripts/lint_auxiliary_window_close_shortcuts.py explicitly ignores that identifier.
Cmux Source Artifacts ✅ Passed All changed paths are intentional source/docs/tests/config/localization files; none match the policy’s artifact or scratch-directory patterns.
Cmux No Test Or Debug Seam In Production Source ✅ Passed No new DEBUG/test-only seam appears in the changed production sources; the new viewport APIs are exercised by BrowserPanel and TerminalController production code.
Cmux No Ambient Global State ✅ Passed No new file-scope API, mutable globals, or runtime singletons were added; viewport state is owned by BrowserPanel.viewportModel and injected into CmuxWebView.
Title check ✅ Passed The title clearly summarizes the main change: adding WKWebView browser viewport support.
Description check ✅ Passed The description covers the summary and validation well, though it does not fully follow the template sections like Demo Video and Review Trigger.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-8056-wkwebview-viewport-resize

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds logical viewport resizing for WKWebView browser surfaces. The main changes are:

  • Per-surface viewport emulation from 1 to 4096 CSS pixels.
  • Aspect-fit presentation across inline, portal, preload, zoom, and screenshot hosts.
  • Exact CSS-pixel screenshot output with bounded full-page tiling.
  • CLI and RPC support for setting and resetting viewport dimensions.
  • Localized errors, help text, documentation, and focused tests.
  • Host-aware screenshot restoration and inspector/fullscreen transitions.

Confidence Score: 5/5

This looks safe to merge.

  • The updated teardown does not reclaim a browser view after a newer host takes ownership.
  • Detached views are not returned to an obsolete host.
  • Normal restoration uses the current host bounds instead of saved pane dimensions.
  • No blocking issue remains in the changed restoration paths.

Important Files Changed

Filename Overview
Sources/Panels/BrowserScreenshotSnapshotter.swift Restores screenshot-hosted browser views according to current host ownership and live container geometry.
Sources/Panels/WKWebView+BrowserViewport.swift Adds shared viewport-host identity, layout projection, and safe restoration helpers.
Packages/macOS/CmuxBrowser/Sources/CmuxBrowser/BrowserViewportRestorationPolicy.swift Defines when temporary screenshot hosting may restore an earlier host or preserve captured geometry.
Packages/macOS/CmuxBrowser/Sources/CmuxBrowser/BrowserViewportLayout.swift Projects logical CSS viewport dimensions into aspect-fitted AppKit frame and bounds geometry.
Sources/Panels/BrowserViewportHostView.swift Coordinates browser hosting while external WebKit layouts temporarily own geometry.

Reviews (26): Last reviewed commit: "fix: keep viewport limit errors browser-..." | Re-trigger Greptile

Comment thread Sources/Panels/BrowserScreenshotSnapshotter.swift Outdated
Comment thread Sources/Panels/WKWebView+BrowserViewport.swift Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests_v2/test_browser_cli_agent_port.py`:
- Around line 320-328: The screenshot tests around the initial and zoomed
viewport paths create temporary directories with tempfile.mkdtemp() without
cleanup. Replace both temporary-directory allocations with
tempfile.TemporaryDirectory() context-managed usage, and construct each
screenshot path from the managed directory so cleanup occurs automatically after
the test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 48af6c2b-0e69-49bc-9e75-9d27b7026b13

📥 Commits

Reviewing files that changed from the base of the PR and between 5be4c07 and 8c47a2d.

📒 Files selected for processing (16)
  • CLI/CMUXCLI+BrowserViewport.swift
  • CLI/cmux.swift
  • Packages/macOS/CmuxBrowser/Sources/CmuxBrowser/BrowserViewportLayout.swift
  • Packages/macOS/CmuxBrowser/Tests/CmuxBrowserTests/BrowserViewportLayoutTests.swift
  • Resources/Localizable.xcstrings
  • Sources/Panels/BrowserPanel+AutomationViewport.swift
  • Sources/Panels/BrowserPanel.swift
  • Sources/Panels/BrowserScreenshotSnapshotter.swift
  • Sources/Panels/CmuxWebView.swift
  • Sources/Panels/WKWebView+BrowserViewport.swift
  • Sources/TerminalController+BrowserViewport.swift
  • cmux.xcodeproj/project.pbxproj
  • docs/cli-contract.md
  • skills/cmux-browser/SKILL.md
  • skills/cmux-browser/references/commands.md
  • tests_v2/test_browser_cli_agent_port.py

Comment thread tests_v2/test_browser_cli_agent_port.py Outdated
@cursor

cursor Bot commented Jul 14, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
Sources/Panels/BrowserPanel+AutomationViewport.swift (1)

18-24: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Keep viewport geometry on one authoritative source of truth.

These paths guess dimensions from WebView/window geometry or hardcoded defaults when reliable layout state is unavailable. That can apply the wrong layout and misreport screenshot/API viewport dimensions.

  • Sources/Panels/BrowserPanel+AutomationViewport.swift#L18-L24: defer layout until a real host container exists, or fail closed.
  • Sources/Panels/BrowserPanel+AutomationViewport.swift#L40-L55: derive the visual size from the viewport/layout model, not window bounds or 1280x720.
  • Sources/Panels/BrowserPanel+AutomationViewport.swift#L58-L63: remove the arbitrary 800x600 container fallback.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/Panels/BrowserPanel`+AutomationViewport.swift around lines 18 - 24,
Keep viewport geometry authoritative across
Sources/Panels/BrowserPanel+AutomationViewport.swift:18-24, deferring layout
until a real host container exists or failing closed; at
Sources/Panels/BrowserPanel+AutomationViewport.swift:40-55, derive visual
dimensions from the viewport/layout model instead of window bounds or 1280x720;
and at Sources/Panels/BrowserPanel+AutomationViewport.swift:58-63, remove the
arbitrary 800x600 fallback. Update the affected automation viewport layout and
sizing flow without introducing alternate geometry sources.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@Sources/Panels/BrowserPanel`+AutomationViewport.swift:
- Around line 18-24: Keep viewport geometry authoritative across
Sources/Panels/BrowserPanel+AutomationViewport.swift:18-24, deferring layout
until a real host container exists or failing closed; at
Sources/Panels/BrowserPanel+AutomationViewport.swift:40-55, derive visual
dimensions from the viewport/layout model instead of window bounds or 1280x720;
and at Sources/Panels/BrowserPanel+AutomationViewport.swift:58-63, remove the
arbitrary 800x600 fallback. Update the affected automation viewport layout and
sizing flow without introducing alternate geometry sources.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 975f4c97-95e9-4d97-bf58-50402af60efc

📥 Commits

Reviewing files that changed from the base of the PR and between 3922fb1 and c134e10.

📒 Files selected for processing (2)
  • Sources/Panels/BrowserPanel+AutomationViewport.swift
  • tests_v2/test_browser_cli_agent_port.py

@cursor

cursor Bot commented Jul 14, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@austinywang
austinywang merged commit 08c9f5d into main Jul 15, 2026
6 checks passed
hhsw2015 pushed a commit to hhsw2015/cmux that referenced this pull request Jul 16, 2026
* test(browser): cover WKWebView viewport emulation

* feat(browser): emulate WKWebView viewports

* docs(browser): explain WKWebView viewport sizing

* fix(browser): preserve viewport across zoom and capture

* fix(browser): close viewport review findings

* test(browser): preserve reconciled host after capture

* fix(browser): preserve current host after capture

* test(browser): clean up viewport screenshots

* fix(browser): import viewport geometry types

* test(browser): cover viewport scale boundaries

* fix(browser): bound and normalize viewport rendering

* test(browser): cover viewport safety policies

* fix(browser): bound viewport capture and inspector transitions

* test(browser): cover inspector and zoomed full-page transitions

* fix(browser): reconcile inspector and snapshot coordinates

* test(browser): cover fullscreen and native zoom capture

* fix(browser): reconcile fullscreen and native zoom capture

* test(browser): assert runtime viewport dimensions

* test(browser): gate runtime viewport regression

* test(browser): await runtime viewport metrics

* test: cover native viewport rounding at page zoom

* fix: match native viewport rounding to WebKit

* fix: preserve browser viewport presentation geometry

* test: cover emulated viewport autoresizing mask

* fix: disable raw viewport autoresizing while emulated

* test: cover emulated viewport precision at page zoom

* fix: preserve emulated viewport precision at page zoom

* test: cover emulated viewport precision at default zoom

* fix: preserve emulated viewport precision at default zoom

* test: reject stale detached browser host restoration

* fix: leave detached browser hosts unowned after capture

* refactor: split browser viewport support types

* test: report browser zoom render limits

* fix: report browser zoom render limits

* docs: keep viewport mode docs on public symbol

* test: keep native browsers outside viewport host

* fix: activate browser viewport host lazily

* test: keep native portal layout passive

* fix: keep native browser portal layout passive

* test: preserve fractional zoom viewport dimensions

* fix: preserve exact viewport dimensions at page zoom

* test: cover near-integer zoom viewport rounding

* ci: run CmuxBrowser package tests

* fix: preserve near-integer zoom viewport dimensions

* fix: keep viewport limit errors browser-generic

(cherry picked from commit 08c9f5d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support resizing WKWebView-backed browser surfaces

1 participant