AetherVSR

Local GPU video super-resolution, with an Electron/WebGPU desktop player and a native Metal/AVFoundation research player. The production C16D2 model performs 2x upscaling on your machine; local video is not uploaded for processing.

The primary product is AetherVSR Desktop. Its local 720p30/60 H.264 MVP is qualified on the measured macOS / Apple Silicon setup. The native Metal path and browser extension have separate, narrower qualification limits.

The latest native binding baseline30 trial ran with an occluded window on M5: 110 useful presentation opportunities in 60.033667 seconds, 1.832 FPS overall, 1.75 FPS in the final 20 seconds, and software-age p95 981.263 ms. It failed the registered gates. Foreground performance, neural30, all 60 FPS timing and the ten-minute soak remain unqualified or NOT_RUN; no favorable rerun was performed. Both native test clips lack audio tracks, so audible playback and A/V synchronization are not qualified by those tests. No Phase 3 or capture integration has started. NVIDIA/AMD/Intel hardware remains unqualified.

Benchmarks and measurement scopes | Roadmap | Architecture | Engineering rules

Commands run from the repository root. These are source-build workflows; there are no published release binaries or installers.

Requires Node.js 22.12.0 or newer. The qualified setup is macOS on Apple Silicon; other desktop platforms are not hardware-qualified.

npm ci
npm run desktop:dev

The command builds and launches the Electron player. Select a local H.264/MP4; the committed 30 FPS and 60 FPS clips match the tested 720p scope. Desktop behavior and limits.

Requires macOS 26 or newer and an Xcode toolchain with the macOS 26 SDK or newer. The measured hardware is Apple M5. Run the separate Swift package:

swift run --package-path native -c release aether-player

Use Open Video to select a local 720p H.264/MP4. This is the partial native research path, not a real-time-qualified replacement for the desktop player. It needs no Screen Recording or Accessibility permission. Native playback scope.

Requires Node.js 22.12.0+ and a Chromium browser with WebGPU.

npm ci
npm run dev

Open the printed URL. A bundled clip loads automatically; the clip picker accepts local video. The harness is a development and measurement surface, not the primary product. Unpacked extension setup is documented below with its compatibility limits.

Measured on 17 independently sourced captured clips that share no creator with the training corpus, under controlled 720p H.264, against the production Catmull-Rom baseline this project ships. The set was frozen before the model was trained and read after model selection closed.

Retraining on a corpus of 151 clips from 108 creators — against 12 clips from 8 creators — improved every compression tier at identical runtime. The inference graph is unchanged; only the weights differ.

What it does not support. The gain is mostly training length, not corpus size: at equal optimizer updates, twelve times the data is worth only +0.10 dB and that is not statistically established. Two content classes are negative at heavy compression (motion −0.09 dB, texture −0.01 dB, both on two clips). The model remains less temporally stable than Catmull-Rom. Faces are measured only on a narrow eight-clip set, of which three survive a creator-disjoint filter. Most per-class results rest on two clips and are indicative only.

So: a real, reproducible improvement across the compression range on independently sourced footage, with the caveat that more data is no longer the lever it looked like. BENCHMARKS.md gives the full evidence, including every withdrawn and corrected claim.

Quality claims are separated into three categories that are never averaged together: regression artefacts (the synthetic reference and the eight-image natural set, both overlapping the training corpus), source-disjoint validation and test splits, and a held-out independent corpus from a different institution. Only the last supports a generalisation claim.

The dataset is split by source image, not by patch, and CI enforces that no photograph appears in two splits. Training never reads the test split. See DECISIONS.md ADR-0028 and ADR-0029, and the Milestone 4.5 section of BENCHMARKS.md, which also documents two defects in the Milestone 4 evaluation and what was rerun because of them.

• The Electron desktop player, standalone harness and browser extension reuse the production WebGPU pipeline and C16D2 model. The native Metal player is a separate implementation with independently checked numerical parity.
• Desktop and harness video is decoded through an HTMLVideoElement. The extension preserves page playback and reveals the original video when enhancement is unsupported.
• requestVideoFrameCallback() as the frame clock, so work is synchronised to presented video frames rather than to display refresh.
• GPUDevice.importExternalTexture() for the frame import, with a copyExternalImageToTexture() fallback that can be forced for testing.
• Exact 2x output: 1280x720 in, 2560x1440 out, always.
• Production neural upscaling uses the 6,291-parameter C16D2 model, with runtime performance policy and explicit baseline controls.
• Two non-neural GPU scalers: hardware bilinear, and a 9-tap bilinear-fused Catmull-Rom bicubic.
• A diagnostic overlay with real GPU timings where timestamp-query is available, and an explicit "not measured" where it is not.
• No CPU pixel readback anywhere in the frame loop.

These earlier harness measurements are not M13 native-playback results. Measured on a MacBook Pro (Apple M5, 24 GB), Chrome for Testing 152, 720p60 H.264 → 1440p:

30 s run; FPS are means over the run, GPU timings aggregate the trailing 240 samples (~4 s at 60 fps). 1779 and 1786 frames upscaled respectively, with 13 and 6 presented frames skipped. Full environment, method, caveats and the things that were not measured are in BENCHMARKS.md. Numbers here are never estimates — see AGENTS.md §2.

The web harness and extension require WebGPU. Tested Chromium builds and hardware are recorded in BENCHMARKS.md; Safari and Firefox are not qualified. Without requestVideoFrameCallback, the harness uses an explicitly degraded animation-frame clock, not an equivalent timing source.

Record focus and visibility while measuring. Backgrounded or occluded windows can throttle frame callbacks. A failed or backgrounded run is not discarded or relabeled as a foreground performance result.

Run these commands separately from the repository root:

npm ci
npm run build:extension

The build generates dist-extension. In Chrome, open chrome://extensions, turn on Developer mode, select Load unpacked, and choose that directory. On an HTTP(S) page, open the native extension action and select Enable current page. Mode is saved per origin; activation is per document, not a persistent site grant. Navigation to a new document requires explicit activation again.

M10 was tested with Chrome for Testing 153.0.8010.12 on an Apple M5. Its three permissions are activeTab, scripting and storage: no permanent host access, web-accessible resources (WAR), or MAIN-world execution. Model delivery is extension-owned; page scripts receive no privileged bridge.

Supported scope: the top document and accessible open shadow roots, with supported visible geometry. Iframes and closed shadow roots are outside scope. Native video controls and showing native text tracks are unsupported. Native PiP and direct-video fullscreen reveal the original video; supported container fullscreen can retain enhancement. Unsupported geometry or media security also leaves original playback available. The page remains authoritative for playback, audio, source, seeking and controls: no page-owned node, style or media attribute is modified, replaced or reparented; only extension-owned DOM is added and removed.

The production extension payload remains byte-identical to the M13 Phase 2 freeze: SHA256 46cebd53b7665ce48772792d2ec2eb73057e915d93ef76374900f30a159ac551. The unchanged 6,291-parameter C16D2 model SHA256 is d76fae7a295cdcdaecb44e39f8c87ff68a59ca1e07fc7cfc347d252d3cad358a. Payload verification. This is a load-unpacked MVP, not store publication or broad player compatibility.

Each of these is a separate command; run them individually.

npm ci
npm run dev
npm run typecheck
npm run lint
npm run test
npm run build

http://127.0.0.1:5173/?clip=/media/aethervsr-testclip-720p60-h264.mp4&filter=catmull-rom

http://localhost:5173/bench.html is a separate entry point holding the Milestone 2 experiments. It is not part of the baseline harness — the ONNX Runtime probe alone fetches a ~26 MB WASM artefact, which is why the two are separate bundles.

Everything is driven from the console so a run is reproducible:

await window.aethervsrEnvironment();
await window.aethervsrIngestBench({ mode: 'ingest', filter: 'catmull-rom', ingestFormat: 'rgba8unorm' }, 3000, 15000);
await window.aethervsrConvVerify([{ width: 17, height: 13, inChannels: 3, outChannels: 4, blockX: 2, activation: 'none', residual: false }]);
await window.aethervsrConvBench([{ label: '16->16', width: 1280, height: 720, inChannels: 16, outChannels: 16, tileX: 8, tileY: 8, blockX: 4, activation: 'relu', useF16: true, residual: false }]);
await window.aethervsrQualityBench(['bilinear', 'catmull-rom']);
await window.aethervsrOrtProbe({ modelUrl: '/models/conv16-720p.onnx', channels: 16, height: 720, width: 1280, iterations: 20, providers: ['webgpu'], outputLocation: 'cpu' });

Keep the window frontmost for aethervsrIngestBench: it is driven by requestVideoFrameCallback, which a backgrounded tab suspends. The harness throws rather than returning an empty result if that happens.

Regenerate the ONNX benchmark fixture with node tools/make-onnx-conv.mjs 16 public/models/conv16-720p.onnx 720 1280.

public/media/ holds four clips (1280x720, at 30 and 60 fps, in VP9/WebM and H.264/MP4). The drawing is a pure function of frame index: a 1-to-16 px frequency wedge, a radial zone plate, thin diagonals, small text, hard edges and continuous sub-pixel motion — chosen to make resampling errors visible.

All four clips are original content generated by this repository's own tool from a procedural drawing — no third-party video, imagery or audio is included — so they are covered by the project's Apache-2.0 licence and are safe to redistribute.

The committed binaries are the benchmark reference, not the recipe. Regenerating on another machine will not be bit-identical: Canvas2D text rasterisation depends on the font stack, and encoding depends on the browser build and refresh rate.

Regenerate with tools/make-test-clip.html, served by the dev server:

http://127.0.0.1:5173/tools/make-test-clip.html?fps=60&seconds=4&codec=h264

Press record, then save the download into public/media/. The generator is paced on requestAnimationFrame for a reason — see DECISIONS.md ADR-0009.

npm test covers metrics, frame clocks, model and shader contracts, runtime policy, desktop/extension ownership and the evidence checkers. CI also runs Python experiment-integrity tests and both Swift packages' build/CPU contracts. CPU and source-text tests do not establish GPU pixel correctness or performance.

At the M13 Phase 2 closure: 1,504 Vitest tests passed, with three explicit skips; 612 Python tests, 80 opt-in fixture tests, and 71 desktop tests passed. The desktop and opt-in groups overlap the main suite and are not additive. Physical native suites passed 18 original Metal tests and 7 playback tests. Hosted native CI skips GPU-only tests; set AETHERVSR_METAL_TESTS=1 only for an intentional physical-device run. Verification and scopes.

Hardware correctness uses golden tensors, cross-backend comparisons and same-decoded-frame CPU/GPU parity with retained numerical artifacts. Native performance, lifecycle and long-run stability are separate acceptance gates.

Resource-lifetime and control contracts use mocked WebGPU objects in unit tests. Those tests check allocation, ownership and control flow, not hardware pixel correctness or performance. Real browser/GPU evidence is separate: M10 passed 3/3 production external-import and 3/3 test sampled-import parity cases with exact input and output RGBA hashes against the same harness within each route. These were paused, instrumented replays, not SR-quality or real-time performance measurements. Native extension journeys and live timing runs supply different evidence; their scopes and unresolved gates are in BENCHMARKS.md.

apps/desktop/                  Electron desktop player and build tooling
native/
  Package.swift                macOS 26 playback package
  macos/                       frozen original/optimized Metal engines and tests
  playback/                    Core Video bridge, F adapter, native player and tests
src/
  core/                        shared WebGPU pipeline, upscalers and runtime policy
  extension/                   preserved MV3 browser extension
  main.ts                      standalone harness entry
  ui/                          harness controls and metrics
public/models/                 production model and golden reference tensors
public/media/                  committed test clips
test/                          TypeScript contracts and evidence-checker tests
tools/                         experiment, training, fixture and validation tooling
tools/m13/                     Metal and native-playback evidence runners/checkers
docs/                          phase protocols, reports and dataset documentation
results/                       compact, source-bound evidence indexes
index.html                     web harness
bench.html                     isolated WebGPU feasibility experiments

AetherVSR is independently engineered. Other projects were read for architectural ideas and are cited; no third-party source was copied. Licences were established by reading each repository rather than assumed — the verdict table is in DECISIONS.md ADR-0010.

Of note: no LICENSE file was present in the fishy-ops/WebVSR snapshot we reviewed, and the GitHub API reported license: null. We treated it as unlicensed for this project and copied nothing from it; anyone reusing it should verify the current state themselves.

M10.6 public-test tooling uses m3u8-parser 7.2.0 (Apache-2.0), verified from the installed package's LICENSE. It parses already observed HLS master responses for catalog identity only. It is a development dependency, excluded from the production extension and shared frame-processing path; upstream notices remain in the installed package. No third-party implementation source was copied.

M10.9 offline screenshot analysis uses the existing Python environment's Pillow 12.3.0 and ImageCms (MIT-CMU), verified from its installed license metadata and LICENSE. It interprets embedded ICC profiles outside the frame loop. The package and its notices remain installed separately; no Pillow source is copied, bundled in the extension, or newly added to the JavaScript dependencies.

Selected references:

• requestVideoFrameCallback — https://wicg.github.io/video-rvfc/
• GPUExternalTexture — https://gpuweb.github.io/gpuweb/#gpuexternaltexture
• Media Playback Quality — https://w3c.github.io/media-playback-quality/
• WebVSR (WebGPU/WGSL SPAN-Lite browser SR) — https://github.com/fishy-ops/WebVSR
• StreamSR / EfRLFN, ICLR 2026 — https://arxiv.org/abs/2602.11339
• SPAN — https://github.com/hongyuanyu/SPAN (Apache-2.0)
• RLFN — https://github.com/bytedance/RLFN (Apache-2.0)
• ONNX Runtime Web, WebGPU EP — https://onnxruntime.ai/docs/tutorials/web/ep-webgpu.html

Apache-2.0.