Skip to content

perf: reuse compiled code across vm pool contexts and prewarm the module graph#10744

Open
sheremet-va wants to merge 3 commits into
mainfrom
perf/vm-pools
Open

perf: reuse compiled code across vm pool contexts and prewarm the module graph#10744
sheremet-va wants to merge 3 commits into
mainfrom
perf/vm-pools

Conversation

@sheremet-va

@sheremet-va sheremet-va commented Jul 8, 2026

Copy link
Copy Markdown
Member

Description

vm pools isolate test files with a fresh VM context per run request, which means every file re-compiles and re-evaluates its entire module graph — including externalized node_modules and the Vitest runtime itself. Profiling jsdom + vmThreads runs of fixtures with real dependencies (lodash, lodash-es, date-fns, moment, rxjs, zod, react, react-dom/server) showed three costs this PR removes:

  1. Re-compilation per context. Compiled code has no per-context state — only its evaluation does. This PR shares it per worker:

    • inlined modules: compiled vm.Script instances are shared across contexts (safe: Vite rewrites dynamic imports to __vite_ssr_dynamic_import__, so no per-context import callback is baked into the script);
    • externalized CJS: per-worker V8 cachedData, produced after the first execution so it carries the compiled module body (the vm.Script itself stays per-context because it embeds the per-context importModuleDynamically closure);
    • externalized ESM: SourceTextModule cachedData, produced before evaluation as V8 requires.

    Entries are keyed by module id and guarded by the exact source text, so invalidated modules replace their entry, and mock: ids never collide with their originals.

  2. An idle server during worker startup. On a cold start every vm worker spends the first ~0.5s of its life importing the environment package (jsdom), while the server has nothing to do; the file's transforms only start after that, serially. vm workers now fire a prewarmModuleGraph RPC before environment.setupVM: the server BFS-transforms the file's import graph through the same fetcher as live fetches during that window. Since the prewarmed transforms set __vitestTmp, the worker's fetchWarmModules snapshot then covers the graph even on the first cold run, and its fetches short-circuit to local disk reads. Prewarm failures are ignored — the worker's own fetch reports errors with the proper import context.

  3. Re-resolution of every import edge, per context. ExternalModulesExecutor.resolve calls import.meta.resolve for every ESM import edge of every externalized module, and Node re-derives the resolved module's package scope on each uncached call — re-parsing large exports maps along the way (date-fns's package.json is 199KB with 741 export entries). Node's own resolution cache is keyed by (specifier, parentURL), so each worker still pays the full external graph once (~3300 calls ≈ 700ms of a 9-worker profile on the deps fixture, ×9 workers). Two changes:

    • relative specifiers (restricted to a conservative charset — anything URL-special falls back to Node) resolve by plain URL join. This is exactly what import.meta.resolve returns for them: relative ESM specifiers never consult package.json, and Node's resolver doesn't check existence either — the executor already recreates ERR_MODULE_NOT_FOUND itself;
    • bare specifiers get a worker-wide resolution cache in front of import.meta.resolve (behind the custom vite/mock resolvers, which are unaffected), and module type/existence detection is memoized per worker. Both follow the same worker-lifetime staleness invariants as the existing fileMap/packageCache.

Isolation semantics are unchanged: evaluation still happens once per context, verified by an e2e test that fails if module state leaks through the shared scripts.

Benchmarks (jsdom, vmThreads, isolate: true, fs module cache on; cold = all caches wiped before every rep, warm = populated caches; median of 3 interleaved reps in a single window, M4/Node 24.13, whole-process wall clock; trivial assertions):

fixture cold warm
342 modules + real deps (lodash, lodash-es, date-fns, react, react-dom/server), 40 test files 2605ms → 1805ms (−31%) 2556ms → 1710ms (−33%)
1022 modules + heavier deps (adds moment, rxjs, zod), 160 test files 7078ms → 5127ms (−28%) 7010ms → 5149ms (−27%)
176 modules, no npm deps, 40 test files 1210ms → 1135ms (−6%) 1200ms → 1136ms (−5%)

vmForks tracks vmThreads (1022-module fixture: 6528ms → 5296ms cold, −19% against the resolution change alone). The split matches the mechanisms: fixtures with externalized dependencies win from the resolution fast path and compile reuse (the costs scale with the external graph), while the no-deps fixture only collects the transform/jsdom-import overlap.

Transform time reporting

The prewarm broke the reported transform time, and investigating that exposed that the stat was already unreliable: it summed each fetch's wall time, but concurrent fetches (parallel workers, and now the prewarm) all wait on the same deduplicated in-flight transforms, so the sum counts the same work many times. Measured on the deps fixture (cold vmThreads run, ~2.5s wall; serial --maxWorkers=1 reference: 356ms of real transform work):

strategy reported "transform"
summed fetch walls, with prewarm 348–495s
summed walls deduplicated to the first in-flight caller 61–82s (a wide BFS still measures queue position, not work)
summed fetch walls on main today, no prewarm, default workers 2.16s on a 2.52s run; warm runs report fs-cache reads as 2.4s of "transform"
this PR: pipeline busy time 1.11s cold, 0ms warm

transformTime is now the union of in-flight transform intervals, measured inside the fetcher around the actual transform (fetchAndProcess) — cache hits and fs-cache reads don't tick the clock. It has a crisp meaning ("wall time during which the server was transforming"), is bounded by the run's duration, and behaves identically with and without the prewarm. Warm runs with a populated fsModuleCache now truthfully report ~0 transform time. Per-module durations (metadata.duration, used by the UI module graph) are still recorded from direct worker fetches only — the prewarm's per-module walls would measure BFS queue position rather than the module's own cost.

@netlify

netlify Bot commented Jul 8, 2026

Copy link
Copy Markdown

Deploy Preview for vitest-dev ready!

Name Link
🔨 Latest commit d320ce8
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/6a4e8325f5986c000847217e
😎 Deploy Preview https://deploy-preview-10744--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@sheremet-va

Copy link
Copy Markdown
Member Author

The same prewarm trick doesn't work with treads/forks because we need to reevaluate the environment for every worker. Here we spawn the workers -> import jsdom in parallel in X workers once -> prewarm every test file

@sheremet-va

Copy link
Copy Markdown
Member Author

The transform changes affects how it's shown in #10710

* deduplicated in-flight transforms, so per-caller wall times overcount the
* actual work by orders of magnitude.
*/
export interface TransformClock {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in short, we calculate the REAL time it takes to transform. instead of summing individual times

@AriPerkkio AriPerkkio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the new transform time measuring also remove time that workers idle when waiting for server to handle their request? Transform time is now measured on main thread side, right?

the server BFS-transforms the file's import graph

What's BFS?

Comment on lines +28 to +35
// Compiled scripts of inlined modules, shared across vm contexts: vm pools
// evaluate every module again in each fresh context, but the compiled script
// holds no per-context state (Vite rewrites dynamic imports to
// `__vite_ssr_dynamic_import__`, so no per-context import callback is baked
// in) — only its evaluation has to happen per context. Keyed by module id
// (`mock:` ids stay distinct from their originals) and guarded by the exact
// code, so invalidated modules replace their entry.
const vmInlineScriptCache = new Map<string, { code: string; script: vm.Script }>()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this increase memory usage a lot? It's kind of introducing memory leak for every script that was evaluated.

@sheremet-va sheremet-va Jul 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I will validate before merging. I think that it's fine since tests usually import the same files anyway. At worst, this holds every source file (which is already just one test file that imports a barrel file)

These are also just strings, not actual export values

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, caching just the source strings shouldn't be as bad as the og vm.Script memory leaks of executed context. But good to check it before merge.

@hi-ogawa hi-ogawa Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I'd add is that, these script/code per file are retained for each worker (thread/fork), so if the same module is evaluated in different worker, then these in-memory cache is also duplicated. So multiplicative factor also depends on maxWorkers.

The analogy I'm imagining is that, basically our main process already have code-level module graph of entire test run. The cache retained in worker runtime side can potentially multiply that by maxWorkers depending on tests and imported modules distribution.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counter point on my own comment though is that, retaining cache/modules in this way on worker runtime side is how forks/threads + no-isolate boosts perf, so the design feels sound.

@sheremet-va

sheremet-va commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

BFS

Breadth-first search, as apposed to DFS (depth-first search). Do they not ask you this in the interviews :O

@AriPerkkio

Copy link
Copy Markdown
Member

Breadth-first search, as apposed to DFS (depth-first search). Do they not ask you this in the interviews :O

Nope, never heard of these terms. I don't think I've ever run into this term in my ~10 year career, or in the engineering studies before that. Happy to ask all the stupid questions out loud. 😇

@sheremet-va

Copy link
Copy Markdown
Member Author

Breadth-first search, as apposed to DFS (depth-first search). Do they not ask you this in the interviews :O

Nope, never heard of these terms. I don't think I've ever run into this term in my ~10 year career, or in the engineering studies before that. Happy to ask all the stupid questions out loud. 😇

These algorithms have been following me for years now 😢

@hi-ogawa

Copy link
Copy Markdown
Collaborator

Haven't read but my review agent said more PRs to split.

The PR could reasonably be split into:

  1. Reuse compiled inlined and externalized VM code. These could be one PR or two because they use different cache mechanisms.
  2. Cache external-module resolution and module information.
  3. Prewarm VM module graphs, including warm snapshots, the RPC contract, error-path coverage, and transform-time accounting.

@sheremet-va

Copy link
Copy Markdown
Member Author

Haven't read but my review agent said more PRs to split.

The PR could reasonably be split into:

  1. Reuse compiled inlined and externalized VM code. These could be one PR or two because they use different cache mechanisms.
  2. Cache external-module resolution and module information.
  3. Prewarm VM module graphs, including warm snapshots, the RPC contract, error-path coverage, and transform-time accounting.

I'd argue the PR is small enough to review and merge.

@hi-ogawa hi-ogawa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vm resolve/script/source caching looks good!
Now following on prewarm story.

cachedData,
importModuleDynamically: options.importModuleDynamically,
} as any)
if (cachedData && (script as any).cachedDataRejected) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type seems available

Suggested change
if (cachedData && (script as any).cachedDataRejected) {
if (cachedData && script.cachedDataRejected) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yeah, AI loves this

@hi-ogawa hi-ogawa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vm prewarm idea sounds good, but the reason on why vm only seems to assume heavy per vm environment setup. Should we also measure bench for non jsdom/happy-dom case i.e. default node environment?

Labeling "request changes" on that ground and also vm worker side memory retain justification.

return err instanceof Error && 'errors' in err
}

export class StateManager {

@hi-ogawa hi-ogawa Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was lost how the clock API is connected. Does doing this hurt? (this allows IDE to link transformStarted/finished)

import type { TransformClock } from './environments/fetchModule'

export class StateManager implements TransformClock {

@sheremet-va

sheremet-va commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

The vm prewarm idea sounds good, but the reason on why vm only seems to assume heavy per vm environment setup. Should we also measure bench for non jsdom/happy-dom case i.e. default node environment?

Labeling "request changes" on that ground and also vm worker side memory retain justification.

I did measure it for every combination. It makes sense to do only if we spawn non isolated workers that import jsdom once (which is the bottleneck - it takes 500ms per worker TO IMPORT. Setting up is cheap). Node doesn’t import anything and happy-dom could benefit from it, but it’s just 200ms.

In short, wamp up only makes sense in jsdom+vm combination and potentially happy-dom+vm.

The warmup only works because the main thread doesn’t do anything otherwise (it just waits for the jsdom to be imported in every worker in parallel).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants