perf: reuse compiled code across vm pool contexts and prewarm the module graph#10744
perf: reuse compiled code across vm pool contexts and prewarm the module graph#10744sheremet-va wants to merge 3 commits into
Conversation
✅ Deploy Preview for vitest-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
The same prewarm trick doesn't work with treads/forks because we need to reevaluate the environment for every worker. Here we spawn the workers -> import jsdom in parallel in X workers once -> prewarm every test file |
|
The transform changes affects how it's shown in #10710 |
| * deduplicated in-flight transforms, so per-caller wall times overcount the | ||
| * actual work by orders of magnitude. | ||
| */ | ||
| export interface TransformClock { |
There was a problem hiding this comment.
in short, we calculate the REAL time it takes to transform. instead of summing individual times
AriPerkkio
left a comment
There was a problem hiding this comment.
Does the new transform time measuring also remove time that workers idle when waiting for server to handle their request? Transform time is now measured on main thread side, right?
the server BFS-transforms the file's import graph
What's BFS?
| // Compiled scripts of inlined modules, shared across vm contexts: vm pools | ||
| // evaluate every module again in each fresh context, but the compiled script | ||
| // holds no per-context state (Vite rewrites dynamic imports to | ||
| // `__vite_ssr_dynamic_import__`, so no per-context import callback is baked | ||
| // in) — only its evaluation has to happen per context. Keyed by module id | ||
| // (`mock:` ids stay distinct from their originals) and guarded by the exact | ||
| // code, so invalidated modules replace their entry. | ||
| const vmInlineScriptCache = new Map<string, { code: string; script: vm.Script }>() |
There was a problem hiding this comment.
Does this increase memory usage a lot? It's kind of introducing memory leak for every script that was evaluated.
There was a problem hiding this comment.
Good question, I will validate before merging. I think that it's fine since tests usually import the same files anyway. At worst, this holds every source file (which is already just one test file that imports a barrel file)
These are also just strings, not actual export values
There was a problem hiding this comment.
Yep, caching just the source strings shouldn't be as bad as the og vm.Script memory leaks of executed context. But good to check it before merge.
There was a problem hiding this comment.
One thing I'd add is that, these script/code per file are retained for each worker (thread/fork), so if the same module is evaluated in different worker, then these in-memory cache is also duplicated. So multiplicative factor also depends on maxWorkers.
The analogy I'm imagining is that, basically our main process already have code-level module graph of entire test run. The cache retained in worker runtime side can potentially multiply that by maxWorkers depending on tests and imported modules distribution.
There was a problem hiding this comment.
Counter point on my own comment though is that, retaining cache/modules in this way on worker runtime side is how forks/threads + no-isolate boosts perf, so the design feels sound.
Breadth-first search, as apposed to DFS (depth-first search). Do they not ask you this in the interviews :O |
Nope, never heard of these terms. I don't think I've ever run into this term in my ~10 year career, or in the engineering studies before that. Happy to ask all the stupid questions out loud. 😇 |
These algorithms have been following me for years now 😢 |
|
Haven't read but my review agent said more PRs to split.
|
I'd argue the PR is small enough to review and merge. |
hi-ogawa
left a comment
There was a problem hiding this comment.
Vm resolve/script/source caching looks good!
Now following on prewarm story.
| cachedData, | ||
| importModuleDynamically: options.importModuleDynamically, | ||
| } as any) | ||
| if (cachedData && (script as any).cachedDataRejected) { |
There was a problem hiding this comment.
The type seems available
| if (cachedData && (script as any).cachedDataRejected) { | |
| if (cachedData && script.cachedDataRejected) { |
There was a problem hiding this comment.
oh yeah, AI loves this
hi-ogawa
left a comment
There was a problem hiding this comment.
The vm prewarm idea sounds good, but the reason on why vm only seems to assume heavy per vm environment setup. Should we also measure bench for non jsdom/happy-dom case i.e. default node environment?
Labeling "request changes" on that ground and also vm worker side memory retain justification.
| return err instanceof Error && 'errors' in err | ||
| } | ||
|
|
||
| export class StateManager { |
There was a problem hiding this comment.
Was lost how the clock API is connected. Does doing this hurt? (this allows IDE to link transformStarted/finished)
import type { TransformClock } from './environments/fetchModule'
export class StateManager implements TransformClock {
I did measure it for every combination. It makes sense to do only if we spawn non isolated workers that import jsdom once (which is the bottleneck - it takes 500ms per worker TO IMPORT. Setting up is cheap). Node doesn’t import anything and happy-dom could benefit from it, but it’s just 200ms. In short, wamp up only makes sense in jsdom+vm combination and potentially happy-dom+vm. The warmup only works because the main thread doesn’t do anything otherwise (it just waits for the jsdom to be imported in every worker in parallel). |
Description
vm pools isolate test files with a fresh VM context per run request, which means every file re-compiles and re-evaluates its entire module graph — including externalized
node_modulesand the Vitest runtime itself. Profiling jsdom +vmThreadsruns of fixtures with real dependencies (lodash, lodash-es, date-fns, moment, rxjs, zod, react, react-dom/server) showed three costs this PR removes:Re-compilation per context. Compiled code has no per-context state — only its evaluation does. This PR shares it per worker:
vm.Scriptinstances are shared across contexts (safe: Vite rewrites dynamic imports to__vite_ssr_dynamic_import__, so no per-context import callback is baked into the script);cachedData, produced after the first execution so it carries the compiled module body (thevm.Scriptitself stays per-context because it embeds the per-contextimportModuleDynamicallyclosure);SourceTextModulecachedData, produced before evaluation as V8 requires.Entries are keyed by module id and guarded by the exact source text, so invalidated modules replace their entry, and
mock:ids never collide with their originals.An idle server during worker startup. On a cold start every vm worker spends the first ~0.5s of its life importing the environment package (jsdom), while the server has nothing to do; the file's transforms only start after that, serially. vm workers now fire a
prewarmModuleGraphRPC beforeenvironment.setupVM: the server BFS-transforms the file's import graph through the same fetcher as live fetches during that window. Since the prewarmed transforms set__vitestTmp, the worker'sfetchWarmModulessnapshot then covers the graph even on the first cold run, and its fetches short-circuit to local disk reads. Prewarm failures are ignored — the worker's own fetch reports errors with the proper import context.Re-resolution of every import edge, per context.
ExternalModulesExecutor.resolvecallsimport.meta.resolvefor every ESM import edge of every externalized module, and Node re-derives the resolved module's package scope on each uncached call — re-parsing largeexportsmaps along the way (date-fns's package.json is 199KB with 741 export entries). Node's own resolution cache is keyed by(specifier, parentURL), so each worker still pays the full external graph once (~3300 calls ≈ 700ms of a 9-worker profile on the deps fixture, ×9 workers). Two changes:import.meta.resolvereturns for them: relative ESM specifiers never consult package.json, and Node's resolver doesn't check existence either — the executor already recreatesERR_MODULE_NOT_FOUNDitself;import.meta.resolve(behind the custom vite/mock resolvers, which are unaffected), and module type/existence detection is memoized per worker. Both follow the same worker-lifetime staleness invariants as the existingfileMap/packageCache.Isolation semantics are unchanged: evaluation still happens once per context, verified by an e2e test that fails if module state leaks through the shared scripts.
Benchmarks (jsdom,
vmThreads,isolate: true, fs module cache on; cold = all caches wiped before every rep, warm = populated caches; median of 3 interleaved reps in a single window, M4/Node 24.13, whole-process wall clock; trivial assertions):vmForkstracksvmThreads(1022-module fixture: 6528ms → 5296ms cold, −19% against the resolution change alone). The split matches the mechanisms: fixtures with externalized dependencies win from the resolution fast path and compile reuse (the costs scale with the external graph), while the no-deps fixture only collects the transform/jsdom-import overlap.Transform time reporting
The prewarm broke the reported transform time, and investigating that exposed that the stat was already unreliable: it summed each fetch's wall time, but concurrent fetches (parallel workers, and now the prewarm) all wait on the same deduplicated in-flight transforms, so the sum counts the same work many times. Measured on the deps fixture (cold
vmThreadsrun, ~2.5s wall; serial--maxWorkers=1reference: 356ms of real transform work):maintoday, no prewarm, default workerstransformTimeis now the union of in-flight transform intervals, measured inside the fetcher around the actual transform (fetchAndProcess) — cache hits and fs-cache reads don't tick the clock. It has a crisp meaning ("wall time during which the server was transforming"), is bounded by the run's duration, and behaves identically with and without the prewarm. Warm runs with a populatedfsModuleCachenow truthfully report ~0 transform time. Per-module durations (metadata.duration, used by the UI module graph) are still recorded from direct worker fetches only — the prewarm's per-module walls would measure BFS queue position rather than the module's own cost.