chore(repo): prebuild shared e2e base workspace to skip per-file create-nx-workspace#35964
Draft
FrozenPandaz wants to merge 3 commits into
Draft
chore(repo): prebuild shared e2e base workspace to skip per-file create-nx-workspace#35964FrozenPandaz wants to merge 3 commits into
FrozenPandaz wants to merge 3 commits into
Conversation
✅ Deploy Preview for nx-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for nx-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
View your CI Pipeline Execution ↗ for commit 6fd5d03
☁️ Nx Cloud last updated this comment at |
…te-nx-workspace Each atomized e2e-ci task pays a ~40s create-nx-workspace cold start in its first newProject hook (the proj-backup template is per-process, so it never crosses files or agents). This adds a cached populate-e2e-base-workspace task that builds a bare workspace template once and outputs it, so Nx restores it to every agent. newProject seeds from the template (~0.3s copy) instead of running create-nx-workspace, falling back to the original build when the template is absent (e.g. pnpm).
…emplate newProject's seeding branch skipped runCreateWorkspace, which sets the module-level projName as a side effect that packageInstall -> getPackageManagerCommand relies on. Without it, the package.json read resolved to .../undefined/package.json and crashed, orphaning the seeded 'proj' dir and cascading into 'directory proj already exists' on subsequent newProject calls.
Declare the proj-backup template (a dependency output) via dependentTasksOutputFiles in e2eInputs so e2e tasks expect to read it, and use the /**/* glob for the populate task's script dir so its own script read is expected. Clears the sandbox violations introduced by the shared base workspace.
fe5814a to
6fd5d03
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current Behavior
The e2e suite is atomized into one task per spec file (
e2e-ci--src/<file>.test.ts). Each task runsnewProject()in its firstbeforeAll/beforeEach, which performs acreate-nx-workspacecold start (~40s). On the tight hook timeout that intermittently flakes — it's the same root cause behind severalExceeded timeout of 60000 ms for a hookfailures.The existing
proj-backuptemplate that makes within-filenewProjectcalls fast is built lazily and lives under the per-process e2e root, so it never crosses files or distributed agents. Every spec file pays the cold start again.Expected Behavior
A new cached
populate-e2e-base-workspacetask builds a bare workspace template once, up front, and declares it as an Nx output (dist/local-registry/proj-backup). Because it's a cached output, Nx restores it to every distributed agent the same way the verdaccio storage already is — keyed on the nx build hash, so it rebuilds automatically when nx changes.newProject()then seeds the workspace from that template viacopySync(~0.3s) instead of runningcreate-nx-workspace(~40s). It falls back to the original build when the template is absent (e.g. pnpm, or the task didn't run), so the change is purely additive. Each test still installs its ownpackages[]on top of the bare base, so what ends up installed per workspace is unchanged.Measured locally
create-nx-workspace --preset=apps, npm).copySync~0.3s vscreate-nx-workspace~40s.Note for reviewers
The remaining trade-off is the ~177 MB artifact restored per agent per run; the net win depends on files-per-agent in the e2e-ci distribution, worth confirming with an A/B before merging. npm only for now (pnpm store symlinks aren't portable) — pnpm falls back to the original path.
Related Issue(s)
N/A — e2e infrastructure / performance.
View session information ↗