fix(core): optimize pnpm lockfile parsing with pre-built indexes by adwait1290 · Pull Request #33750 · nrwl/nx

adwait1290 · 2025-12-08T06:25:55Z

Current Behavior

When parsing pnpm lockfiles to build the project graph:

matchPropValue function

Uses separate Object.values() + Object.keys() iterations
Two passes over the same data to find a value and get its key

Hoisted Dependencies Lookup

For each package, calls Object.keys(hoistedDeps).find(k => k.startsWith(...))
O(n) search performed m times = O(n*m) total

Expected Behavior

Single-pass Iteration

BEFORE: matchPropValue
┌─────────────────────────────────────────────────────────────┐
│  const index = Object.values(record).findIndex(v === key)   │
│  if (index > -1)                                            │
│    return Object.keys(record)[index]  ← Another iteration!  │
│                                                             │
│  = 2 iterations over the same data                          │
└─────────────────────────────────────────────────────────────┘

AFTER: Object.entries() single pass
┌─────────────────────────────────────────────────────────────┐
│  for (const [name, version] of Object.entries(record)) {    │
│    if (version === key) return name  ← Early exit           │
│  }                                                          │
│                                                             │
│  = 1 iteration with early exit on match                     │
└─────────────────────────────────────────────────────────────┘

Pre-built Index for Hoisted Dependencies

BEFORE: O(n*m) Hoisted Lookup
┌─────────────────────────────────────────────────────────────┐
│  for each package (n packages):                             │
│    Object.keys(hoistedDeps).find(k => k.startsWith(...))    │
│    ← O(m) search where m = hoisted deps                     │
│                                                             │
│  Total: O(n * m)                                            │
│                                                             │
│  Example: 500 packages × 200 hoisted deps                   │
│         = 100,000 string comparisons                        │
└─────────────────────────────────────────────────────────────┘

AFTER: Pre-built Index Map O(n+m)
┌─────────────────────────────────────────────────────────────┐
│  Build hoistedKeysByPackage Map once:  O(m)                 │
│    Map<packageName, hoistedKey>                             │
│                                                             │
│  for each package (n packages):                             │
│    hoistedKeysByPackage.get(packageName)  O(1)              │
│                                                             │
│  Total: O(n + m)                                            │
│                                                             │
│  Example: 500 packages + 200 hoisted deps                   │
│         = 700 operations (vs 100,000)                       │
└─────────────────────────────────────────────────────────────┘

Impact

For large pnpm monorepos:

Metric	Before	After	Improvement
matchPropValue iterations	2	1	50% fewer iterations
Hoisted lookup complexity	O(n×m)	O(n+m)	~100x for large repos
String comparisons	n×m	n+m	Dramatic reduction

Related Issue(s)

Contributes to #32669, #32254

Merge Dependencies

This PR has no dependencies and can be merged independently.

Must be merged BEFORE: #33751

PNPM Parser Optimizations: - Use Object.entries() for single-pass iteration in matchPropValue instead of separate Object.values() + Object.keys() calls - Pre-build packageName -> key index Map for hoisted dependencies lookup (O(n*m) -> O(n+m) where n=packages, m=hoisted deps) Impact: Faster project graph creation for pnpm monorepos, especially those with many packages and hoisted dependencies.

netlify · 2025-12-08T06:25:59Z

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	`4156b4f`

vercel · 2025-12-08T06:26:00Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Updated (UTC)
nx-dev	Ready	Preview	Dec 9, 2025 5:03pm

chatgpt-codex-connector · 2025-12-08T06:26:01Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

leosvelperez

Thanks for contributing! Please make hoistedKeysByPackage a required parameter.

…ify usage

nx-cloud · 2025-12-10T07:28:56Z

View your CI Pipeline Execution ↗ for commit 4156b4f

Command	Status	Duration	Result
`nx affected --targets=lint,test,test-kt,build,e...`	✅ Succeeded	8m 38s	View ↗
`nx run-many -t check-imports check-lock-files c...`	✅ Succeeded	2m 32s	View ↗
`nx-cloud record -- nx-cloud conformance:check`	✅ Succeeded	12s	View ↗
`nx-cloud record -- nx format:check`	✅ Succeeded	2s	View ↗
`nx-cloud record -- nx sync:check`	✅ Succeeded	<1s	View ↗

☁️ Nx Cloud last updated this comment at 2025-12-10 09:07:23 UTC

leosvelperez

Thanks!

meeroslav · 2025-12-10T16:28:19Z

The hoisted calculation improvement was a good catch. On a test repo we measured 5x improvement on that code section.

Sadly, the matchPropValue is now performing worse. On test repo it brought the total duration of matching versions from ~650 to 900+ milliseconds.

Here are the reasons why:

Native methods are heavily optimized: findIndex() is implemented in optimized C++ code in the JavaScript engine, while for...of runs in the JavaScript layer with more overhead per iteration.
Object.entries() is expensive: It creates an array of arrays (tuples), which involves more object allocation than Object.values(), which creates a flat array. Even though you call both Object.values() and Object.keys(), creating two flat arrays can be faster than creating one array of tuple objects.
Destructuring has overhead: The [name, version] destructuring happens on every loop iteration, adding cost.
Array access is extremely fast: Object.keys(record)[index] is a simple array index access, which is highly optimized. Modern engine makes this nearly free.
The record is small, so the "double iteration" penalty is negligible due to native method optimizations.

github-actions · 2025-12-16T00:12:29Z

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

adwait1290 requested review from a team and meeroslav as code owners December 8, 2025 06:25

adwait1290 requested a review from leosvelperez December 8, 2025 06:25

vercel Bot deployed to Preview December 8, 2025 06:31 View deployment

adwait1290 mentioned this pull request Dec 8, 2025

perf(js): pre-build package index for O(n+m) pnpm lockfile pruning #33751

Closed

leosvelperez reviewed Dec 9, 2025

View reviewed changes

Comment thread packages/nx/src/plugins/js/lock-file/pnpm-parser.ts Outdated

Comment thread packages/nx/src/plugins/js/lock-file/pnpm-parser.ts Outdated

leosvelperez requested changes Dec 9, 2025

View reviewed changes

adwait1290 requested review from FrozenPandaz and vsavkin as code owners December 9, 2025 16:57

chore: address review - make hoistedKeysByPackage non-optional, simpl…

4156b4f

…ify usage

adwait1290 force-pushed the perf/pnpm-parser-optimizations branch from 700a583 to 4156b4f Compare December 9, 2025 16:57

vercel Bot deployed to Preview December 9, 2025 17:03 View deployment

leosvelperez approved these changes Dec 10, 2025

View reviewed changes

leosvelperez changed the title ~~perf(nx): optimize pnpm lockfile parsing with pre-built indexes~~ fix(core): optimize pnpm lockfile parsing with pre-built indexes Dec 10, 2025

leosvelperez merged commit 58a69f9 into nrwl:master Dec 10, 2025
16 of 20 checks passed

github-actions Bot locked as resolved and limited conversation to collaborators Dec 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(core): optimize pnpm lockfile parsing with pre-built indexes#33750

fix(core): optimize pnpm lockfile parsing with pre-built indexes#33750
leosvelperez merged 2 commits into
nrwl:masterfrom
adwait1290:perf/pnpm-parser-optimizations

adwait1290 commented Dec 8, 2025 •

edited

Loading

Uh oh!

netlify Bot commented Dec 8, 2025 •

edited

Loading

Uh oh!

vercel Bot commented Dec 8, 2025 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Dec 8, 2025

Uh oh!

Uh oh!

Uh oh!

leosvelperez left a comment

Uh oh!

nx-cloud Bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

leosvelperez left a comment

Uh oh!

Uh oh!

meeroslav commented Dec 10, 2025

Uh oh!

github-actions Bot commented Dec 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

adwait1290 commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current Behavior

matchPropValue function

Hoisted Dependencies Lookup

Expected Behavior

Single-pass Iteration

Pre-built Index for Hoisted Dependencies

Impact

Related Issue(s)

Merge Dependencies

Uh oh!

netlify Bot commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👷 Deploy request for nx-docs pending review.

Uh oh!

vercel Bot commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Dec 8, 2025

Uh oh!

Uh oh!

Uh oh!

leosvelperez left a comment

Choose a reason for hiding this comment

Uh oh!

nx-cloud Bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leosvelperez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

meeroslav commented Dec 10, 2025

Uh oh!

github-actions Bot commented Dec 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

adwait1290 commented Dec 8, 2025 •

edited

Loading

netlify Bot commented Dec 8, 2025 •

edited

Loading

vercel Bot commented Dec 8, 2025 •

edited

Loading

nx-cloud Bot commented Dec 10, 2025 •

edited

Loading