Skip to content

perf(kernel): ~67% faster assembly loading via a runtime index#5164

Open
mrgrain wants to merge 3 commits into
mainfrom
mrgrain/perf/kernel/runtime-assembly-index
Open

perf(kernel): ~67% faster assembly loading via a runtime index#5164
mrgrain wants to merge 3 commits into
mainfrom
mrgrain/perf/kernel/runtime-assembly-index

Conversation

@mrgrain

@mrgrain mrgrain commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Loading packages is a large part of the fixed startup cost of any jsii host, and for big libraries such as aws-cdk-lib it dominates. The kernel parsed a package's entire .jsii assembly up front -- tens of megabytes of JSON -- even though a typical run only ever looks up a tiny fraction of the declared types (well under 1% in a large CDK synth). This makes loading roughly two-thirds faster on a warm package cache, so every host starts doing useful work sooner.

Benchmark

image

The results are more pronounced on smaller apps, where startup time is more costly. The larger an app gets, the less relative improvement we can see.

Cold package cache

The flip-side is that we need to do more work on the first run. For a cold package cache, we are now slower:

image

Technical details

When a package is served from the on-disk package cache, the kernel now builds a compact "runtime index" alongside the cached assembly the first time it is loaded: a small header, plus -- for every type -- its kind and the byte range of its definition within a single bodies blob. Documentation and source-location fields, which the runtime never reads, are stripped from the stored bodies. Type definitions are then parsed lazily, on first lookup, so types that are never used are never parsed. Because the bodies are stored decompressed, warm loads also skip following the gzip redirect and decompressing the assembly.

The index is built once per cached package -- whether it was just extracted or had been cached by an earlier run that predates this feature -- and carries a format version, so changing the layout transparently invalidates and rebuilds older indices. Using and building the index is strictly best-effort: validating loads, uncached loads, and any read or write failure fall back to a full eager parse, so correctness never depends on the cache.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@mergify mergify Bot added the contribution/core This is a PR that came from AWS. label Jun 15, 2026
@mrgrain mrgrain marked this pull request as draft June 15, 2026 15:19
@mrgrain mrgrain marked this pull request as ready for review June 15, 2026 15:49
@rix0rrr rix0rrr self-assigned this Jun 16, 2026
Comment thread packages/@jsii/kernel/src/kernel.ts
Comment thread packages/@jsii/kernel/src/runtime-index.ts
Comment thread packages/@jsii/kernel/src/runtime-index.ts Outdated
Comment thread packages/@jsii/kernel/src/runtime-index.ts Outdated
@mergify

mergify Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Thank you for contributing! ❤️ I will now look into making sure the PR is up-to-date, then proceed to try and merge it!

@mergify mergify Bot added the pr/ready-to-merge This PR is ready to be merged. label Jun 16, 2026
@mergify

mergify Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Merging (with squash)...

@mergify mergify Bot added the queued label Jun 16, 2026
@mergify

mergify Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-06-16 11:30 UTC · Rule: default-squash
  • 🟠 Checks running · in-place
  • 🚫 Left the queue2026-06-16 11:32 UTC · at 87ccc296e6767faf7876eef5ed229718357c240a

This pull request spent 2 minutes 1 second in the queue, with no time running CI.

Waiting for
  • status-success=Integration test (jsii-pacmak)
  • status-success=Unit Tests
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Integration test (jsii-pacmak)
    • check-skipped = Integration test (jsii-pacmak)
    • check-success = Integration test (jsii-pacmak)
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Build
    • check-skipped = Build
    • check-success = Build
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Unit Tests
    • check-skipped = Unit Tests
    • check-success = Unit Tests
All conditions
  • status-success=Integration test (jsii-pacmak)
  • status-success=Unit Tests
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Integration test (jsii-pacmak)
    • check-skipped = Integration test (jsii-pacmak)
    • check-success = Integration test (jsii-pacmak)
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Build
    • check-skipped = Build
    • check-success = Build
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Unit Tests
    • check-skipped = Unit Tests
    • check-success = Unit Tests
  • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
  • label!=pr/no-squash

Reason

Pull request #5164 has been dequeued

Queue conditions are not satisfied:

  • label!=pr/do-not-merge
  • status-success=Integration test (jsii-pacmak)
  • status-success=Unit Tests
  • status-success=Validate PR Title

Failing checks:

Hint

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.
If you do update this pull request, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio queue comment.

@mrgrain mrgrain added the pr/do-not-merge This PR should not be merged at this time. label Jun 16, 2026
@mergify mergify Bot added dequeued and removed queued labels Jun 16, 2026
@mrgrain mrgrain force-pushed the mrgrain/perf/kernel/runtime-assembly-index branch 2 times, most recently from a2f8847 to 8f1bb4d Compare June 16, 2026 12:57
Loading a package made the kernel parse the package's entire `.jsii`
assembly up front, even though a given execution only ever looks up a
tiny fraction of the declared types (often well under 1%). For large
assemblies such as aws-cdk-lib this is tens of megabytes of JSON parsed
to use a few hundred kilobytes of it.

When a package is served from the on-disk package cache, the kernel now
builds a compact "runtime index" alongside the cached assembly: a header
plus, for every type, its kind and the byte range of its (documentation-
stripped) definition within a single bodies blob. Type definitions are
then parsed lazily, on first lookup, so untouched types are never parsed.

The index is built once per cached package the first time it is loaded --
whether it was just extracted or had been cached by an earlier run that
predates this feature -- and carries a format version so that changing
the layout transparently invalidates and rebuilds older indices. Building
and using the index is strictly best-effort: validation loads, uncached
loads, and any read/write failure fall back to a full eager parse, so
correctness never depends on the cache.
@mergify mergify Bot removed the dequeued label Jun 16, 2026
@mrgrain mrgrain force-pushed the mrgrain/perf/kernel/runtime-assembly-index branch from 8f1bb4d to 952f3bc Compare June 16, 2026 12:57
@mrgrain mrgrain changed the title perf(kernel): ~67% faster assembly loading via a lazy cache index perf(kernel): ~67% faster assembly loading via a runtime index Jun 16, 2026
Comment thread packages/@jsii/kernel/src/runtime-index.ts Outdated
Comment thread packages/@jsii/kernel/src/runtime-index.ts
mrgrain added 2 commits June 16, 2026 18:20
… file

Replaces the JSON-object-per-type index with a columnar binary layout, and
makes the on-disk format self-describing.

The cache entry now holds two files:

  .jsii.runtime.v1.json        small manifest (schema, version, data path,
                               assembly metadata, byte layout)
  .jsii.runtime-index.v1       binary data file with four contiguous regions
                               laid out as
                                 names   : "fqn0
fqn1
..."  (UTF-8)
                                 kinds   : uint8 per type     (TypeKind ordinal)
                                 offsets : uint32le[N+1]      (into defs)
                                 defs    : type-definition JSON, doc-stripped

LazyTypes now consumes the columns directly: kinds and offsets are read as
typed-array views over the file (zero parsing), only the names blob is decoded,
and a type definition's bytes are sliced from the defs region and JSON-parsed
on first lookup. On aws-cdk-lib (~20k types) this is ~3.3x faster to load and
~43% smaller than the previous JSON map.

Both filenames are versioned, so a cache entry shared by multiple jsii
runtimes of different format versions never has one version clobber another's
files. The version is recorded both in the manifest filename and in a `version`
field inside the manifest, as an integrity check. The data file's path is also
recorded inside the manifest (defaulting to the versioned basename); the reader
resolves it relative to the manifest's directory and rejects absolute paths or
`..` traversal as defense-in-depth against a tampered or corrupted cache.

The format is also documented under
gh-pages/content/specification/7-runtime-index.md.

Test isolation: the kernel test suite previously read and pruned the
developer's real package cache via `defaultCacheRoot()`, which has caused
flakes when prior runs leave the cache inconsistent. A jest globalSetup now
points `JSII_RUNTIME_PACKAGE_CACHE_ROOT` at a fresh tmpdir for the duration
of the run (with a matching teardown), and `kernel.test.ts`'s afterAll prune
honours the override.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution/core This is a PR that came from AWS. pr/do-not-merge This PR should not be merged at this time. pr/ready-to-merge This PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants