Skip to content

perf(interp): hoist loop-invariant container state in JIT loops - #154

Merged
siyul-park merged 3 commits into
mainfrom
perf/jit-loop-hoist
Jul 17, 2026
Merged

perf(interp): hoist loop-invariant container state in JIT loops#154
siyul-park merged 3 commits into
mainfrom
perf/jit-loop-hoist

Conversation

@siyul-park

@siyul-park siyul-park commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

Implements #153: an entryLoop trace plan now hoists one loop-invariant container's derivation out of the loop body into a per-entry prologue.

  • Detection (interp/jit_plan.go): hoistable picks the most-accessed ref local whose recorded ARRAY_GET/ARRAY_SET steps agree on one primitive typed-array itab, provided no block writes that local and the plan contains no CALL/RETURN_CALL (a BL would clobber the hoisted registers). Container provenance comes from a conservative per-block marker stack; anything unknown declines.
  • Prologue (interp/jit_arm64.go, arm64Lowerer.hoist): once per native entry — load the container local, tag + itab guard against an empty-snapshot exit resuming at the header, load dataPtr/len into registers that stay live for the whole body. Safe with no pinning because a call-free trace loop callable has only forward control flow.
  • Access fast paths: matching ARRAY_GET keeps only its bounds exit, guardIndex, and the element load (shared tail; only the (dataPtr, n) source branches). A hoisted primitive ARRAY_SET keeps bounds + store and stops being a state barrier (no flush(flushSnapshot) + local-cache clear). Non-matching containers, ref arrays, globals/upvals, and terminal stores use the existing paths byte-for-byte.
  • Livelock fix (interp/interp.go): a loop fallback exit that resumes at the header itself made no progress, and the header dispatch slot holds the native stub — redispatching would livelock. The loop wrapper now runs the shadowed threaded handler (i.exits[root]) once. The hoist prologue's shape guard is the first exit that can resume at the header; covered by a regression test that alternates the container between an array and null across entries.

Measured effect (honest)

  • Sieve's two loop callables shrink ~40%: vm_jit_entry_bytes_total 179,992 → 107,168 (inner) and 209,804 → 124,644 (scan).
  • Sieve(256) wall time is unchanged within noise (interleaved A/B on M4 Pro: ~1–2%). The removed per-access loads sat off the out-of-order critical path, so the issue's projected 2.5–3 µs did not materialize.
  • TypedArraySum(256) and AllocationGraph(128) unchanged (interleaved A/B).
  • Profiling shows where the remaining Sieve gap actually lives, filed as follow-ups: loop-kind branch legs are never folded (one native exit/re-entry per prime in the scan loop), and the 256-copy forward-chained body pays a per-iteration operand flush/reload/budget chain (a pure fill loop runs slower native than threaded today).

Test plan

  • go test -race ./... (root and benchmarks/ modules)
  • New TestARM64_HoistedContainerLoop: native execution with only loop-exit exits, prologue shape deopt with exact refcounts, mid-loop bounds deopt with identical error, LOCAL_SET-in-loop rejection, mixed hoisted + slow-path containers — all against a threaded oracle comparing results and rc[1:]
  • New TestHoistable plan-level unit tests
  • make fuzz, make lint
  • make coverage-check — fails from the pre-existing goenv go1.25/go1.26 toolchain split, unrelated to this diff

Closes #153.

Summary by CodeRabbit

  • Performance

    • Improved ARM64 JIT performance for eligible loops by hoisting loop-invariant array container metadata and reusing derived slice-header information, reducing redundant per-access checks/loads (notably shrinking loop-callable size by ~40%).
    • Added faster hoisted array get/set paths within supported loop traces.
  • Bug Fixes

    • Prevented potential loop livelocks when deoptimization resumes at the same loop header position.
  • Tests

    • Added ARM64 coverage for hoisted-container loop correctness and profiling expectations.
    • Expanded unit coverage for loop-hoisting eligibility.
  • Documentation

    • Documented loop-invariant container hoisting rules and updated benchmark notes accordingly.

An entryLoop trace plan now hoists one loop-invariant container: hoistable
picks the most-accessed ref local whose recorded ARRAY_GET/ARRAY_SET steps
agree on one primitive typed-array itab, provided no block writes the local
and the plan is call-free. The backend prologue derives the heap cell, tag
and itab guard, and slice header once per native entry, so matching accesses
keep only the bounds check and element op, and a hoisted primitive ARRAY_SET
stops being a state barrier.

A prologue guard exit resumes at the loop header, whose dispatch slot holds
the native stub; the loop wrapper now runs the shadowed threaded handler once
for any fallback that resumes at the header, so such exits cannot livelock.

Sieve's loop callables shrink ~40% (vm_jit_entry_bytes_total 179,992 ->
107,168 and 209,804 -> 124,644); wall time is unchanged within noise on
M4 Pro because the removed loads sat off the out-of-order critical path.
The remaining gap is native entry/exit churn (loop-kind branch legs are
never folded) and the per-iteration operand flush, tracked as follow-ups.

Closes #153.
@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 79bc1aa6-c62b-4ae2-b1be-6ec30bce7542

📥 Commits

Reviewing files that changed from the base of the PR and between a0795ec and aa05fa4.

📒 Files selected for processing (4)
  • docs/jit-internals.md
  • interp/jit_arm64.go
  • interp/jit_plan.go
  • interp/jit_plan_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/jit-internals.md
  • interp/jit_arm64.go

📝 Walkthrough

Walkthrough

Loop plans detect invariant array containers. ARM64 loop entries hoist guarded slice headers for array accesses, while interpreter fallback prevents header-resume livelocks. Tests and documentation cover eligibility, execution parity, guards, errors, and benchmark results.

Changes

Loop-invariant container hoisting

Layer / File(s) Summary
Plan hoist eligibility
interp/jit_plan.go, interp/jit_plan_test.go
Loop plans identify eligible invariant containers by tracking array accesses, writes, calls, stack provenance, and itab consistency.
ARM64 hoisted access lowering
interp/jit.go, interp/jit_arm64.go
Loop entry lowering caches guarded slice-header registers and reuses them for matching array reads and primitive writes.
Loop installation and header fallback
interp/interp.go, interp/jit_arm64_test.go
Loop handlers receive the root anchor and invoke the saved threaded handler when deoptimization resumes at the same header.
Validation and documentation
interp/jit_arm64_test.go, AGENTS.md, docs/jit-internals.md, docs/benchmarks.md
Tests cover hoisting eligibility, guards, bounds errors, mutation, shared containers, parity, invariants, implementation details, and benchmark results.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LoopPlan
  participant arm64Lowerer
  participant NativeLoop
  participant Interpreter.loop
  participant ThreadedHandler
  LoopPlan->>arm64Lowerer: provide hoist descriptor
  arm64Lowerer->>NativeLoop: guard container and cache slice header
  NativeLoop->>NativeLoop: reuse cached header for array accesses
  NativeLoop->>Interpreter.loop: deopt at loop header
  Interpreter.loop->>ThreadedHandler: advance on same-header resume
Loading

Possibly related issues

  • Issue 153 — Directly covers the loop-invariant container hoisting implemented here.

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 45.45% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main JIT loop-container hoisting change.
Description check ✅ Passed The description covers the changes, related issue, extra context, and testing, though it doesn't use the template's exact headings.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/jit-loop-hoist

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.48598% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.63%. Comparing base (57f0abd) to head (aa05fa4).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
interp/jit_plan.go 67.64% 24 Missing and 9 partials ⚠️
interp/interp.go 0.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #154      +/-   ##
==========================================
+ Coverage   29.57%   29.63%   +0.06%     
==========================================
  Files          86       86              
  Lines       60187    60292     +105     
==========================================
+ Hits        17801    17870      +69     
- Misses      41119    41146      +27     
- Partials     1267     1276       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/jit-internals.md`:
- Line 346: Update the trace-loop hoisting description around “every matching
access” to explicitly state that hoisted I64 loads retain their value/boxability
side exit, while terminal primitive ARRAY_SET operations continue through the
existing path. Keep the documentation terse, factual, and consistent with the
surrounding formatting.

In `@interp/jit_arm64.go`:
- Line 4465: Update the LDR emission in the surrounding hoisted-local handling
to avoid silently truncating (f.base+h.local)*8 when the layout exceeds the
encodable immediate range. Prefer the existing register-offset addressing path
for oversized offsets, or explicitly reject layouts beyond 4,095 slots while
preserving the current immediate path for valid offsets.

In `@interp/jit_plan.go`:
- Around line 405-447: Filter hits before selecting best in the hoist-ranking
logic so only itabs supported by the ARM64 prologue and non-terminal-store
candidates can win the single hoist slot. Update the ARRAY_GET/ARRAY_SET hit
collection or selection around best and bestHits, preserving valid
primitive-array candidates when hotter unsupported ref-array or terminal-store
candidates exist. Add a regression test covering mixed ref/primitive candidates.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1f044da9-5d3f-45a7-9904-17e0d525289c

📥 Commits

Reviewing files that changed from the base of the PR and between 57f0abd and f1b57da.

📒 Files selected for processing (9)
  • AGENTS.md
  • docs/benchmarks.md
  • docs/jit-internals.md
  • interp/interp.go
  • interp/jit.go
  • interp/jit_arm64.go
  • interp/jit_arm64_test.go
  • interp/jit_plan.go
  • interp/jit_plan_test.go

Comment thread docs/jit-internals.md Outdated
Comment thread interp/jit_arm64.go Outdated
Comment thread interp/jit_plan.go Outdated
@siyul-park
siyul-park merged commit 1d652d6 into main Jul 17, 2026
5 checks passed
@siyul-park
siyul-park deleted the perf/jit-loop-hoist branch July 17, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hoist loop-invariant heap-cell/itab/slice-header reloads in JIT container loops

1 participant