Skip to content

fix(interp): two native frame defects on the self-call and return paths - #193

Merged
siyul-park merged 2 commits into
mainfrom
fix/jit-return-retain-ordering
Aug 16, 2026
Merged

siyul-park merged 2 commits into
mainfrom
fix/jit-return-retain-ordering

Conversation

@siyul-park

@siyul-park siyul-park commented Aug 16, 2026 •

Copy link
Copy Markdown
Owner

Split out of #191. Two independent defects, both present on main today, unrelated to the performance work that PR carries.


1. A native self-call could branch into a foreign frame — SIGSEGV

selfCall lowers recursion as a BL back to ctx.head, re-entering the plan's entry prologue. That is only the callee's real entry when the live frame is the plan's own — but selfCall checked neither ctx.kind nor len(ctx.frames).

The trace frontend reaches it with a foreign frame live. Any CALL inside a function-entry-anchored trace whose callee ref matches the anchor is recorded as an ordinary self CALL, including one nested inside an already-inlined callee's body. So a trace that inlines A → B and then sees B call back into A arrives at selfCall with two frames, and the BL lays A's parameter prologue over B's activation.

When the two frame shapes differ, the result is not a wrong answer — it is a fault inside the Go runtime:

unexpected fault address 0x76cf8e843080
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x76cf8e843080 pc=0x105f9c414]

Reproduced deterministically on main, at three distinct wild addresses across runs, faulting inside asm/arm64.invoke. TestARM64_SelfCallFromInlinedFrame segfaults without the fix and passes with it.

Of the two conditions only the frame count is load-bearing today — verified by isolating them: ctx.kind != entryFunction alone does not stop the crash, len(ctx.frames) != 1 alone does. A loop plan cannot currently carry a self CALL, because the tracer cuts one anchored at a loop header. But ctx.head is equally not an entry prologue in that case, so both are checked together where the BL is emitted.

The static frontend never reached this: directCall rejects a self callee outright, and it is the only path that sets step.known.


2. ret deopted on every ref return

ret took the retain on a returned ref after guardFrame had already read the backing local's refcount. guardFrame deopts whenever rc <= pending, and the most ordinary ref-returning shape — allocate a node into a local, return it — sits exactly at that boundary, because the local is the only holder, so rc == pending == 1.

Measured on main with the program the test uses: 512 guard-value exits against 1024 native entries — half of all native calls deopted at their return. After the fix: 0.

Execution stayed correct throughout, since the interpreter completes the RETURN either way, so a spurious deopt is invisible to both a value oracle and a refcount oracle. TestARM64_RefReturn asserts the guard-value exit count directly, alongside threaded-twin parity checks that rule out the retain leaking or freeing.

Taking the retain first raises rc above pending, the guard passes, and the following releaseFrame decrement leaves exactly the one live reference the caller now owns — the same net transfer threaded retire performs.

stitch is deliberately left alone: its callee returns cannot reach the same boundary, because call()'s inline path rejects any callee holding a ref local, so an inlined return is backed by a parameter or an upvalue whose refcount exceeds the frame's pending count. docs/jit-internals.md records that asymmetry.


Test plan

  • TestARM64_SelfCallFromInlinedFrame — mutual recursion through ref-parameter callees at depths 1–16; SIGSEGVs on main, passes with the fix
  • TestARM64_RefReturn — fails on main with guardFrame spuriously deopted a RETURN of a singly-owned frame local
  • each guard isolated to confirm which condition is load-bearing
  • go test -race ./..., make check-generated, go vet ./...

Refs #164

🤖 Generated with Claude Code

ret took the retain on a returned ref after guardFrame had already read the
backing local's refcount. guardFrame deopts whenever rc <= pending, and the
most ordinary ref-returning shape - allocate a node into a local, return it -
sits exactly at that boundary, because the local is the only holder and so
rc == pending == 1. Every such RETURN therefore left native code.

The regression test's program measures 512 guard-value exits against 1024
native entries before this change and none after, so half of all native calls
were deopting at their return. Execution stayed correct throughout, since the
interpreter completes the RETURN either way, which is why neither a value nor
a refcount oracle catches this: the test asserts the guard-value exit count
directly.

Taking the retain first raises rc above pending, the guard passes, and the
following releaseFrame decrement leaves exactly the one live reference the
caller now owns - the same net transfer threaded retire performs.

stitch is deliberately left alone. Its callee returns cannot reach the same
boundary: call()'s inline path rejects any callee holding a ref local, so an
inlined return is backed by a parameter or an upvalue, whose refcount exceeds
the frame's pending count.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 16, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@siyul-park, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 46 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 78ba1901-1565-4eeb-ba19-3c8d3aaf66f5

📥 Commits

Reviewing files that changed from the base of the PR and between 44dab81 and 080a2d3.

📒 Files selected for processing (3)
  • docs/jit-internals.md
  • interp/jit_arm64.go
  • interp/jit_arm64_test.go

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

selfCall lowers recursion as a BL back to ctx.head, re-entering the plan's
entry prologue. That is only the callee's real entry when the live frame is the
plan's own, but selfCall checked neither ctx.kind nor len(ctx.frames), and the
trace frontend reaches it with a foreign frame live.

Any CALL inside a function-entry-anchored trace whose callee ref matches the
anchor is recorded as an ordinary self CALL, including one nested inside an
already-inlined callee's body. So a trace that inlines A -> B and then sees B
call back into A arrives at selfCall with two frames, and the BL lays A's
parameter prologue over B's activation. When the two frame shapes differ the
result is not a wrong answer but a fault inside the Go runtime: the added test
segfaults deterministically without this change, at three distinct wild
addresses across runs, inside asm/arm64.invoke.

Of the two conditions only the frame count is load-bearing today - a loop plan
cannot currently carry a self CALL, because the tracer cuts one anchored at a
loop header - but ctx.head is equally not an entry prologue in that case, so
both are checked together where the BL is emitted.

The static frontend never reached this: directCall rejects a self callee
outright, and it is the only path that sets step.known.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@siyul-park siyul-park changed the title fix(interp): retain native returns before the frame guard fix(interp): two native frame defects on the self-call and return paths Aug 16, 2026
@siyul-park
siyul-park merged commit 06c8152 into main Aug 16, 2026
4 of 5 checks passed
@siyul-park
siyul-park deleted the fix/jit-return-retain-ordering branch August 16, 2026 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant