Skip to content

refactor(interp): minimize complexity across runtime, JIT, and generator#161

Merged
siyul-park merged 10 commits into
mainfrom
refactor/interp-quality
Jul 19, 2026
Merged

refactor(interp): minimize complexity across runtime, JIT, and generator#161
siyul-park merged 10 commits into
mainfrom
refactor/interp-quality

Conversation

@siyul-park

@siyul-park siyul-park commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

Complexity-minimizing refactor of interp/ plus its code generator, per docs/coding-patterns.md §0 (no compatibility guarantees; internal package).

  • Merged duplicate/aliased symbols: keep (pure alias of alloc) deleted everywhere including generator emission; Cache.rearm inlined into its only caller; trace outcome type + kind field unified as status, removing the word collision with types.Kind and prof.CaptureOutcome; four ARM64 const emitters merged into one constant(); CMP+BEQ skip shells share unlessEqual.
  • Deduplicated logic: seed() shared by New/Reset; function-metadata pass folded into the constant boxing pass; retain guards reuse alive(); skipCall reuses zeroBoxed; attempt() owns the Compile → recordCompile → error-metric core repeated by compile/exit/shared; grow() centralizes the six per-function table extensions in bind.
  • Latent bug fixed: install wrote natives[a.addr] guarded only by len(code), panicking on a JIT compile of a dynamically bound function. natives deliberately keeps its New-time size (a native frame suspended across a trap fallback may cache the journal base), so install now skips out-of-range slots and documents why.
  • Naming: Tracer receiver rt; globalReprsglobalDecls with the declared-vs-observed distinction documented; rc-slice locals no longer shadow-named hits.
  • Generator (internal/cmd/geninterp): accessor descriptor table replaces the parallel per-op switches in loader.decode/read/slotField; dead guard() removed; new hand-written interp/threader.go compile-time helpers absorb per-variant decode+bounds+kind boilerplate — threaded.go 65,188 → 64,323 lines with runtime closures fully specialized as before.
  • Docs: AGENTS.md invariants referencing dead symbols (spillSafe, tree.branchIPs(), arm64Lowerer.walk) rewritten to the current noSpill/noSpillArch/aborted-fragment-planning contracts.

Net: −779 lines across 13 files.

Intentionally skipped (recorded per §0.9)

  • threaded.go's 3× GLOBAL/LOCAL/UPVAL fusion-closure duplication (~52k lines): closures must stay fully specialized; deduping needs runtime accessor indirection, which defeats the fused-closure model.
  • zero/zeroBoxed merge: different dispatch (Repr() vs exact kind) and retain semantics.
  • work/sideExit snapshot embed, next/path prologue extract: pure churn, hides mutation.
  • planner slot/lowering value merge and emitExits/retainDeferred dedup: architecture boundary / deferred-ref invariant.
  • funcSlot array-of-structs: hot-path cache risk; needs its own benchmark-gated PR.

Test plan

  • go test -race ./... all pass
  • make generate && make check-generated clean
  • make lint clean
  • make fuzz smoke pass
  • make benchmark-pr before/after: neutral (Reset 48→32ns, TypedArraySum 7.3→6.1µs improved; Fused/JITWarm/New within noise)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Performance
    • Improved JIT trace planning, compilation, and threaded interpreter execution, with more consistent handling of constants, slots, and control-flow.
  • Reliability
    • Strengthened runtime safety checks for bounds/type/retention, including safer side-exit/native installation and improved boxing/allocation behavior.
    • REPL .save now rejects host-only constants (host functions/objects).
  • Documentation
    • Updated guidance for JIT invariants and interpreter trace naming.
    • Clarified and modernized public API documentation to use the codec-based approach (e.g., WithCodec, NewHostObject).

- drop Interpreter.keep alias; emit alloc from geninterp
- inline Cache.rearm into its only caller
- rename trace outcome type/field to status; Tracer receiver r -> t
- rename globalReprs -> globalDecls; document decl-vs-observed split
- rename rc-slice locals shadowed as hits
- fold function-metadata pass into the constant boxing pass in New
- extract seed() shared by New and Reset
- reuse alive() for constant retain guards
- reuse zeroBoxed for skipCall zero returns
compile, exit, and shared repeated Compile + recordCompile + error metric;
attempt() now owns that core while acquisition and delivery stay local.
bind repeated six grow blocks; grow() now owns extension of every
per-function table. natives stays fixed at its New-time size because a
native frame suspended across a trap fallback may cache the journal
base, so install skips natives slots beyond it instead of panicking on
dynamically bound functions.
- merge the four const emitters into constant()
- share the CMP+BEQ skip shell via unlessEqual
- inline single-caller enterBlock into emitBlock
- accessor descriptor table replaces parallel per-op switches in
  loader.decode/read and slotField; threaderFunc shares the closure
  wrapper; dead guard() removed
- new interp/threader.go compile-time helpers (local/global/upval/
  constant) absorb per-variant decode+bounds+kind boilerplate; runtime
  closures stay fully specialized
- threaded.go 65,188 -> 64,323 lines; compile+fused benchmarks neutral
spillSafe/tree.branchIPs/arm64Lowerer.walk no longer exist; describe the
plan-level noSpill/noSpillArch contract and aborted-fragment planning
rule instead. Track globalDecls rename in the naming audit.
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 31b189c9-7ab1-4efa-8128-ca2d16e4193a

📥 Commits

Reviewing files that changed from the base of the PR and between 410fa2f and 151e89a.

📒 Files selected for processing (22)
  • cli/repl_test.go
  • docs/host-integration.md
  • docs/jit-internals.md
  • docs/testing.md
  • internal/cmd/geninterp/lower.go
  • interp/cache.go
  • interp/cache_test.go
  • interp/coroutine.go
  • interp/coroutine_test.go
  • interp/host.go
  • interp/host_test.go
  • interp/interp.go
  • interp/interp_test.go
  • interp/jit_arm64.go
  • interp/jit_plan.go
  • interp/jit_plan_test.go
  • interp/marshal.go
  • interp/pool.go
  • interp/pool_test.go
  • interp/threaded.go
  • interp/trace.go
  • interp/trace_test.go
🔥 Files not summarized due to errors (1)
  • interp/threaded.go: Server error: no LLM provider could handle the message

📝 Walkthrough

Walkthrough

The change refactors codec and host-object integration, interpreter allocation and cache ownership, generated threader lowering, trace status handling and planning, ARM64 lowering, and related tests. Documentation updates describe the revised APIs, JIT invariants, initialization layout, and testing ownership.

Changes

Interpreter runtime and host integration

Layer / File(s) Summary
Codec planning and host-object conversion
interp/marshal.go, interp/host.go, interp/interp.go, interp/host_test.go
The marshaler API becomes Codec; conversion plans and interpreter codec state are restructured; NewHostObject and conversion-aware field ownership are added.
Allocation, cache, coroutine, and pool ownership
interp/interp.go, interp/cache.go, interp/coroutine.go, interp/pool.go, related tests
Boxed references use alloc, cache and coroutine types become private, cache request coverage is centralized, and pool wiring uses private cache/tracer helpers.

JIT pipeline

Layer / File(s) Summary
Trace status capture and plan construction
interp/trace.go, interp/jit_plan.go, interp/trace_test.go, interp/jit_plan_test.go
Trace classification changes from kind/outcome to status, aborted traces are excluded from planning, and capture, splitting, publication, and tests use the status model.
Generated threader access and lowering
internal/cmd/geninterp/generate.go, internal/cmd/geninterp/lower.go
Generated slot and constant accessors validate bounds and representations, standalone and guarded lowering paths are separated, and generated handlers use a common wrapper.
ARM64 lowering
interp/jit_arm64.go
Coroutine layout references, block restoration, constant lowering, and conditional retain/release emission are consolidated.

Documentation and validation

Layer / File(s) Summary
Runtime API and invariant documentation
AGENTS.md, docs/host-integration.md, docs/jit-internals.md, docs/testing.md
Documentation describes codec and host-object contracts, private JIT ownership, trace statuses, spilling invariants, and updated test ownership.
Source layout and REPL validation
docs/coding-patterns.md, docs/symbol-naming-audit.md, instr/parse.go, cli/repl_test.go
Initialization placement and retained symbol naming are updated, parser initialization is relocated, and REPL saving of host-only constants is tested.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

  • siyul-park/minivm#115 — Covers overlapping cache, tracing, allocation ownership, and JIT planning changes.

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the refactor across interp runtime, JIT, and generator.
Description check ✅ Passed The description is substantive and covers changes, skipped items, and testing, but it does not follow the template headings exactly.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/interp-quality
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch refactor/interp-quality

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.44.1)
interp/threaded.go

ast-grep timed out on this file


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.49518% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 30.13%. Comparing base (8c413ae) to head (410fa2f).

Files with missing lines Patch % Lines
interp/trace.go 86.95% 15 Missing and 3 partials ⚠️
interp/interp.go 57.14% 14 Missing and 1 partial ⚠️
interp/jit_plan.go 44.44% 1 Missing and 4 partials ⚠️
internal/cmd/geninterp/lower.go 93.75% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #161      +/-   ##
==========================================
+ Coverage   29.74%   30.13%   +0.38%     
==========================================
  Files          86       86              
  Lines       60377    59742     -635     
==========================================
+ Hits        17960    18004      +44     
+ Misses      41148    40467     -681     
- Partials     1269     1271       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
internal/cmd/geninterp/lower.go (1)

3316-3369: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Redundant if/else in mapNew now generates identical code in both branches.

After the keep(m) path was removed, both the TraceKeys || TraceValues branch and its Else() branch call i.alloc(m) — the generated Go will be:

if typ.TraceKeys || typ.TraceValues {
    addr = i.alloc(m)
} else {
    addr = i.alloc(m)
}

The branch condition no longer has any effect; this should collapse to a single unconditional assignment.

As per coding guidelines, **/*.go should follow docs/coding-patterns.md for "simplification," which this dead branch violates.

♻️ Proposed fix
-			jen.Var().Add(jen.List(jen.Id("addr"))).Add(jen.Id("int")),
-			jen.If(jen.Id("typ").Dot("TraceKeys").Op("||").Add(jen.Id("typ").Dot("TraceValues"))).Block(jen.List(jen.Id("addr")).Op("=").List(jen.Id("i").Dot("alloc").Call(jen.Id("m")))).Else().Block(jen.List(jen.Id("addr")).Op("=").List(jen.Id("i").Dot("alloc").Call(jen.Id("m")))),
+			jen.List(jen.Id("addr")).Op(":=").List(jen.Id("i").Dot("alloc").Call(jen.Id("m"))),
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/cmd/geninterp/lower.go` around lines 3316 - 3369, Remove the
redundant TraceKeys/TraceValues conditional in mapNew and replace it with one
unconditional assignment of i.alloc(m) to addr. Preserve the surrounding map
construction and stack-pointer logic unchanged.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@interp/interp.go`:
- Around line 901-906: Update the JIT native-call path in the code generating
the `LDR(callee, natives, op.callee*8)` instruction to check whether `op.callee`
is within the native table length before reading the slot. When it is out of
range, use the existing fallback path instead of emitting or executing the
native-slot load; preserve the current native dispatch for valid indices.

---

Nitpick comments:
In `@internal/cmd/geninterp/lower.go`:
- Around line 3316-3369: Remove the redundant TraceKeys/TraceValues conditional
in mapNew and replace it with one unconditional assignment of i.alloc(m) to
addr. Preserve the surrounding map construction and stack-pointer logic
unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b35a3ffa-1af6-4aff-91d4-98144cfef16e

📥 Commits

Reviewing files that changed from the base of the PR and between 8c413ae and 410fa2f.

📒 Files selected for processing (15)
  • AGENTS.md
  • docs/coding-patterns.md
  • docs/symbol-naming-audit.md
  • instr/parse.go
  • internal/cmd/geninterp/generate.go
  • internal/cmd/geninterp/lower.go
  • interp/cache.go
  • interp/interp.go
  • interp/jit_arm64.go
  • interp/jit_plan.go
  • interp/jit_plan_test.go
  • interp/marshal.go
  • interp/threaded.go
  • interp/trace.go
  • interp/trace_test.go
💤 Files with no reviewable changes (1)
  • interp/cache.go
👮 Files not reviewed due to content moderation or server errors (1)
  • interp/threaded.go

Comment thread interp/interp.go
Comment on lines +901 to 906
// natives keeps its New-time size: growing it in bind could dangle the
// journal base cached by a native frame suspended across a trap
// fallback, so dynamically bound functions never get a natives slot.
if entry.kind == entryFunction && a.addr < len(i.natives) {
atomic.StorePointer(&i.natives[a.addr], entry.callable.Addr())
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify all readers of i.natives[...] bounds-check the index before access.
rg -nP -C3 '\.natives\[' --type=go

Repository: siyul-park/minivm

Length of output: 155


🏁 Script executed:

#!/bin/bash
set -euo pipefail
git status --short
printf '\n--- interp/interp.go excerpt ---\n'
sed -n '860,940p' interp/interp.go
printf '\n--- all natives references ---\n'
rg -n -C 2 'natives' interp --type=go || true

Repository: siyul-park/minivm

Length of output: 5604


🏁 Script executed:

#!/bin/bash
set -euo pipefail
printf '%s\n' '--- interp/interp.go around native handling ---'
sed -n '1,240p' interp/interp.go
printf '\n%s\n' '--- broader natives search ---'
rg -n -C 2 'natives' . --type=go || true

Repository: siyul-park/minivm

Length of output: 7963


🏁 Script executed:

#!/bin/bash
set -euo pipefail
printf '%s\n' '--- journalNatives uses ---'
rg -n -C 3 'journalNatives|op\.callee|callee > 4095|LDR\(callee, natives' interp --type=go
printf '\n%s\n' '--- anchor/address definitions ---'
rg -n -C 2 'type anchor|struct .*anchor|a\.addr|addr int' interp --type=go
printf '\n%s\n' '--- call emission around native slot access ---'
sed -n '1380,1450p' interp/jit_arm64.go

Repository: siyul-park/minivm

Length of output: 23900


Guard the JIT native-slot read too. interp/jit_arm64.go:1421-1427 still emits LDR(callee, natives, op.callee*8) without a len(i.natives) check, so dynamically bound functions can still read past the table and crash or dispatch to garbage. Fall back when the slot is out of range.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@interp/interp.go` around lines 901 - 906, Update the JIT native-call path in
the code generating the `LDR(callee, natives, op.callee*8)` instruction to check
whether `op.callee` is within the native table length before reading the slot.
When it is out of range, use the existing fallback path instead of emitting or
executing the native-slot load; preserve the current native dispatch for valid
indices.

@siyul-park
siyul-park merged commit 7a5abf7 into main Jul 19, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant