Skip to content

refactor(geninterp): unify opcode lowering semantics#142

Merged
siyul-park merged 26 commits into
mainfrom
feature/geninterp-lowering
Jul 12, 2026
Merged

refactor(geninterp): unify opcode lowering semantics#142
siyul-park merged 26 commits into
mainfrom
feature/geninterp-lowering

Conversation

@siyul-park

@siyul-park siyul-park commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Changes Made

  • unify standalone and fusion generation behind one opcode-indexed lowering table
  • represent lowering with explicit pattern matches, resolved steps, virtual values, and state
  • use the same semantic paths for source, ref, index, call, scalar, and branch operations
  • remove duplicated standalone/fusion renderers and family-specific validation switches
  • preserve consumed-vs-borrowed ownership for promoted I64 and ref values
  • preserve exact stack, frame, IP, trap, and reference-count behavior across fused execution
  • preserve fusion cursor advancement by the first absorbed opcode
  • simplify source loading and call/frame generation without adding runtime IR or dispatch helpers
  • standardize generator symbols with concise role names such as source, ref, index, call, scalar, and branch
  • update fusion and opcode-extension documentation to match the implemented pipeline

Related Issues

None.

Additional Information

Validation completed with:

  • make check
  • make check-generated
  • exact/fusion ownership parity tests for promoted I64 and refs
  • exact/fusion trap-state parity tests
  • Linux ARM64 build and test coverage
  • race, vet, formatting, and module-tidy checks

The final naming cleanup leaves generated interp/threaded.go byte-for-byte unchanged. The completed semantic refactor keeps all measured fusion benchmarks at 0 B/op and 0 allocs/op; the full benchmark comparison against main produced a -2.12% geomean runtime change.

Summary by CodeRabbit

  • Performance

    • Improved threaded interpreter execution with more consistent instruction handling and optimized operation fusion.
  • Bug Fixes

    • Improved stack safety and bounds checking across instructions.
    • Corrected reference ownership/retention behavior during fused execution.
    • Ensured fused-mode behavior matches exact execution for numeric traps and reference dropping/ownership, strengthening error handling (e.g., overflow and divide-by-zero).
  • Documentation

    • Clarified fusion pattern generation behavior and stricter interpreter opcode contributor guidance, including updated verification commands.

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 368f91e0-a390-48a1-900b-53eb66d5eb72

📥 Commits

Reviewing files that changed from the base of the PR and between 5bb735d and 30ff7d7.

📒 Files selected for processing (5)
  • internal/cmd/geninterp/generate_test.go
  • internal/cmd/geninterp/lower.go
  • internal/cmd/geninterp/validate.go
  • interp/interp_test.go
  • interp/threaded.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/cmd/geninterp/validate.go
  • internal/cmd/geninterp/lower.go
👮 Files not reviewed due to content moderation or server errors (1)
  • interp/threaded.go

📝 Walkthrough

Walkthrough

The PR replaces legacy fusion lowering with a catalog, resolution, and composition pipeline, updates threaded interpreter handlers and ownership semantics, strengthens generator validation, and adds exact-versus-fused runtime equivalence tests and contributor documentation.

Changes

Fusion lowering and threaded execution

Layer / File(s) Summary
Pattern descriptors and validation
internal/cmd/geninterp/pattern.go, internal/cmd/geninterp/validate.go, docs/fusion.md
Pattern matching now uses exclusion-aware descriptors, stack simulation, overlap checks, explicit composition validation, and stricter fusion-generation rules.
Lowering IR and generated composition
internal/cmd/geninterp/lower.go, internal/cmd/geninterp/generate.go, internal/cmd/geninterp/generate_test.go, docs/guides/add-opcode.md
Unified lowering descriptors and loaders emit standalone and composed handlers through lowerers; generator tests cover deterministic output, composition, mapping, and validation failures.
Threaded runtime semantics and equivalence tests
interp/threaded.go, interp/interp_test.go
Handlers update instruction-pointer movement, stack/reference handling, data access, comparisons, and frame setup; tests compare exact and fused execution state, ownership, and divide-by-zero traps.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

  • siyul-park/minivm#115 — Directly overlaps the threaded interpreter correctness and generator-driven fusion behavior changed here.

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main geninterp refactor.
Description check ✅ Passed The description follows the required template and includes all requested sections.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/geninterp-lowering

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.44.1)
interp/threaded.go

ast-grep timed out on this file


Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.11321% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 26.95%. Comparing base (bd1ef28) to head (5bb735d).

Files with missing lines Patch % Lines
internal/cmd/geninterp/validate.go 96.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #142      +/-   ##
==========================================
- Coverage   27.76%   26.95%   -0.82%     
==========================================
  Files          83       83              
  Lines       60416    59198    -1218     
==========================================
- Hits        16776    15957     -819     
+ Misses      42447    42024     -423     
- Partials     1193     1217      +24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
interp/interp_test.go (1)

1908-1927: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicated snapshot/live-refcount helpers across the three new sub-tests.

Each of the three new t.Run blocks defines its own snapshot type and a nearly identical run closure, all repeating the same live-refcount loop:

live := 0
for _, rc := range i.rc[1:] {
    if rc > 0 {
        live += rc
    }
}

Additionally, factoring run out as a closure shared across multiple cases entries departs from every other multi-case test in this file, which inlines setup/execution/assertions directly inside each t.Run(tt.name, func(t *testing.T) {...}) (e.g. the "host call releases the consumed callable ref...", "fused UPVAL_GET+CONST binop..." tables). As per coding guidelines, **/*_test.go should "Inline test setup, execution, and assertions unless the documented exception in §6.8 applies" — please confirm this shared-helper pattern falls under that documented exception, since it's a new style relative to the rest of the file.

At minimum, the duplicated live-refcount computation is worth consolidating.

♻️ Proposed dedup for the live-refcount computation
+func liveRefCount(i *Interpreter) int {
+	live := 0
+	for _, rc := range i.rc[1:] {
+		if rc > 0 {
+			live += rc
+		}
+	}
+	return live
+}

Then each run closure becomes live: liveRefCount(i).

Also applies to: 1979-1996, 2045-2063

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@interp/interp_test.go` around lines 1908 - 1927, Consolidate the repeated
live-reference counting from the three test cases into a shared liveRefCount
test helper, then have each run closure use it when populating snapshot.live.
Review the local snapshot/run helpers in the affected t.Run blocks and inline
setup, execution, and assertions unless the project’s documented exception
explicitly permits this shared closure pattern; preserve all existing snapshot
fields and test behavior.

Source: Coding guidelines

internal/cmd/geninterp/generate_test.go (1)

249-336: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff

This subtest asserts private implementation shape rather than behavior.

uses role based lowering names parses the generator's own sources and asserts exact struct field names/ordering (Lines 288-293) and the presence/absence of internal helper function names (Lines 295-318). This couples the test to internal structure rather than observable generator behavior, so ordinary refactors (renaming a helper, reordering a field) will break it even when generated output is unchanged.

Consider narrowing this to behavior that matters (e.g., generated-output assertions like the neighboring compose subtests), or at minimum limiting it to the naming invariants you truly want to freeze.

As per coding guidelines: "Tests must follow docs/coding-patterns.md §6 and assert behavior rather than private implementation shape."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/cmd/geninterp/generate_test.go` around lines 249 - 336, The subtest
uses AST inspection in “uses role based lowering names” to freeze private type
fields and helper names instead of validating behavior. Replace the exact
struct-field and function presence/absence assertions with generated-output or
other observable behavior assertions, retaining only naming invariants required
by the public generator contract and preserving the existing compose/lower
behavior checks.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/cmd/geninterp/validate.go`:
- Around line 97-103: Update validateStack so opcodes where typ.Pop and typ.Push
are both nil are skipped with continue rather than terminating validation. This
must allow later instructions in the same pattern to continue through the stack
checks while preserving existing behavior for effectful opcodes.

In `@interp/threaded.go`:
- Around line 10520-10522: The fused push overflow guard incorrectly checks
i.sp+1, causing false panics with one free stack slot and missing overflow when
the stack is full. Update the generator logic in overflowAt or the fused push
path in internal/cmd/geninterp/lower.go so the emitted guard checks the actual
second-write condition, using i.sp == len(i.stack), while preserving correct
overflow handling for both pushes.

---

Nitpick comments:
In `@internal/cmd/geninterp/generate_test.go`:
- Around line 249-336: The subtest uses AST inspection in “uses role based
lowering names” to freeze private type fields and helper names instead of
validating behavior. Replace the exact struct-field and function
presence/absence assertions with generated-output or other observable behavior
assertions, retaining only naming invariants required by the public generator
contract and preserving the existing compose/lower behavior checks.

In `@interp/interp_test.go`:
- Around line 1908-1927: Consolidate the repeated live-reference counting from
the three test cases into a shared liveRefCount test helper, then have each run
closure use it when populating snapshot.live. Review the local snapshot/run
helpers in the affected t.Run blocks and inline setup, execution, and assertions
unless the project’s documented exception explicitly permits this shared closure
pattern; preserve all existing snapshot fields and test behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a71e2af3-f4be-4324-ab7a-4057e1761f85

📥 Commits

Reviewing files that changed from the base of the PR and between bd1ef28 and 5bb735d.

📒 Files selected for processing (9)
  • docs/fusion.md
  • docs/guides/add-opcode.md
  • internal/cmd/geninterp/generate.go
  • internal/cmd/geninterp/generate_test.go
  • internal/cmd/geninterp/lower.go
  • internal/cmd/geninterp/pattern.go
  • internal/cmd/geninterp/validate.go
  • interp/interp_test.go
  • interp/threaded.go

Comment thread internal/cmd/geninterp/validate.go
Comment thread interp/threaded.go Outdated
@siyul-park

Copy link
Copy Markdown
Owner Author

Review follow-up:

  • Removed the AST-based private naming/field-shape test; behavior and generated-output tests remain.
  • Did not add a package-level liveRefCount test helper. docs/coding-patterns.md §6.8 defaults to local visible test flow and permits helpers only when clearly better; the proposed helper would add shared white-box test API for a small loop. Existing snapshot closures remain local to their exact/fused comparison cases.
  • Fixed both functional findings in 30ff7d7 with regression coverage.
  • Local make check passes.

@siyul-park
siyul-park merged commit 0978e9d into main Jul 12, 2026
2 of 3 checks passed
@siyul-park
siyul-park deleted the feature/geninterp-lowering branch July 12, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant