Skip to content

test: reorganize specifications and benchmark suite#146

Merged
siyul-park merged 72 commits into
mainfrom
feature/test-benchmark-specs
Jul 14, 2026
Merged

test: reorganize specifications and benchmark suite#146
siyul-park merged 72 commits into
mainfrom
feature/test-benchmark-specs

Conversation

@siyul-park

@siyul-park siyul-park commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • reorganize tests as executable specifications around public symbols, with related cases grouped under subtests and redundant fixtures/helpers removed
  • add ownership, coverage, parity, fuzz, and concurrency checks while keeping expensive test paths bounded
  • rebuild the interpreter and VM-kernel benchmark suites with explicit lifecycle boundaries and canonical workloads
  • add optional cross-runtime comparisons for native Go, wazero, Tengo, gopher-lua, Goja, gpython, and Yaegi
  • fix correctness issues exposed by the rework, including jumps into instruction operands, unknown kind formatting, and default reference-array initialization
  • replace stale benchmark claims with the latest Apple M4 Pro median results and document the exact methodology and runtime modes

Benchmark highlights

  • RecursiveFib(35): minivm/default 47.05 ms, wazero 44.15 ms, native Go 19.13 ms
  • minivm remains zero-allocation on the measured scalar and recursive steady-state paths
  • the complete 9-workload comparison matrix, including ns/op, B/op, and allocs/op, is recorded in docs/benchmarks.md
  • jit comparisons explicitly mean WithThreshold(0) and do not claim a pre-warmed native trace

Verification

  • go test ./...
  • cd benchmarks && go test ./...
  • cd benchmarks && go test -tags=compare ./...
  • cd benchmarks && go test -tags=compare -run='^$' -bench='.' -benchmem -benchtime=300ms -count=3 ./...
  • git diff --check

Summary by CodeRabbit

  • New Features
    • Added dedicated benchmark workflows for nightly and PR runs, plus cross-runtime comparison support.
    • Introduced new benchmark/verification make targets, including coverage-check.
  • Bug Fixes
    • Strengthened verification to reject branch targets that don’t land on instruction boundaries.
    • Improved default array initialization and reference/null element handling.
  • Documentation
    • Reworked README and updated benchmark/test methodology docs with clearer execution modes and ownership/coverage expectations.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7b2bec82-7c37-4905-a26e-a0f54695a599

📥 Commits

Reviewing files that changed from the base of the PR and between 039be62 and d1510f8.

📒 Files selected for processing (3)
  • README.ko.md
  • README.md
  • docs/coding-patterns.md

📝 Walkthrough

Walkthrough

The PR adds canonical benchmark workflows and comparison harnesses, introduces testing ownership documentation and coverage enforcement, expands fuzz/example/API tests, strengthens verifier and runtime validation, and reorganizes interpreter, JIT, assembler, instruction, type, CLI, profiling, and code-generation implementations.

Changes

Benchmarking and test governance

Layer / File(s) Summary
Benchmark suite and execution tiers
benchmarks/*, Makefile, .github/workflows/*, docs/benchmarks.md
Adds canonical VM kernels, cross-runtime comparisons, benchmark targets, scheduled/manual workflows, and documented execution tiers.
Testing ownership and validation
docs/testing.md, AGENTS.md, .claude/CLAUDE.md, docs/README.md
Documents test ownership, opcode coverage, verification commands, and testing references.
Runtime and API coverage
interp/*_test.go, types/*_test.go, program/*_test.go, instr/*_test.go
Adds API, semantic parity, fuzz, example, constructor, parser, verifier, and ownership-oriented tests.
Implementation reorganization
interp/*, asm/*, instr/*, types/*, program/*, internal/cmd/geninterp/*
Reorders internal implementations, centralizes generated-output synchronization, strengthens verifier target checks, and updates selected runtime retention paths.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

Sequence Diagram(s)

sequenceDiagram
  participant CI
  participant Makefile
  participant CanonicalBenchmarks
  participant ComparisonBenchmarks
  CI->>Makefile: run benchmark-pr or benchmark-nightly
  Makefile->>CanonicalBenchmarks: execute canonical VM benchmarks
  Makefile->>ComparisonBenchmarks: execute compare-tagged runtime benchmarks
  CanonicalBenchmarks-->>CI: return benchmark report
  ComparisonBenchmarks-->>CI: return comparison report
Loading
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is relevant, but it does not follow the required template sections or include related issues and additional information. Rewrite the description using the required headings: Changes Made, Related Issues, and Additional Information, and place the current summary and verification details under them.
Docstring Coverage ⚠️ Warning Docstring coverage is 10.76% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main change: reorganizing tests and the benchmark suite.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/test-benchmark-specs

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.44346% with 287 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.36%. Comparing base (b1845d2) to head (039be62).

Files with missing lines Patch % Lines
interp/interp.go 56.38% 90 Missing and 9 partials ⚠️
interp/marshal.go 60.00% 44 Missing and 4 partials ⚠️
asm/arm64/encoder.go 63.41% 19 Missing and 11 partials ⚠️
asm/assembler.go 10.00% 18 Missing ⚠️
interp/jit_plan.go 43.33% 17 Missing ⚠️
internal/cmd/geninterp/lower.go 94.21% 8 Missing and 6 partials ⚠️
types/map.go 55.17% 13 Missing ⚠️
types/parse.go 73.68% 5 Missing and 5 partials ⚠️
types/struct.go 80.64% 6 Missing ⚠️
asm/operand.go 0.00% 5 Missing ⚠️
... and 11 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #146      +/-   ##
==========================================
+ Coverage   27.55%   29.36%   +1.81%     
==========================================
  Files          86       86              
  Lines       59954    59969      +15     
==========================================
+ Hits        16519    17612    +1093     
+ Misses      42214    41109    -1105     
- Partials     1221     1248      +27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
internal/cmd/geninterp/lower.go (1)

1465-1536: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Advance c.ip by the full fused pattern width. source.compile doesn’t move c.ip on the non-standalone numeric path, so width(first) only skips the first opcode and can leave the compiler cursor on later bytes from the same fused pattern.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/cmd/geninterp/lower.go` around lines 1465 - 1536, Update numeric to
advance c.ip by the full fused pattern width rather than width(first), using the
complete input pattern represented by inputs and the consumer. Preserve the
existing standalone-consumer behavior while ensuring source.compile bytes are
skipped exactly once on the fused numeric path.
analysis/blocks.go (1)

31-140: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Declaration order regression: Run now follows its callee Blocks.

(*BasicBlocksAnalysis).Run (the caller) was relocated to after Blocks (the callee), reversing the callers-before-callees order.

♻️ Proposed fix
+func (p *BasicBlocksAnalysis) Run(m *pass.Manager, fn *types.Function) ([]*BasicBlock, error) {
+	return Blocks(fn)
+}
+
 func Blocks(fn *types.Function) ([]*BasicBlock, error) {
 	...
 }
-
-func (p *BasicBlocksAnalysis) Run(m *pass.Manager, fn *types.Function) ([]*BasicBlock, error) {
-	return Blocks(fn)
-}

As per coding guidelines, "Follow docs/coding-patterns.md for declaration order: callers before callees, except With* option functions may appear immediately above the constructor they configure."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@analysis/blocks.go` around lines 31 - 140, Restore callers-before-callees
declaration order by moving (*BasicBlocksAnalysis).Run above the Blocks function
in the basic-block analysis declarations. Keep Run’s existing delegation to
Blocks unchanged and do not alter Blocks or other logic.

Source: Coding guidelines

🧹 Nitpick comments (5)
interp/trace_test.go (1)

250-250: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Test name doesn't track the method under test.

Per the naming convention, this should key off the method being exercised (tracer.anchors(...), line 262), e.g. TestTracer_Anchors, rather than a behavior-descriptive OrdersAnchors.

As per coding guidelines, **/*_test.go: "Tests must use one top-level test per public symbol (Test<Func> or Test<Type>_<Method>)".

✏️ Proposed rename
-func TestTracer_OrdersAnchors(t *testing.T) {
+func TestTracer_Anchors(t *testing.T) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@interp/trace_test.go` at line 250, Rename TestTracer_OrdersAnchors to
TestTracer_Anchors so the test name follows the tracer.anchors method under test
and the required Test<Type>_<Method> convention.

Source: Coding guidelines

optimize/optimizer_test.go (1)

359-414: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

New parity/execution-preservation tests skip program.Verify, unlike their sibling tests. Both newly added tests only assert interpreter-output equality and never call program.Verify on the programs involved, while every other test in these files that exercises a pass/optimizer does call it.

  • optimize/optimizer_test.go#L359-L414: add require.NoError(t, program.Verify(tt.prog)) and require.NoError(t, program.Verify(optimized)) in the "semantic parity" loop, mirroring the other O3 subtests above it.
  • transform/dce_test.go#L195-L217: add require.NoError(t, program.Verify(prog)) after building and after running NewDeadCodeEliminationPass().Run(...) in the "preserves execution" subtest.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@optimize/optimizer_test.go` around lines 359 - 414, Add program.Verify
assertions to both parity test sites: in optimize/optimizer_test.go lines
359-414, verify tt.prog before execution and optimized after
NewOptimizer(O3).Optimize; in transform/dce_test.go lines 195-217, verify prog
after construction and again after NewDeadCodeEliminationPass().Run. Preserve
the existing execution-result assertions.
transform/cd_test.go (1)

118-142: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract a shared "preserves execution" test helper. All three files implement the identical before/after interpreter-comparison sequence (interp.New(prog, WithTick(1), WithThreshold(-1)) → run → pop → apply pass → rebuild interp → run → pop → require.Equal), differing only in the program and pass under test. Consolidating into one helper avoids three copies drifting independently as the interpreter API evolves.

  • transform/cd_test.go#L118-L142: replace the inline before/after block with a call to a shared helper (e.g., requirePreservesExecution(t, prog, func(p *program.Program) error { _, err := NewConstantDeduplicationPass().Run(pass.NewManager(), p); return err })).
  • transform/cf_test.go#L2076-L2099: replace with the same shared helper, passing a closure that registers analysis.NewBasicBlocksAnalysis() before running NewConstantFoldingPass().
  • transform/as_test.go#L148-L171: replace with the same shared helper, passing a closure that registers analysis.NewBasicBlocksAnalysis() before running NewAlgebraicSimplificationPass().
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@transform/cd_test.go` around lines 118 - 142, Extract the duplicated
before/after interpreter comparison into a shared requirePreservesExecution test
helper. Update transform/cd_test.go:118-142, transform/cf_test.go:2076-2099, and
transform/as_test.go:148-171 to call it; pass each pass via a closure,
preserving BasicBlocks analysis registration for constant folding and algebraic
simplification, and return the pass error for assertion by the helper.
types/parse.go (1)

10-14: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Doc comment omits the capture-parsing step.

The comment describes only FunctionType / local types / disassembly lines, but the implementation (lines 29-43) also consumes leading capture lines between the type line and locals. Update the doc to mention captures for accuracy.

📝 Suggested doc fix
 // ParseFunction parses lines from Function.String() output:
 //
 //	line 0:       FunctionType string ("func(params) returns")
+//	lines 1..:    capture type strings, each prefixed with "capture " (optional)
 //	lines 1..k-1: local type strings (one per line)
 //	lines k..:    disassembly lines ("0000:\t…")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@types/parse.go` around lines 10 - 14, Update the ParseFunction doc comment to
document the leading capture lines consumed between the FunctionType line and
local type strings, while preserving the existing descriptions of locals and
disassembly.
instr/parse_test.go (1)

179-256: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Wrap each representative value under t.Run for consistency with the guideline.

TestReadU8TestParseI32 each bundle several independent value checks in one flat body. TestParse in this same file was just refactored to name sub-cases via t.Run; these new tests could follow the same pattern so a failing case is unambiguous and isolated.

As per coding guidelines, "/*_test.go: ... place sub-cases under t.Run ...".

♻️ Example for one of the tests
 func TestReadU8(t *testing.T) {
-	require.Equal(t, 0xAB, ReadU8(0xDEADBEEF000000AB))
-	require.Equal(t, 0xFF, ReadU8(0xFFFFFFFFFFFFFFFF))
-	require.Equal(t, 0, ReadU8(0))
+	t.Run("mid byte", func(t *testing.T) { require.Equal(t, 0xAB, ReadU8(0xDEADBEEF000000AB)) })
+	t.Run("all ones", func(t *testing.T) { require.Equal(t, 0xFF, ReadU8(0xFFFFFFFFFFFFFFFF)) })
+	t.Run("zero", func(t *testing.T) { require.Equal(t, 0, ReadU8(0)) })
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@instr/parse_test.go` around lines 179 - 256, Update TestReadU8 through
TestParseI32 so each representative input/output check runs in its own t.Run
subtest with a descriptive case name. Preserve all existing test values and
assertions while making failures independently identifiable and consistent with
TestParse.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/benchmark.yml:
- Around line 16-17: Update both checkout steps in the canonical and comparison
jobs to set persist-credentials to false, while preserving their existing
actions/checkout configuration and job behavior.

In `@asm/buffer_test.go`:
- Line 13: Update the deferred cleanup around buffer.Free in the test so it uses
a deferred closure that captures the returned error and asserts it is nil,
rather than discarding the result. Preserve the existing cleanup timing and use
the test’s established assertion mechanism.

In `@asm/buffer.go`:
- Around line 62-73: Update Buffer.Free to attempt m.free() for every mapping in
b.old and also b.mem even when an earlier cleanup fails. Record the first
returned error, clear b.old after processing all archived mappings, and return
that first error after freeing the current mapping.

In `@docs/plans/test-benchmark-deep-correction.md`:
- Around line 25-31: Insert a blank line between the progress table and the “##
Task 1: Ref-array Defaults and Broken Fixtures” heading to satisfy Markdown
formatting requirements.

In `@instr/kind_test.go`:
- Around line 30-37: Update TestKind_IsNumeric to place each IsNumeric case
under t.Run subtests, covering both expected-true numeric kinds and
expected-false nonnumeric kinds. Keep the single top-level test for the
IsNumeric symbol and preserve the existing assertions and case coverage.

In `@internal/cmd/geninterp/lower.go`:
- Around line 354-432: Update loader.constant’s KindAny out-of-bounds branch to
advance c.ip by l.width when !l.standalone before returning the deferred
segmentation-fault panic, mirroring loader.bounds. Preserve the existing
standalone behavior and panic path.

In `@interp/coroutine_test.go`:
- Around line 24-34: Refactor TestCoroutine_Refs into separate t.Run sub-cases
for the multi-field coroutine and the value-only coroutine scenarios. Keep each
scenario’s existing setup and assertions unchanged within its respective
subtest.

In `@interp/interp_test.go`:
- Around line 1560-1570: Update the “I32Add/Straight/JITCold” subtest to check
the error returned by vm.Close(), matching the error-handling pattern used by
its sibling subtests, instead of using an unchecked defer.

---

Outside diff comments:
In `@analysis/blocks.go`:
- Around line 31-140: Restore callers-before-callees declaration order by moving
(*BasicBlocksAnalysis).Run above the Blocks function in the basic-block analysis
declarations. Keep Run’s existing delegation to Blocks unchanged and do not
alter Blocks or other logic.

In `@internal/cmd/geninterp/lower.go`:
- Around line 1465-1536: Update numeric to advance c.ip by the full fused
pattern width rather than width(first), using the complete input pattern
represented by inputs and the consumer. Preserve the existing
standalone-consumer behavior while ensuring source.compile bytes are skipped
exactly once on the fused numeric path.

---

Nitpick comments:
In `@instr/parse_test.go`:
- Around line 179-256: Update TestReadU8 through TestParseI32 so each
representative input/output check runs in its own t.Run subtest with a
descriptive case name. Preserve all existing test values and assertions while
making failures independently identifiable and consistent with TestParse.

In `@interp/trace_test.go`:
- Line 250: Rename TestTracer_OrdersAnchors to TestTracer_Anchors so the test
name follows the tracer.anchors method under test and the required
Test<Type>_<Method> convention.

In `@optimize/optimizer_test.go`:
- Around line 359-414: Add program.Verify assertions to both parity test sites:
in optimize/optimizer_test.go lines 359-414, verify tt.prog before execution and
optimized after NewOptimizer(O3).Optimize; in transform/dce_test.go lines
195-217, verify prog after construction and again after
NewDeadCodeEliminationPass().Run. Preserve the existing execution-result
assertions.

In `@transform/cd_test.go`:
- Around line 118-142: Extract the duplicated before/after interpreter
comparison into a shared requirePreservesExecution test helper. Update
transform/cd_test.go:118-142, transform/cf_test.go:2076-2099, and
transform/as_test.go:148-171 to call it; pass each pass via a closure,
preserving BasicBlocks analysis registration for constant folding and algebraic
simplification, and return the pass error for assertion by the helper.

In `@types/parse.go`:
- Around line 10-14: Update the ParseFunction doc comment to document the
leading capture lines consumed between the FunctionType line and local type
strings, while preserving the existing descriptions of locals and disassembly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 559eaba0-4b3f-4c92-9e33-f1d3696d7df1

📥 Commits

Reviewing files that changed from the base of the PR and between b1845d2 and 4738363.

⛔ Files ignored due to path filters (1)
  • benchmarks/go.sum is excluded by !**/*.sum
📒 Files selected for processing (137)
  • .claude/CLAUDE.md
  • .github/workflows/benchmark.yml
  • .github/workflows/ci.yml
  • AGENTS.md
  • Makefile
  • README.md
  • analysis/blocks.go
  • analysis/blocks_test.go
  • analysis/gvn.go
  • analysis/gvn_test.go
  • asm/amd64/encoder_test.go
  • asm/arm64/encoder.go
  • asm/arm64/encoder_test.go
  • asm/arm64/frame.go
  • asm/arm64/instr.go
  • asm/arm64/instr_test.go
  • asm/assembler.go
  • asm/assembler_test.go
  • asm/buffer.go
  • asm/buffer_test.go
  • asm/instr.go
  • asm/link_test.go
  • asm/memory.go
  • asm/memory_stub.go
  • asm/operand.go
  • asm/reg.go
  • asm/reg_test.go
  • asm/rewriter.go
  • benchmarks/README.md
  • benchmarks/alloc_test.go
  • benchmarks/benchmark_test.go
  • benchmarks/call.go
  • benchmarks/call_test.go
  • benchmarks/compare_disabled_test.go
  • benchmarks/compare_runtime_test.go
  • benchmarks/compare_wazero_test.go
  • benchmarks/control.go
  • benchmarks/control_test.go
  • benchmarks/fib_test.go
  • benchmarks/fusion_test.go
  • benchmarks/go.mod
  • benchmarks/jit_issue101_test.go
  • benchmarks/jit_issue60_test.go
  • benchmarks/memory.go
  • benchmarks/memory_test.go
  • benchmarks/numeric.go
  • benchmarks/numeric_test.go
  • benchmarks/programs.go
  • cli/cli_test.go
  • cli/profile.go
  • cli/repl.go
  • cli/repl_test.go
  • debug/debugger_test.go
  • debug/example_test.go
  • docs/README.md
  • docs/benchmarks.md
  • docs/plans/test-benchmark-deep-correction.md
  • docs/plans/test-benchmark-rework.md
  • docs/testing.md
  • instr/builder_test.go
  • instr/code.go
  • instr/code_test.go
  • instr/fuzz_test.go
  • instr/instr.go
  • instr/instr_test.go
  • instr/kind.go
  • instr/kind_test.go
  • instr/opcode.go
  • instr/opcode_test.go
  • instr/parse.go
  • instr/parse_test.go
  • instr/type_test.go
  • internal/cmd/geninterp/generate.go
  • internal/cmd/geninterp/generate_test.go
  • internal/cmd/geninterp/lower.go
  • internal/cmd/geninterp/main.go
  • interp/coroutine_test.go
  • interp/error.go
  • interp/error_test.go
  • interp/example_test.go
  • interp/fuzz_test.go
  • interp/host_test.go
  • interp/interp.go
  • interp/interp_test.go
  • interp/jit.go
  • interp/jit_arm64.go
  • interp/jit_arm64_test.go
  • interp/jit_plan.go
  • interp/marshal.go
  • interp/marshal_test.go
  • interp/pool_test.go
  • interp/threaded.go
  • interp/trace_test.go
  • optimize/example_test.go
  • optimize/fuzz_test.go
  • optimize/optimizer_test.go
  • pass/manager_test.go
  • pass/pipeline_test.go
  • prof/collector.go
  • prof/collector_test.go
  • prof/jit_metrics.go
  • prof/jit_test.go
  • prof/prof_test.go
  • prof/profiler.go
  • prof/profiler_test.go
  • program/builder.go
  • program/builder_test.go
  • program/example_test.go
  • program/fuzz_test.go
  • program/parse_test.go
  • program/program_test.go
  • program/verify.go
  • program/verify_test.go
  • transform/as_test.go
  • transform/cd_test.go
  • transform/cf_test.go
  • transform/dce_test.go
  • transform/gvn_test.go
  • types/array.go
  • types/array_test.go
  • types/boxed.go
  • types/boxed_test.go
  • types/closure_test.go
  • types/error_test.go
  • types/function.go
  • types/function_test.go
  • types/fuzz_test.go
  • types/iterator_test.go
  • types/map.go
  • types/map_test.go
  • types/parse.go
  • types/primitive_test.go
  • types/string_test.go
  • types/struct.go
  • types/struct_test.go
  • types/value.go
  • types/value_test.go
💤 Files with no reviewable changes (9)
  • prof/prof_test.go
  • benchmarks/alloc_test.go
  • benchmarks/fusion_test.go
  • prof/jit_test.go
  • benchmarks/jit_issue60_test.go
  • benchmarks/fib_test.go
  • benchmarks/programs.go
  • benchmarks/jit_issue101_test.go
  • instr/instr_test.go

Comment on lines +16 to +17
- name: Checkout Code
uses: actions/checkout@v7

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Set persist-credentials: false on both checkout steps.

Neither job pushes to the repo (permissions: contents: read), yet actions/checkout@v7 defaults to persisting the GITHUB_TOKEN in the git config for the rest of the job. Since these jobs run make benchmark-nightly/make benchmark-compare, which per the PR objectives pull in third-party runtimes for comparison, an unnecessarily persisted token widens the blast radius if any of that tooling is compromised.

🔒 Proposed fix
       - name: Checkout Code
         uses: actions/checkout@v7
+        with:
+          persist-credentials: false

Apply to both the canonical and comparison job checkout steps.

Also applies to: 35-36

🧰 Tools
🪛 zizmor (1.26.1)

[warning] 16-17: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/benchmark.yml around lines 16 - 17, Update both checkout
steps in the canonical and comparison jobs to set persist-credentials to false,
while preserving their existing actions/checkout configuration and job behavior.

Source: Linters/SAST tools

Comment thread asm/buffer_test.go
func TestNewBuffer(t *testing.T) {
buffer, err := NewBuffer(1)
require.NoError(t, err)
defer buffer.Free()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Check the deferred Free error.

defer buffer.Free() discards cleanup failures, so this test can pass even if freeing the executable buffer fails. Wrap the call in a deferred closure that asserts the error.

Static analysis also reports this unchecked return value.

Proposed fix
-	defer buffer.Free()
+	defer func() {
+		require.NoError(t, buffer.Free())
+	}()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
defer buffer.Free()
defer func() {
require.NoError(t, buffer.Free())
}()
🧰 Tools
🪛 golangci-lint (2.12.2)

[error] 13-13: Error return value of buffer.Free is not checked

(errcheck)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@asm/buffer_test.go` at line 13, Update the deferred cleanup around
buffer.Free in the test so it uses a deferred closure that captures the returned
error and asserts it is nil, rather than discarding the result. Preserve the
existing cleanup timing and use the test’s established assertion mechanism.

Source: Linters/SAST tools

Comment thread asm/buffer.go
Comment on lines +62 to +73
// Free releases the current and archived mmap mappings.
func (b *Buffer) Free() error {
b.mu.Lock()
defer b.mu.Unlock()
for _, m := range b.old {
if err := m.free(); err != nil {
return err
}
}
b.old = nil
return b.mem.free()
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "## git status"
git status --short

echo
echo "## repo files of interest"
git ls-files | rg '^(AGENTS\.md|docs/coding-patterns\.md|asm/buffer\.go|asm/.*\.go|.*memory.*\.go)$'

echo
echo "## outline asm/buffer.go"
ast-grep outline asm/buffer.go --view expanded || true

echo
echo "## outline candidate memory files"
for f in $(git ls-files | rg '(^|/)memory.*\.go$|(^|/)mmap.*\.go$|(^|/)buffer.*\.go$'); do
  echo "--- $f"
  ast-grep outline "$f" --view expanded || true
done

echo
echo "## search for free methods in asm"
rg -n "func \\(.*\\) free\\(" asm || true

echo
echo "## search for Buffer.Free usage"
rg -n "func \\(b \\*Buffer\\) Free\\(|\\.Free\\(\\)" asm || true

Repository: siyul-park/minivm

Length of output: 5450


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "## docs/coding-patterns relevant excerpt"
if [ -f docs/coding-patterns.md ]; then
  rg -n -A3 -B3 "Free|resource|cleanup|error|mmap|buffer" docs/coding-patterns.md || true
fi

echo
echo "## asm/buffer.go relevant section"
sed -n '1,180p' asm/buffer.go

echo
echo "## files containing type Buffer or mmap wrapper"
rg -n "type Buffer|type .*mmap|func \\(.*\\) free\\(" asm || true

Repository: siyul-park/minivm

Length of output: 8969


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "## search for double-free guards / unmap implementation"
rg -n "munmap|Unmap|syscall\\.Munmap|Close\\(|freed|already.*free|idempot" . || true

Repository: siyul-park/minivm

Length of output: 22507


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "## targeted grep for memory.free implementation"
rg -n -A20 -B10 "func \\(.*\\) free\\(" asm || true

Repository: siyul-park/minivm

Length of output: 3856


Free() should continue cleaning up after an error.

If one archived mapping fails to unmap, the rest of b.old and b.mem stay mapped, and b.old remains populated for a retry. Free everything, then return the first error.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@asm/buffer.go` around lines 62 - 73, Update Buffer.Free to attempt m.free()
for every mapping in b.old and also b.mem even when an earlier cleanup fails.
Record the first returned error, clear b.old after processing all archived
mappings, and return that first error after freeing the current mapping.

Comment on lines +25 to +31
| Task | Scope | Status | Commit | Review |
|---|---|---|---|---|
| 1 | Ref-array defaults and broken kernel fixtures | In progress | - | - |
| 2 | Canonical kernels, JIT proof, compare isolation | Pending | - | - |
| 3 | Make targets, CI tiers, selection gates | Pending | - | - |
| 4 | Documentation, full verification, final review | Pending | - | - |
## Task 1: Ref-array Defaults and Broken Fixtures

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add blank line between table and heading.

markdownlint (MD058) flags the Progress table (lines 25-30) immediately followed by ## Task 1 with no blank line.

As per coding guidelines, **/*.md changes must "keep edits terse, factual, current, and Markdown-valid."

📝 Proposed fix
 | 4 | Documentation, full verification, final review | Pending | - | - |
+
 ## Task 1: Ref-array Defaults and Broken Fixtures
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| Task | Scope | Status | Commit | Review |
|---|---|---|---|---|
| 1 | Ref-array defaults and broken kernel fixtures | In progress | - | - |
| 2 | Canonical kernels, JIT proof, compare isolation | Pending | - | - |
| 3 | Make targets, CI tiers, selection gates | Pending | - | - |
| 4 | Documentation, full verification, final review | Pending | - | - |
## Task 1: Ref-array Defaults and Broken Fixtures
| Task | Scope | Status | Commit | Review |
|---|---|---|---|---|
| 1 | Ref-array defaults and broken kernel fixtures | In progress | - | - |
| 2 | Canonical kernels, JIT proof, compare isolation | Pending | - | - |
| 3 | Make targets, CI tiers, selection gates | Pending | - | - |
| 4 | Documentation, full verification, final review | Pending | - | - |
## Task 1: Ref-array Defaults and Broken Fixtures
🧰 Tools
🪛 markdownlint-cli2 (0.23.0)

[warning] 30-30: Tables should be surrounded by blank lines

(MD058, blanks-around-tables)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/plans/test-benchmark-deep-correction.md` around lines 25 - 31, Insert a
blank line between the progress table and the “## Task 1: Ref-array Defaults and
Broken Fixtures” heading to satisfy Markdown formatting requirements.

Sources: Coding guidelines, Linters/SAST tools

Comment thread instr/kind_test.go
Comment on lines +30 to +37
func TestKind_IsNumeric(t *testing.T) {
for _, kind := range []Kind{KindI1, KindI8, KindI32, KindI64, KindF32, KindF64} {
require.True(t, kind.IsNumeric(), kind.String())
}
for _, kind := range []Kind{KindRef, KindAny} {
require.False(t, kind.IsNumeric(), kind.String())
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Put IsNumeric cases under t.Run.

Lines 31-36 execute multiple cases without subtests, reducing failure isolation and violating the repository test structure.

As per coding guidelines: tests must use one top-level test per public symbol and place sub-cases under t.Run.

Proposed fix
 func TestKind_IsNumeric(t *testing.T) {
-	for _, kind := range []Kind{KindI1, KindI8, KindI32, KindI64, KindF32, KindF64} {
-		require.True(t, kind.IsNumeric(), kind.String())
-	}
-	for _, kind := range []Kind{KindRef, KindAny} {
-		require.False(t, kind.IsNumeric(), kind.String())
+	tests := []struct {
+		kind Kind
+		want bool
+	}{
+		{KindI1, true}, {KindI8, true}, {KindI32, true},
+		{KindI64, true}, {KindF32, true}, {KindF64, true},
+		{KindRef, false}, {KindAny, false},
+	}
+	for _, tt := range tests {
+		t.Run(tt.kind.String(), func(t *testing.T) {
+			require.Equal(t, tt.want, tt.kind.IsNumeric())
+		})
 	}
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func TestKind_IsNumeric(t *testing.T) {
for _, kind := range []Kind{KindI1, KindI8, KindI32, KindI64, KindF32, KindF64} {
require.True(t, kind.IsNumeric(), kind.String())
}
for _, kind := range []Kind{KindRef, KindAny} {
require.False(t, kind.IsNumeric(), kind.String())
}
}
func TestKind_IsNumeric(t *testing.T) {
tests := []struct {
kind Kind
want bool
}{
{KindI1, true}, {KindI8, true}, {KindI32, true},
{KindI64, true}, {KindF32, true}, {KindF64, true},
{KindRef, false}, {KindAny, false},
}
for _, tt := range tests {
t.Run(tt.kind.String(), func(t *testing.T) {
require.Equal(t, tt.want, tt.kind.IsNumeric())
})
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@instr/kind_test.go` around lines 30 - 37, Update TestKind_IsNumeric to place
each IsNumeric case under t.Run subtests, covering both expected-true numeric
kinds and expected-false nonnumeric kinds. Keep the single top-level test for
the IsNumeric symbol and preserve the existing assertions and case coverage.

Source: Coding guidelines

Comment on lines +354 to +432
func (l loader) bounds(current step, field string) (jen.Code, error) {
condition := jen.Id(l.index).Op(">=").Len(jen.Id("c").Dot(field))
if current.kind == instr.KindAny {
body := []jen.Code{}
if !l.standalone {
body = append(body, jen.Id("c").Dot("ip").Op("+=").Lit(l.width))
}
body = append(body,
jen.Return(jen.Func().Params(jen.Op("*").Id("Interpreter")).Block(jen.Panic(jen.Id("ErrSegmentationFault")))),
)
return jen.If(condition).Block(body...), nil
}
name, ok := kindName(current.kind)
if !ok {
return nil, fmt.Errorf("unsupported source kind %s", current.kind)
}
expected := jen.Qual("github.com/siyul-park/minivm/types", "Kind"+name)
return guard(field, l.index, expected, l.label), nil
}

func (l loader) constant(result *value, current step) error {
condition := jen.Id(l.index).Op(">=").Len(jen.Id("c").Dot("constants"))
if current.kind == instr.KindAny {
result.compile = append(result.compile,
jen.If(condition).Block(
jen.Return(jen.Func().Params(jen.Op("*").Id("Interpreter")).Block(jen.Panic(jen.Id("ErrSegmentationFault")))),
),
)
} else {
result.compile = append(result.compile, jen.If(condition).Block(reject(l.label)))
}
result.compile = append(result.compile, jen.Id(l.boxed).Op(":=").Id("c").Dot("constants").Index(jen.Id(l.index)))
if current.kind == instr.KindAny {
return nil
}

name, ok := kindName(current.kind)
if !ok {
return fmt.Errorf("unsupported source kind %s", current.kind)
}
expected := jen.Qual("github.com/siyul-park/minivm/types", "Kind"+name)
if name == "I64" {
result.compile = append(result.compile,
jen.Switch(jen.Id(l.boxed).Dot("Kind").Call()).Block(
jen.Case(jen.Qual("github.com/siyul-park/minivm/types", "KindI64")),
jen.Case(jen.Qual("github.com/siyul-park/minivm/types", "KindRef")).Block(
jen.Id("constantRef").Op(":=").Id(l.boxed).Dot("Ref").Call(),
jen.If(jen.Id("constantRef").Op("<").Lit(0).Op("||").Id("constantRef").Op(">=").Len(jen.Id("c").Dot("heap"))).Block(reject(l.label)),
jen.List(jen.Id("_"), jen.Id("ok")).Op(":=").Id("c").Dot("heap").Index(jen.Id("constantRef")).Assert(jen.Qual("github.com/siyul-park/minivm/types", "I64")),
jen.If(jen.Op("!").Id("ok")).Block(reject(l.label)),
),
jen.Default().Block(reject(l.label)),
),
)
} else {
result.compile = append(result.compile,
jen.If(jen.Id(l.boxed).Dot("Kind").Call().Op("!=").Add(expected)).Block(reject(l.label)),
)
}
if name != "Ref" || current.typ == nil {
return nil
}
guardName := current.typ.Name()
if guardName == "" {
return fmt.Errorf("unsupported constant guard %s", current.typ)
}
ref := fmt.Sprintf("c%d", l.slot)
result.compile = append(result.compile,
jen.Id(ref).Op(":=").Id(l.boxed).Dot("Ref").Call(),
jen.If(jen.Id(ref).Op("<").Lit(0).Op("||").Id(ref).Op(">=").Len(jen.Id("c").Dot("heap"))).Block(reject(l.label)),
)
return jen.Func().Params(jen.Id("c").Op("*").Id("threader")).Params(
jen.Func().Params(jen.Id("i").Op("*").Id("Interpreter")),
).Block(code...)
guard := jen.List(jen.Id("_"), jen.Id("ok")).Op(":=").Id("c").Dot("heap").Index(jen.Id(ref)).Assert(jen.Qual("github.com/siyul-park/minivm/types", guardName))
if current.exclude {
result.compile = append(result.compile, jen.If(guard, jen.Id("ok")).Block(reject(l.label)))
} else {
result.compile = append(result.compile, guard, jen.If(jen.Op("!").Id("ok")).Block(reject(l.label)))
}
return nil
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect reject()'s definition and the outer Compile() driver's use of c.ip
# to confirm whether constant()'s KindAny branch needs an explicit c.ip advance.
rg -n "func reject" internal/cmd/geninterp/*.go
rg -n "func .*Compile\(" internal/cmd/geninterp/*.go asm/*.go interp/*.go
rg -n "\.ip \+=|\.ip =" internal/cmd/geninterp/lower.go | head -60

Repository: siyul-park/minivm

Length of output: 626


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the relevant code paths around reject() and Compile().
sed -n '1848,1898p' internal/cmd/geninterp/lower.go
sed -n '220,340p' interp/jit.go
sed -n '340,430p' interp/jit.go

Repository: siyul-park/minivm

Length of output: 8431


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the surrounding lowering flow and all places where c.ip is advanced.
sed -n '300,470p' internal/cmd/geninterp/lower.go
printf '\n---\n'
rg -n "c\\.ip \\+=|ip \\+=|width\\)" internal/cmd/geninterp/lower.go

Repository: siyul-park/minivm

Length of output: 7939


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the lowering epilogue around the shared c.ip increment and how result.compile is emitted.
sed -n '1760,1835p' internal/cmd/geninterp/lower.go
printf '\n---\n'
sed -n '1835,1885p' internal/cmd/geninterp/lower.go

Repository: siyul-park/minivm

Length of output: 4325


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the rest of finish() and nearby emitted ip-advance logic.
sed -n '470,540p' internal/cmd/geninterp/lower.go
printf '\n---\n'
sed -n '540,620p' internal/cmd/geninterp/lower.go

Repository: siyul-park/minivm

Length of output: 5252


loader.constant() needs the same IP advance in the KindAny bailout.

The standalone path already advances c.ip in load(), but the non-standalone path reaches finish() only if this branch doesn’t return early. Here it does, so i.fr.ip += l.width never runs. Mirror bounds() and advance the IP before returning the deferred panic.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/cmd/geninterp/lower.go` around lines 354 - 432, Update
loader.constant’s KindAny out-of-bounds branch to advance c.ip by l.width when
!l.standalone before returning the deferred segmentation-fault panic, mirroring
loader.bounds. Preserve the existing standalone behavior and panic path.

Comment thread interp/coroutine_test.go
Comment on lines 24 to 34
func TestCoroutine_Refs(t *testing.T) {
co := &Coroutine{
ref: 3,
image: []types.Boxed{types.BoxI32(1), types.BoxRef(5)},
upvals: []types.Boxed{types.BoxRef(7), types.BoxF64(2)},
value: types.BoxRef(9),
}
require.ElementsMatch(t, []types.Ref{3, 5, 7, 9}, co.Refs(nil))
require.Equal(t, []types.Ref{1, 3, 5, 7, 9}, co.Refs([]types.Ref{1}))

require.Empty(t, (&Coroutine{value: types.BoxI32(1)}).Refs(nil))
require.Equal(t, []types.Ref{1}, (&Coroutine{value: types.BoxI32(1)}).Refs([]types.Ref{1}))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Split into t.Run sub-cases.

The two scenarios (multi-field coroutine vs. value-only coroutine) are distinct cases and should be under t.Run, per the test-file guideline.

♻️ Proposed refactor
 func TestCoroutine_Refs(t *testing.T) {
-	co := &Coroutine{
-		ref:    3,
-		image:  []types.Boxed{types.BoxI32(1), types.BoxRef(5)},
-		upvals: []types.Boxed{types.BoxRef(7), types.BoxF64(2)},
-		value:  types.BoxRef(9),
-	}
-	require.Equal(t, []types.Ref{1, 3, 5, 7, 9}, co.Refs([]types.Ref{1}))
-
-	require.Equal(t, []types.Ref{1}, (&Coroutine{value: types.BoxI32(1)}).Refs([]types.Ref{1}))
+	t.Run("mixed refs", func(t *testing.T) {
+		co := &Coroutine{
+			ref:    3,
+			image:  []types.Boxed{types.BoxI32(1), types.BoxRef(5)},
+			upvals: []types.Boxed{types.BoxRef(7), types.BoxF64(2)},
+			value:  types.BoxRef(9),
+		}
+		require.Equal(t, []types.Ref{1, 3, 5, 7, 9}, co.Refs([]types.Ref{1}))
+	})
+
+	t.Run("no refs", func(t *testing.T) {
+		require.Equal(t, []types.Ref{1}, (&Coroutine{value: types.BoxI32(1)}).Refs([]types.Ref{1}))
+	})
 }
As per coding guidelines, "place sub-cases under `t.Run`".
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func TestCoroutine_Refs(t *testing.T) {
co := &Coroutine{
ref: 3,
image: []types.Boxed{types.BoxI32(1), types.BoxRef(5)},
upvals: []types.Boxed{types.BoxRef(7), types.BoxF64(2)},
value: types.BoxRef(9),
}
require.ElementsMatch(t, []types.Ref{3, 5, 7, 9}, co.Refs(nil))
require.Equal(t, []types.Ref{1, 3, 5, 7, 9}, co.Refs([]types.Ref{1}))
require.Empty(t, (&Coroutine{value: types.BoxI32(1)}).Refs(nil))
require.Equal(t, []types.Ref{1}, (&Coroutine{value: types.BoxI32(1)}).Refs([]types.Ref{1}))
}
func TestCoroutine_Refs(t *testing.T) {
t.Run("mixed refs", func(t *testing.T) {
co := &Coroutine{
ref: 3,
image: []types.Boxed{types.BoxI32(1), types.BoxRef(5)},
upvals: []types.Boxed{types.BoxRef(7), types.BoxF64(2)},
value: types.BoxRef(9),
}
require.Equal(t, []types.Ref{1, 3, 5, 7, 9}, co.Refs([]types.Ref{1}))
})
t.Run("no refs", func(t *testing.T) {
require.Equal(t, []types.Ref{1}, (&Coroutine{value: types.BoxI32(1)}).Refs([]types.Ref{1}))
})
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@interp/coroutine_test.go` around lines 24 - 34, Refactor TestCoroutine_Refs
into separate t.Run sub-cases for the multi-field coroutine and the value-only
coroutine scenarios. Keep each scenario’s existing setup and assertions
unchanged within its respective subtest.

Source: Coding guidelines

Comment thread interp/interp_test.go
Comment on lines +1560 to +1570
if runtime.GOARCH == "arm64" {
t.Run("I32Add/Straight/JITCold", func(t *testing.T) {
vm := New(program.New(benchmarkNumeric), WithTick(1), WithThreshold(0))
defer vm.Close()

require.NoError(t, vm.Run(context.Background()))
value, err := vm.Pop()
require.NoError(t, err)
require.Equal(t, types.I32(42), value)
require.Greater(t, vm.samples.Value("vm_jit_emits_total"), float64(0))
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Check vm.Close()'s error instead of a bare defer.

golangci-lint (errcheck) flags this unchecked defer vm.Close(). The two sibling subtests added in this same block ("Array/Get/JITExit", "I32Div/JITDeopt") already check the error via require.NoError(t, vm.Close()) — this one is the odd one out.

🔧 Proposed fix
 		t.Run("I32Add/Straight/JITCold", func(t *testing.T) {
 			vm := New(program.New(benchmarkNumeric), WithTick(1), WithThreshold(0))
-			defer vm.Close()
+			defer func() { require.NoError(t, vm.Close()) }()
 
 			require.NoError(t, vm.Run(context.Background()))
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if runtime.GOARCH == "arm64" {
t.Run("I32Add/Straight/JITCold", func(t *testing.T) {
vm := New(program.New(benchmarkNumeric), WithTick(1), WithThreshold(0))
defer vm.Close()
require.NoError(t, vm.Run(context.Background()))
value, err := vm.Pop()
require.NoError(t, err)
require.Equal(t, types.I32(42), value)
require.Greater(t, vm.samples.Value("vm_jit_emits_total"), float64(0))
})
t.Run("I32Add/Straight/JITCold", func(t *testing.T) {
vm := New(program.New(benchmarkNumeric), WithTick(1), WithThreshold(0))
defer func() { require.NoError(t, vm.Close()) }()
require.NoError(t, vm.Run(context.Background()))
value, err := vm.Pop()
require.NoError(t, err)
require.Equal(t, types.I32(42), value)
require.Greater(t, vm.samples.Value("vm_jit_emits_total"), float64(0))
})
🧰 Tools
🪛 golangci-lint (2.12.2)

[error] 1563-1563: Error return value of vm.Close is not checked

(errcheck)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@interp/interp_test.go` around lines 1560 - 1570, Update the
“I32Add/Straight/JITCold” subtest to check the error returned by vm.Close(),
matching the error-handling pattern used by its sibling subtests, instead of
using an unchecked defer.

Source: Linters/SAST tools

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@benchmarks/compare_test.go`:
- Around line 128-132: Update the result-conversion logic in
benchmarks/compare_test.go at lines 128-132, 167-172, 204-205, and 215-216 to
compare each native lua.LNumber value before converting it. Validate that the
runtime number is an exact integer within int32 range, then cast it to int32
only after validation; preserve the existing comparison behavior and error
handling.

In `@docs/benchmarks.md`:
- Line 74: Update the benchmark timing description around “result recovery” to
explicitly distinguish runner-specific boundaries: note that Tengo’s
compiled.Get and Gpython’s integer conversion occur after b.StopTimer(), while
Lua and Goja perform result conversion inside the timed invocation. Clarify that
timing and allocation columns are therefore not directly equivalent across
runtimes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 38aa8cd1-ddf0-4532-b09d-c79e20cbae72

📥 Commits

Reviewing files that changed from the base of the PR and between 4738363 and 039be62.

📒 Files selected for processing (8)
  • .github/workflows/ci.yml
  • .gitignore
  • Makefile
  • benchmarks/benchmark_test.go
  • benchmarks/compare_test.go
  • docs/benchmarks.md
  • docs/testing.md
  • interp/interp_test.go
💤 Files with no reviewable changes (1)
  • .github/workflows/ci.yml
🚧 Files skipped from review as they are similar to previous changes (2)
  • Makefile
  • benchmarks/benchmark_test.go

Comment on lines +128 to +132
number, ok := result.(lua.LNumber)
if !ok {
return fmt.Errorf("run returned %s", result.Type())
}
value = int32(number)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Map the benchmark file structure first.
ast-grep outline benchmarks/compare_test.go --view expanded || true

# Show the relevant sections with line numbers.
sed -n '1,260p' benchmarks/compare_test.go | nl -ba | sed -n '1,260p'

# Find the exact helper/closure names involved.
rg -n "int32\\(|lua\\.LNumber|ToInteger\\(|py\\.Int|goja\\.Undefined|result :=|call\\(" benchmarks/compare_test.go

Repository: siyul-park/minivm

Length of output: 1718


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Probe the semantics of the involved conversions in a read-only way.
cat > /tmp/verify_numeric_conversions.go <<'EOF'
package main

import (
	"fmt"
	"math"
)

func main() {
	fmt.Println("int32(42.5) =", int32(42.5))
	fmt.Println("int32(2147483648) =", int32(2147483648))
	fmt.Println("int32(-2147483649) =", int32(-2147483649))
	fmt.Println("math.Trunc(42.5) == 42.5:", math.Trunc(42.5) == 42.5)
}
EOF

go run /tmp/verify_numeric_conversions.go

Repository: siyul-park/minivm

Length of output: 437


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '110,230p' benchmarks/compare_test.go | nl -ba

Repository: siyul-park/minivm

Length of output: 195


🏁 Script executed:

#!/bin/bash
set -euo pipefail
python3 - <<'PY'
print("probe")
PY

Repository: siyul-park/minivm

Length of output: 161


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the exact benchmark comparison sites.
sed -n '120,225p' benchmarks/compare_test.go | awk '{printf "%4d  %s\n", NR+119, $0}'

Repository: siyul-park/minivm

Length of output: 3149


🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '120,225p' benchmarks/compare_test.go | awk '{printf "%4d  %s\n", NR+119, $0}'

Repository: siyul-park/minivm

Length of output: 3149


🌐 Web query:

gpython py.Int underlying type documentation

💡 Result:

In the gpython project, which is a partial implementation of the Python 3.4 interpreter written in Go, py.Int serves as a wrapper type to represent Python integers [1][2][3]. Underlyingly, py.Int is defined as a type based on the Go int64 primitive [1]. As a Go-based implementation of Python, gpython uses native Go types for several Python objects to facilitate performance and integration [1]. Specifically, the gpython codebase utilizes py.Int as an immutable type that is passed by value within the interpreter's runtime [1]. While py.Int corresponds to standard integer values, gpython also includes py.BigInt for cases requiring arbitrary-precision integers, and internal operations often convert between these types or to an int64 representation during hash calculations [1]. For further technical context, you can examine the gpython source code repository directly, specifically files within the py/ directory where such core types are defined [1][2][4].

Citations:


🌐 Web query:

goja Value.ToInteger truncates float documentation

💡 Result:

In the Goja library, the Value.ToInteger method behaves by converting a value to an integer by truncating the fractional part of a floating-point number, consistent with standard ECMAScript ToInteger conversion rules [1][2]. When called on a floating-point value, Goja internally utilizes logic that effectively truncates the number towards zero [1][2]. Specifically, implementations typically use a helper function (often named floatToIntClip or similar) that converts the float64 to an int64 [1][2]. This process: 1. Handles special cases: NaN values are converted to 0 [1][2]. 2. Clamps the result: Values exceeding the range of int64 are clamped to math.MaxInt64 or math.MinInt64 [1][2]. 3. Truncates: For finite numbers within the integer range, it performs a standard truncation of the decimal component [1][2]. This behavior aligns with the JavaScript specification, which requires that conversion to an integer type discards the fractional part [3]. Because Goja is designed to execute ECMAScript, it adheres to these specifications [3]. Developers using Goja should be aware that this truncation to an integer can result in a loss of precision, and in some contexts (such as setting timeouts or intervals in integrated systems like k6), passing non-integer values may lead to unexpected results or the need for explicit float-based handling if fractional precision is required [3].

Citations:


🌐 Web query:

goja Value.ToInteger method docs ECMAScript truncates NaN Infinity

💡 Result:

In the Goja library, the Value.ToInteger method does not simply truncate values like NaN or Infinity; instead, it adheres to specific handling logic designed to convert ECMAScript values to a 64-bit integer (int64) [1][2][3]. When ToInteger is called on a floating-point value in Goja, it utilizes an internal function—commonly referred to in the source code as floatToIntClip—which processes the number using the following rules [3][4]: 1. NaN: If the value is NaN, it returns 0 [3][4]. 2. Infinity: If the value is positive infinity, it is clipped to the maximum value of a 64-bit integer (math.MaxInt64). If it is negative infinity, it is clipped to the minimum value of a 64-bit integer (math.MinInt64) [3][4]. 3. Finite Numbers: If the number is finite, it is cast to int64, while also being clipped to the bounds of the int64 range (math.MinInt64 to math.MaxInt64) [3][4]. This behavior differs from a simple truncation or a raw float-to-int conversion, as it explicitly handles non-finite numbers by clipping them to the edges of the representable 64-bit integer range [3][4]. You can find these implementations in the value.go file within the Goja source code [2][3][4].

Citations:


🌐 Web query:

pkg.go.dev goja Value ToInteger int64 truncates

💡 Result:

The ToInteger method on the goja Value interface returns an int64 representation of the JavaScript value, adhering to ECMAScript specifications which involve truncation towards zero [1][2]. In the Goja implementation, ToInteger performs the conversion by truncating any fractional part, effectively rounding towards zero [1][3]. For example: - Integer values (valueInt) are returned as their underlying int64 representation [1][2]. - Floating-point values (valueFloat) are converted by truncating the decimal portion [1][2]. - Non-numeric types are converted according to JavaScript rules (e.g., null, undefined, or false often result in 0) [1][2]. Because the return type of ToInteger is int64, any JavaScript number that exceeds the range of an int64 (or is NaN/Infinity) will be subjected to the limitations of the Go int64 type after the conversion process, which typically involves standard Go integer overflow behavior if the resulting truncated value cannot be represented [1][2]. In practical usage within Goja, ToInteger is intended to implement the ECMAScript ToInteger abstract operation, which explicitly specifies truncation of fractional parts [1][3].

Citations:


🌐 Web query:

gopher-lua LNumber underlying float64 docs

💡 Result:

In GopherLua, the LNumber type is explicitly defined as a float64 [1][2][3]. This design choice mirrors the behavior of the original Lua 5.1 specification, where numbers are represented as double-precision floating-point values [4]. Because LNumber is a float64, it can represent integers exactly only up to the limit of safe integer precision for floating-point numbers (typically 2^53) [5][4]. Developers should be aware that conversions involving large integers may result in a loss of precision [4]. Key implementation details include: - Underlying Type: LNumber is a type alias for float64 [1][2]. - Type System: It satisfies the LValue interface [1][2]. - Utility Functions: The library provides helper functions like isInteger to check if an LNumber can be represented as an integer without losing fractional data [5][6]. - String Formatting: When converting to a string, the library checks if the value is an integer; if so, it formats it as an integer, otherwise it formats it as a float [7][8].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show the benchmark cases and the expressions that determine expected outputs.
rg -n "benchmark(GopherLua|Goja|Gpython|Yaegi|Tengo|Native|Wazero)|want|fib|sieve|sum|branchTree|iterative|recursive|typedArray" benchmarks/compare_test.go

# Print the whole file in manageable chunks around the benchmark definitions.
sed -n '1,120p' benchmarks/compare_test.go
sed -n '120,240p' benchmarks/compare_test.go
sed -n '240,420p' benchmarks/compare_test.go
sed -n '420,620p' benchmarks/compare_test.go

Repository: siyul-park/minivm

Length of output: 19537


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect benchmark sources used by the comparison helpers.
sed -n '1,240p' benchmarks/compare_test.go | awk '{printf "%4d  %s\n", NR, $0}'

Repository: siyul-park/minivm

Length of output: 7182


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Collect exact lines for the benchmark cases and return-value handling.
python3 - <<'PY'
from pathlib import Path
p = Path('benchmarks/compare_test.go')
for i, line in enumerate(p.read_text().splitlines(), 1):
    if any(k in line for k in ('benchmarkGopherLua', 'benchmarkGoja', 'benchmarkGpython', 'benchmarkYaegi',
                               'benchmarkTengo', 'benchmarkNative', 'benchmarkWazero',
                               'want', 'fib', 'sieve', 'sum', 'branchTree', 'iterative', 'recursive', 'typedArray')):
        print(f"{i:4d}  {line}")
PY

Repository: siyul-park/minivm

Length of output: 3265


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p = Path('benchmarks/compare_test.go')
for i, line in enumerate(p.read_text().splitlines(), 1):
    if any(k in line for k in (
        'benchmarkGopherLua', 'benchmarkGoja', 'benchmarkGpython', 'benchmarkYaegi',
        'benchmarkTengo', 'benchmarkNative', 'benchmarkWazero',
        'want', 'fib', 'sieve', 'sum', 'branchTree', 'iterative', 'recursive', 'typedArray'
    )):
        print(f"{i:4d}  {line}")
PY

Repository: siyul-park/minivm

Length of output: 3265


Avoid truncating runtime results before the comparison
int32(...) here can hide a bad return value by truncating or wrapping before the assertion. Compare the native runtime value first, then cast only after confirming it is an exact int32-range integer.

  • benchmarks/compare_test.go#L128-L132
  • benchmarks/compare_test.go#L167-L172
  • benchmarks/compare_test.go#L204-L205
  • benchmarks/compare_test.go#L215-L216
🧰 Tools
🪛 ast-grep (0.44.1)

[warning] 131-131: Narrowing a non-constant integer to a smaller fixed-width type (int8/int16/int32, uint8/uint16/uint32) can silently overflow or wrap, yielding negative or truncated values that are dangerous in size, length, or index logic. Validate the source value is within the target type's range before converting (e.g. bounds-check, or use a checked helper), and avoid narrowing untrusted or len()/parsed values.
Context: int32(number)
Note: [CWE-190] Integer Overflow or Wraparound.

(integer-overflow-narrowing-conversion-go)

📍 Affects 1 file
  • benchmarks/compare_test.go#L128-L132 (this comment)
  • benchmarks/compare_test.go#L167-L172
  • benchmarks/compare_test.go#L204-L205
  • benchmarks/compare_test.go#L215-L216
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmarks/compare_test.go` around lines 128 - 132, Update the
result-conversion logic in benchmarks/compare_test.go at lines 128-132, 167-172,
204-205, and 215-216 to compare each native lua.LNumber value before converting
it. Validate that the runtime number is an exact integer within int32 range,
then cast it to int32 only after validation; preserve the existing comparison
behavior and error handling.

Source: Linters/SAST tools

Comment thread docs/benchmarks.md
The `jit` label therefore means **threshold zero**, not “fully warmed native code.” It can be slower than `default` when early compilation produces incomplete traces or when the workload is dominated by unsupported allocation and mutation paths.

### Warmup vs. Steady-State Allocations
Each runtime measures an already prepared callable through result recovery. Compilation, module construction, fixture injection, warmup, and minivm `Reset` are excluded. Runtime-specific host-call and result-conversion costs remain part of the measured invocation, so small differences should not be interpreted as VM-core instruction throughput alone.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Clarify the benchmark timing boundary in docs/benchmarks.md:74

This description implies result conversion is timed uniformly, but Tengo’s compiled.Get and Gpython’s integer conversion happen after b.StopTimer(), while Lua and Goja do their conversions inside the timed call. Call out those runner-specific differences so the timing and allocation columns aren’t treated as equivalent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/benchmarks.md` at line 74, Update the benchmark timing description
around “result recovery” to explicitly distinguish runner-specific boundaries:
note that Tengo’s compiled.Get and Gpython’s integer conversion occur after
b.StopTimer(), while Lua and Goja perform result conversion inside the timed
invocation. Clarify that timing and allocation columns are therefore not
directly equivalent across runtimes.

@siyul-park
siyul-park merged commit 7717228 into main Jul 14, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant