Skip to content

feat(interp): run VM calls from host functions nested - #310

Merged
siyul-park merged 2 commits into
mainfrom
feat/reentrant-invoke
Nov 10, 2026
Merged

siyul-park merged 2 commits into
mainfrom
feat/reentrant-invoke

Conversation

@siyul-park

Copy link
Copy Markdown
Owner

Summary

Fixes #250. A Go func bound to a VM function, for example through Unmarshal, returned ErrInterpreterBusy whenever Run was active. So a host function could not call back into the VM, e.g. a comparator for a host sort.

  • Interpreter.hosts counts running host calls. The generated host call increments it before Fn and decrements it after a normal return. Run clears it on return, since a Fn that panicked skips the decrement.
  • invoke now has three cases:
    • No Run active: the path is unchanged.
    • Run active and a host function running: nest.
    • Anything else during Run: ErrInterpreterBusy.
  • nest follows engine.reenter's pattern:
    1. Push the callee and arguments above the host's stack. The host's argument slice stays untouched.
    2. Write a CALL-only frame and raise floor to it, so handler search and the dispatch recover stop at the boundary.
    3. Dispatch on the invoke's context.
    4. Restore fr/fp/floor/hosts/ctx/done/the frame slot.
      On a trap or cancellation it unwinds the nested frames and stack, and returns the error to the Go func.
  • No JIT change is needed. A native ExitCall to a host target deopts and replays, so a host function always runs from threaded code. Inside a served call (reenter), engine.depth is already raised, so a nested dispatch that enters native code nests below the suspended activations. That last part relies on fix(jit): abandon only the native stack the latest entry started #308.
  • A nested call cannot yield: coroutines do not cross the host boundary. This is documented.

Tests (TestRegistry_Unmarshal subcases, red on main with interpreter busy)

  • A host function calls a VM function 100 times, threaded and with WithThreshold(1). Results are correct and live refs return to baseline.
  • Divide-by-zero in the nested call, uncaught: Run errors with ErrDivideByZero and refs balance after Reset.
  • The same trap caught by a guest try around the host call. The caught code is the trap's own (TrapCodeDivideByZero), because the host returns the nested *RuntimeError.
  • A cancelled context in the nested call gives context.Canceled.
  • A hook calling the func during Run, outside any host function, still gets ErrInterpreterBusy.
  • go test -race on the interp tests, make check (pre-commit) and git diff --check: pass.

Docs

docs/host-integration.md, README.md, README.ko.md, and the Go docs of HostFunction and ErrInterpreterBusy.

Not covered

No JIT case for a trap inside a nested native call. Threshold 1 runs the success path only.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 08e00cbe-7b28-4544-9b44-3ce4b3341484
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@siyul-park
siyul-park force-pushed the feat/reentrant-invoke branch from 8a3394d to be589af Compare November 9, 2026 23:40
A Go func bound to a VM function returned ErrInterpreterBusy whenever
Run was active, so a host function could not call back into the VM
(a comparator for a host sort, for example).

Host calls now count themselves in Interpreter.hosts. While one runs,
invoke pushes the callee and arguments above the host's stack, writes a
CALL-only frame, raises the floor to it, and dispatches nested on the
same interpreter, restoring every saved register afterwards. A trap or
cancellation unwinds the nested frames and returns as the Go func's
error. ErrInterpreterBusy remains for calls during Run from outside a
host function.

Fixes #250

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@siyul-park
siyul-park force-pushed the feat/reentrant-invoke branch from 22c4cae to bba9d39 Compare November 10, 2026 00:02
@siyul-park
siyul-park merged commit 77348eb into main Nov 10, 2026
2 checks passed
@siyul-park
siyul-park deleted the feat/reentrant-invoke branch November 10, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(interp): re-entrant calls into the VM from host functions

1 participant