siyul-park · siyul-park · Jul 18, 2026 · Jul 18, 2026 · Jul 18, 2026 · Jul 18, 2026
diff --git a/docs/benchmarks.md b/docs/benchmarks.md
@@ -21,7 +21,7 @@ For implementation details, see `docs/jit-internals.md`. For profiling counters,
 
 ## Measurement Environment
 
-All numbers in this document were measured on July 16, 2026:
+All numbers in this document were measured on July 16, 2026 (the `Sieve(256)` minivm rows were re-measured on July 18, 2026 after issue #155):
 
 - Apple M4 Pro, 12 cores
 - `darwin/arm64`
@@ -59,8 +59,9 @@ go test -run='^$' -bench='^Benchmark(Array|Struct|TypedMap|Map)_Refs$' \
 
 - `RecursiveFib(35)` places `minivm/default` at **46.30 ms**, within about **2.6%** of wazero's **45.11 ms**, while remaining allocation-free after warmup.
 - Adaptive native traces reduce `IterativeFib(30)` from **738.6 ns** threaded to **76.14 ns**, `TypedArraySum(256)` from **6.299 us** to **669.0 ns**, and `BranchTree(96)` from **981.8 ns** to **235.1 ns**.
-- Primitive array mutation stays on the native loop path in `Sieve(256)`: deferred-ownership elision drops the per-element retain/release pair, so a runtime-allocated array reaches the same cheap native path a typed-array constant already used. `default` is **4.722 us** and eager `jit` is **4.719 us**, versus **16.172 us** threaded. All three modes allocate `1,048 B` in `2` allocations.
-- Loop-invariant container hoisting (issue #153) removes the per-access heap-cell derivation, itab guard, and slice-header reload from hoisted loop bodies. It shrinks Sieve's two loop callables about **40%** (`vm_jit_entry_bytes_total` 179,992 → 107,168 and 209,804 → 124,644) but leaves `Sieve(256)` wall time unchanged within run-to-run noise (interleaved A/B on M4 Pro: ~1-2%): the removed loads sat off the out-of-order critical path. The dominant remaining costs are native entry/exit round trips (the scan loop exits once per prime because loop-kind branch legs are never folded) and the per-iteration operand flush, both follow-up work.
+- Primitive array mutation stays on the native loop path in `Sieve(256)`: deferred-ownership elision drops the per-element retain/release pair, so a runtime-allocated array reaches the same cheap native path a typed-array constant already used. All three modes allocate `1,048 B` in `2` allocations.
+- Loop-invariant container hoisting (issue #153) removes the per-access heap-cell derivation, itab guard, and slice-header reload from hoisted loop bodies. It shrinks the loop callables but leaves wall time unchanged: the removed loads sat off the out-of-order critical path.
+- Branch-leg folding (issue #155) records native loop exits as branches and folds hot legs that rejoin the header back into the native loop as real back-edges. On `Sieve(256)` this removes the per-prime entry/exit round trips (scan-loop native entries drop from ~55 to ~1 per run) and cuts `default` from **4.72 us** to **1.61 us** and eager `jit` to **1.57 us**, versus **15.9 us** threaded (interleaved A/B on M4 Pro). The remaining gap to wazero (~0.66 us) is dominated by the per-iteration operand flush.
 - Threshold-zero `jit` is not a warmed-JIT guarantee. It matches `default` on Sieve and BranchTree, but is slower on IterativeFib, TypedArraySum, and recursive Fibonacci because it can compile before representative traces are learned.
 - Allocation-heavy workloads remain interpreter-bound. `AllocationGraph(128)` is fastest in minivm's threaded mode at **7.617 us**; adaptive and eager modes add profiling cost without native coverage.
 - Indirect recursion reaches the native self-call path in adaptive mode: `IndirectRecursiveFib(20)` is **54.94 us** in `default`, versus **568 us** threaded and **42.3 us** in wazero. Eager `jit` stays at **589 us**, consistent with the threshold-zero note above.
@@ -95,9 +96,9 @@ Each minivm kernel times `Interpreter.Run` only. Result extraction, reset, fixtu
 | IterativeFib(30) | Goja | 2,218 | 368 | 20 |
 | IterativeFib(30) | gpython | 2,590 | 2,448 | 88 |
 | IterativeFib(30) | Yaegi | 2,801 | 2,036 | 101 |
-| Sieve(256) | minivm/default | 4,722 | 1,048 | 2 |
-| Sieve(256) | minivm/threaded | 16,172 | 1,048 | 2 |
-| Sieve(256) | minivm/jit | 4,719 | 1,048 | 2 |
+| Sieve(256) | minivm/default | 1,610 | 1,048 | 2 |
+| Sieve(256) | minivm/threaded | 15,933 | 1,048 | 2 |
+| Sieve(256) | minivm/jit | 1,572 | 1,048 | 2 |
 | Sieve(256) | native Go | 238.3 | 0 | 0 |
 | Sieve(256) | wazero | 662.8 | 8 | 1 |
 | Sieve(256) | Tengo | 53,437 | 122,504 | 1,611 |

diff --git a/docs/jit-internals.md b/docs/jit-internals.md
@@ -144,7 +144,9 @@ Non-eager functions initially keep the ordinary generated `BR` handler. When per
 
 A pool shares one `Tracer`. Tree mutations are locked. `rootAt` returns a stable snapshot containing immutable trace pointers plus copied branch and hit containers.
 
-`tracePlan` converts that snapshot into flat plan blocks. It excludes aborted fragments and standalone loop roots from learned continuations, sorts continuation roots deterministically, connects internal paths by block ID, and derives spill policy from the final plan rather than exposing the trace tree to lowering.
+`tracePlan` converts that snapshot into flat plan blocks. It excludes aborted fragments and loop-kind legs (a loop-kind leg is a loop root of its own: anchored at this header it is the root itself and its edge already wires to the root block, anchored elsewhere it is a different loop with its own native entry), sorts continuation roots deterministically, connects internal paths by block ID, and derives spill policy from the final plan rather than exposing the trace tree to lowering.
+
+In a loop plan, a partial leg whose cut lands on the plan's own header (same function, depth 0) folds into the loop back-edge: `split` emits a real branch terminator instead of a fallback, `wire` resolves it onto the root block, and lowering takes the committing-flush native back-edge, so an in-loop branch that rejoins the header no longer exits native code (issue #155). A cut record that directly follows an explicit branch with the same target ends the split without materializing a spurious block, leaving the branch edge for `wire` to resolve. Cuts inside an inlined frame or to any other location keep the deopt fallback.
 
 ## Backend
 
@@ -154,7 +156,7 @@ A pool shares one `Tracer`. Tree mutations are locked. `rootAt` returns a stable
 
 Every plan block passes through one `emitBlock` path and every edge carries an explicit block ID or an unresolved threaded-fallback anchor. Bytecode locations describe source positions only; block IDs preserve distinct inlined contexts even when they share the same `(function, IP)`. A state-backed block reloads VM homes, while a profiled successor may continue with the current symbolic state.
 
-Caller continuations are ordinary blocks in the same flat block pool. A cold edge carries the continuation block IDs that must run after an inlined callee returns. Deferred edges always receive an independent label and symbolic snapshot; the backend does not merge states by bytecode anchor.
+Caller continuations are ordinary blocks in the same flat block pool. A cold edge carries the continuation block IDs that must run after an inlined callee returns. A deferred edge receives a label and a canonical symbolic snapshot (register-free values, reset locals); `label` shares the label of a previously scheduled continuation only when block, tail, and canonical snapshot are identical, and its ledger keeps consumed work items so folded legs that branch into one another (a loop nest) converge instead of exhausting the continuation limit. States are never merged by bytecode anchor alone.
 
 ## Trace ABI
 
@@ -297,13 +299,15 @@ Recorded forward branches become guarded exits or learned branch continuations.
 
 `BR_IF` and `BR_TABLE` emit the recorded path. Unrecorded targets deoptimize.
 
-When a side exit becomes hot, the tracer records that target. A later compile may fold it into the same native callable as a pending block. Loop roots are never folded as ordinary continuations: they deoptimize at the header and use their standalone loop entry, which preserves back-edge and safepoint semantics.
+When a side exit becomes hot, the tracer records that target. A later compile may fold it into the same native callable as a pending block. The loop wrapper records every fallback exit as a branch, so loop anchors recompile through the same side-exit machinery as entries. Loop roots are never folded as ordinary continuations: a leg that rejoins this plan's header folds into the native back-edge (see Trace Snapshots), while a leg that is another loop's root deoptimizes and uses that loop's standalone entry, which preserves back-edge and safepoint semantics.
+
+A loop callable normally exits through a trap, but a folded depth-0 `RETURN` leg emits `ret()` and a folded completed leg emits `complete()`, both returning with `trapNone`. The loop wrapper handles this like the entry wrappers: a function loop performs the threaded `RETURN` frame teardown, and a module loop marks the frame exhausted.
 
 Pending blocks reload from VM stack slots, run through a FIFO worklist, and stop at a bounded pending cap. The trace frontend orders learned roots once; the backend does not repeatedly sort pending work.
 
 A cold branch edge may carry caller-continuation block IDs. The side trace body lowers first; on callee `RETURN`, lowering stitches the result into the caller frame and follows those IDs. The continuation reloads from VM stack slots before continuing.
 
-Each deferred profiled edge gets its own label and symbolic snapshot. Static state-backed blocks share labels only through explicit block IDs, never through bytecode-anchor equality.
+A deferred profiled edge gets a label and canonical snapshot, shared only with an identical scheduled continuation (same block, tail, and snapshot). Static state-backed blocks share labels only through explicit block IDs, never through bytecode-anchor equality.
 
 Solo interpreters recompile a side exit when its hit count first reaches the hot-exit threshold. Pooled interpreters also rearm on later threshold multiples so a peer can recover a missed shared-cache publication.
 

diff --git a/interp/interp.go b/interp/interp.go
@@ -1065,15 +1065,7 @@ func (i *Interpreter) call(root anchor, callable asm.Callable, stats counters) f
 		}
 
 		if i.journal[journalTrap] == trapNone {
-			// Frame teardown the threaded RETURN handler does.
-			f := i.fr
-			i.sp = f.bp + f.returns
-			if f.release {
-				i.release(f.ref)
-			}
-			f.code = nil
-			i.fp--
-			i.fr = &i.frames[i.fp-1]
+			i.popFrame()
 			return
 		}
 
@@ -1115,8 +1107,7 @@ func (i *Interpreter) start(root anchor, callable asm.Callable, stats counters)
 
 		i.sp = int(i.journal[journalSP])
 		if i.journal[journalTrap] == trapNone {
-			i.fr.ip = len(i.code[i.fr.addr])
-			i.fr.code = i.code[i.fr.addr]
+			i.complete()
 			return
 		}
 
@@ -1138,10 +1129,11 @@ func (i *Interpreter) start(root anchor, callable asm.Callable, stats counters)
 
 // loop wraps a native loop Callable installed at a loop header. Unlike entry,
 // the header is reached mid-function with the frame already live, so loop never
-// pushes or tears down a frame and never returns normally — it always exits
-// through a trap. A spent budget yields to the safepoint and the Run loop
-// re-enters native at the header; a guarded side exit or the loop-exit edge
-// leaves deopt with i.fr at the resume IP for threaded dispatch to continue.
+// pushes a frame. It returns normally when a folded return or completion leg
+// finishes native execution; every other exit is a trap.
+// A spent budget yields to the safepoint and the Run loop re-enters native at
+// the header; a guarded side exit or the loop-exit edge leaves deopt with
+// i.fr at the resume IP for threaded dispatch to continue.
 func (i *Interpreter) loop(root anchor, callable asm.Callable, stats counters) func(*Interpreter) {
 	return func(i *Interpreter) {
 		stats.enter()
@@ -1155,6 +1147,14 @@ func (i *Interpreter) loop(root anchor, callable asm.Callable, stats counters) f
 			panic(err)
 		}
 		i.sp = int(i.journal[journalSP])
+		if i.journal[journalTrap] == trapNone {
+			if root.addr == 0 {
+				i.complete()
+			} else {
+				i.popFrame()
+			}
+			return
+		}
 		i.deopt()
 		switch i.journal[journalTrap] {
 		case trapOverflow:
@@ -1166,6 +1166,9 @@ func (i *Interpreter) loop(root anchor, callable asm.Callable, stats counters) f
 			}
 		case trapFallback:
 			stats.exit(i.journal[journalExitID])
+			// Record the exit as a branch so the tracer captures the leg and a
+			// hot in-loop branch recompiles the tree with the leg folded in.
+			i.exit(root)
 			// An exit that resumes at the header itself made no progress — the
 			// header slot holds this native stub, so dispatching it again would
 			// livelock (the hoist prologue's shape guard exits here). Run the
@@ -1179,6 +1182,22 @@ func (i *Interpreter) loop(root anchor, callable asm.Callable, stats counters) f
 	}
 }
 
+func (i *Interpreter) popFrame() {
+	f := i.fr
+	i.sp = f.bp + f.returns
+	if f.release {
+		i.release(f.ref)
+	}
+	f.code = nil
+	i.fp--
+	i.fr = &i.frames[i.fp-1]
+}
+
+func (i *Interpreter) complete() {
+	i.fr.ip = len(i.code[i.fr.addr])
+	i.fr.code = i.code[i.fr.addr]
+}
+
 func (i *Interpreter) exit(root anchor) {
 	hits := i.tracer.branch(i, root, anchor{addr: i.fr.addr, ip: i.fr.ip})
 	if i.cache != nil {

diff --git a/interp/interp_test.go b/interp/interp_test.go
@@ -1602,6 +1602,156 @@ func TestInterpreter_Run(t *testing.T) {
 		})
 	}
 
+	if runtime.GOARCH == "arm64" {
+		t.Run("ARM64 in-loop branch rejoins the header natively", func(t *testing.T) {
+			const size = int32(8)
+			b := program.NewBuilder()
+			b.Locals(types.TypeI32Array, types.TypeI32, types.TypeI32)
+			loop := b.Label()
+			odd := b.Label()
+			advance := b.Label()
+			done := b.Label()
+			b.ConstGet(types.TypedArray[int32]{0, 1, 2, 3, 4, 5, 6, 7}).Emit(instr.LOCAL_SET, 0)
+			b.Emit(instr.I32_CONST, 0).Emit(instr.LOCAL_SET, 1)
+			b.Emit(instr.I32_CONST, 0).Emit(instr.LOCAL_SET, 2)
+			b.Bind(loop)
+			b.Emit(instr.LOCAL_GET, 1).Emit(instr.I32_CONST, uint64(uint32(size))).Emit(instr.I32_GE_S).BrIf(done)
+			b.Emit(instr.LOCAL_GET, 0).Emit(instr.LOCAL_GET, 1).Emit(instr.ARRAY_GET)
+			b.Emit(instr.I32_CONST, 1).Emit(instr.I32_AND).BrIf(odd)
+			b.Br(advance)
+			b.Bind(odd)
+			b.Emit(instr.LOCAL_GET, 2).Emit(instr.I32_CONST, 1).Emit(instr.I32_ADD).Emit(instr.LOCAL_SET, 2)
+			b.Bind(advance)
+			b.Emit(instr.LOCAL_GET, 1).Emit(instr.I32_CONST, 1).Emit(instr.I32_ADD).Emit(instr.LOCAL_SET, 1)
+			b.Br(loop)
+			b.Bind(done)
+			b.Emit(instr.LOCAL_GET, 2)
+			prog, err := b.Build()
+			require.NoError(t, err)
+
+			profile := prof.New()
+			const runs = 32
+			func() {
+				jit := New(prog, WithTick(1), WithThreshold(0), WithProfiler(profile))
+				threaded := New(prog, WithTick(1), WithThreshold(-1))
+				defer func() {
+					require.NoError(t, threaded.Close())
+					require.NoError(t, jit.Close())
+				}()
+				for n := 0; n < runs; n++ {
+					require.NoError(t, jit.Run(context.Background()))
+					require.NoError(t, threaded.Run(context.Background()))
+					got, err := jit.PopBoxed()
+					require.NoError(t, err)
+					want, err := threaded.PopBoxed()
+					require.NoError(t, err)
+					require.Equal(t, want, got, "result diverged from threaded on iteration %d", n)
+					require.Equal(t, types.BoxI32(size/2), got)
+					require.Equal(t, threaded.rc[1:], jit.rc[1:], "refcount diverged from threaded on iteration %d", n)
+					jit.Reset()
+					threaded.Reset()
+				}
+			}()
+
+			entries := float64(0)
+			for _, metric := range profile.Metrics() {
+				if metric.Name != "vm_jit_native_entries_total" {
+					continue
+				}
+				labels := map[string]string{}
+				for _, label := range metric.Labels {
+					labels[label.Key] = label.Value
+				}
+				if labels["func"] == "0" && labels["kind"] == "loop" && labels["frontend"] == "trace" {
+					entries += metric.Value
+				}
+			}
+			require.Greater(t, entries, float64(0), "expected a scan-loop native entry metric")
+			require.Less(t, entries/runs, float64(8), "in-loop branch still exits the native loop")
+		})
+
+		t.Run("ARM64 folded return leg tears down the loop frame", func(t *testing.T) {
+			const size = int32(8)
+			body := types.NewFunctionBuilder(nil).Captures(types.TypeI32Array).Returns(types.TypeI32)
+			body.Locals(types.TypeI32, types.TypeI32)
+			done := body.Label()
+			body.Emit(instr.New(instr.I32_CONST, 0)).Emit(instr.New(instr.LOCAL_SET, 0))
+			body.Emit(instr.New(instr.I32_CONST, 0)).Emit(instr.New(instr.LOCAL_SET, 1))
+			loop := body.Label()
+			body.Bind(loop)
+			body.Emit(instr.New(instr.LOCAL_GET, 0)).Emit(instr.New(instr.I32_CONST, uint64(uint32(size)))).Emit(instr.New(instr.I32_GE_S)).BrIf(done)
+			body.Emit(instr.New(instr.LOCAL_GET, 1)).Emit(instr.New(instr.UPVAL_GET, 0)).Emit(instr.New(instr.LOCAL_GET, 0)).Emit(instr.New(instr.ARRAY_GET)).Emit(instr.New(instr.I32_ADD)).Emit(instr.New(instr.LOCAL_SET, 1))
+			body.Emit(instr.New(instr.LOCAL_GET, 0)).Emit(instr.New(instr.I32_CONST, 1)).Emit(instr.New(instr.I32_ADD)).Emit(instr.New(instr.LOCAL_SET, 0))
+			body.Br(loop)
+			body.Bind(done)
+			body.Emit(instr.New(instr.LOCAL_GET, 1)).Emit(instr.New(instr.RETURN))
+			fn, err := body.Build()
+			require.NoError(t, err)
+
+			b := program.NewBuilder()
+			b.Const(fn)
+			b.ConstGet(types.TypedArray[int32]{3, 3, 3, 3, 3, 3, 3, 3})
+			b.ConstGet(fn).Emit(instr.CLOSURE_NEW).Emit(instr.CALL)
+			prog, err := b.Build()
+			require.NoError(t, err)
+
+			jit := New(prog, WithTick(1), WithThreshold(0))
+			threaded := New(prog, WithTick(1), WithThreshold(-1))
+			t.Cleanup(func() { require.NoError(t, threaded.Close()) })
+			t.Cleanup(func() { require.NoError(t, jit.Close()) })
+			for n := 0; n < 32; n++ {
+				require.NoError(t, jit.Run(context.Background()))
+				require.NoError(t, threaded.Run(context.Background()))
+				got, err := jit.PopBoxed()
+				require.NoError(t, err)
+				want, err := threaded.PopBoxed()
+				require.NoError(t, err)
+				require.Equal(t, want, got, "result diverged from threaded on iteration %d", n)
+				require.Equal(t, types.BoxI32(3*size), got)
+				require.Equal(t, threaded.rc[1:], jit.rc[1:], "refcount diverged from threaded on iteration %d", n)
+				jit.Reset()
+				threaded.Reset()
+			}
+		})
+
+		t.Run("ARM64 folded completed leg finishes top-level code", func(t *testing.T) {
+			const size = int32(8)
+			b := program.NewBuilder()
+			b.Locals(types.TypeI32Array, types.TypeI32, types.TypeI32)
+			loop := b.Label()
+			done := b.Label()
+			b.ConstGet(types.TypedArray[int32]{1, 2, 3, 4, 5, 6, 7, 8}).Emit(instr.LOCAL_SET, 0)
+			b.Emit(instr.I32_CONST, 0).Emit(instr.LOCAL_SET, 1)
+			b.Emit(instr.I32_CONST, 0).Emit(instr.LOCAL_SET, 2)
+			b.Bind(loop)
+			b.Emit(instr.LOCAL_GET, 1).Emit(instr.I32_CONST, uint64(uint32(size))).Emit(instr.I32_GE_S).BrIf(done)
+			b.Emit(instr.LOCAL_GET, 2).Emit(instr.LOCAL_GET, 0).Emit(instr.LOCAL_GET, 1).Emit(instr.ARRAY_GET).Emit(instr.I32_ADD).Emit(instr.LOCAL_SET, 2)
+			b.Emit(instr.LOCAL_GET, 1).Emit(instr.I32_CONST, 1).Emit(instr.I32_ADD).Emit(instr.LOCAL_SET, 1)
+			b.Br(loop)
+			b.Bind(done)
+			b.Emit(instr.LOCAL_GET, 2)
+			prog, err := b.Build()
+			require.NoError(t, err)
+
+			jit := New(prog, WithTick(1), WithThreshold(0))
+			threaded := New(prog, WithTick(1), WithThreshold(-1))
+			t.Cleanup(func() { require.NoError(t, threaded.Close()) })
+			t.Cleanup(func() { require.NoError(t, jit.Close()) })
+			for n := 0; n < 32; n++ {
+				require.NoError(t, jit.Run(context.Background()))
+				require.NoError(t, threaded.Run(context.Background()))
+				got, err := jit.PopBoxed()
+				require.NoError(t, err)
+				want, err := threaded.PopBoxed()
+				require.NoError(t, err)
+				require.Equal(t, want, got, "result diverged from threaded on iteration %d", n)
+				require.Equal(t, types.BoxI32(36), got)
+				require.Equal(t, threaded.rc[1:], jit.rc[1:], "refcount diverged from threaded on iteration %d", n)
+				jit.Reset()
+				threaded.Reset()
+			}
+		})
+	}
 	modes := []struct {
 		name string
 		opts []func(*option)