Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/host-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,12 @@ converted the same way (it marshals as `ref`). Implement **both** interfaces for
a round-trip: a direction the type does not implement returns
`ErrUnsupportedMarshalType`.

Custom producer types can marshal to a heap value implementing
`types.Iterator`. Bytecode consumes that value with `RESUME`, `CORO_DONE`, and
`CORO_VALUE`, the same opcodes used for coroutine handles. An iterator yields
one `types.Value` at a time from `Current`; if it retains heap refs, implement
`types.Traceable` so the collector can see its backing state and current value.

### External types — `WithConverter`

For a type you do **not** own (so you cannot add `MarshalVM` / `UnmarshalVM`),
Expand Down
9 changes: 5 additions & 4 deletions docs/instruction-set.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,17 +293,18 @@ Map keys use primitive value identity for `i32`, `i64`, `f32`, and `f64`; all re
| `MAP_DELETE` | `{}` | `map key →` | ◐ | Delete entry; missing key is a no-op. JIT keeps framed entries by exiting locally to the threaded handler. |
| `MAP_CLEAR` | `{}` | `map →` | ◐ | Delete all entries. JIT keeps framed entries by exiting locally to the threaded handler. |
| `MAP_KEYS` | `{}` | `map → array` | ⬜ | Snapshot keys into a new `[]K` array (`K` = map key type), in unspecified order. Enables guest map iteration with `ARRAY_LEN`/`ARRAY_GET` + `MAP_GET`. |
| `MAP_ITER` | `{}` | `map → iterator` | ⬜ | Create a `types.MapIterator` over the map without snapshotting keys, advance it to the first key, and transfer map ownership into the iterator. The iterator yields keys only; use `MAP_GET` when the value is needed. Iteration order and mutation visibility are unspecified, matching Go map range semantics. |

## Coroutines

A function whose body contains `YIELD` is a coroutine-function: its `CALL` allocates a `Coroutine` handle and runs the body until the first `YIELD` (suspend) or `RETURN` (finish), yielding the handle instead of plain returns. `RESUME` re-enters a suspended handle. A `YIELD` in the entry frame (`fp == 1`) is an interpreter escape: it panics the private `errYield`, `Run` returns the exported `ErrYield` without losing state, and the next `Run` resumes after the `YIELD`.
A function whose body contains `YIELD` is a coroutine-function: its `CALL` allocates a `Coroutine` handle and runs the body until the first `YIELD` (suspend) or `RETURN` (finish), yielding the handle instead of plain returns. `RESUME` re-enters a suspended handle. A `YIELD` in the entry frame (`fp == 1`) is an interpreter escape: it panics the private `errYield`, `Run` returns the exported `ErrYield` without losing state, and the next `Run` resumes after the `YIELD`. `RESUME`, `CORO_DONE`, and `CORO_VALUE` also accept custom `types.Iterator` heap values; iterators are single-value producers whose `Current` value is read with `CORO_VALUE`.

| Opcode | Widths | Stack | JIT | Description |
|---|---|---|---|---|
| `YIELD` | `{}` | `value → result` | ◐ | Suspend the current coroutine frame, capturing `value` into the handle and unwinding to the caller; `RESUME` later delivers its in-value as `result`. In the entry frame it escapes via `ErrYield`. JIT records a `YIELD` reached in the trace's anchor frame as a terminal and lowers it to an unconditional deopt that runs the threaded suspend; a `YIELD` inside an inlined callee frame aborts the trace. |
| `RESUME` | `{}` | `coro in → coro` | ◐ | Resume a suspended handle, delivering `in` as the pending `YIELD`'s result, running to the next `YIELD`/`RETURN`, and pushing the handle back; traps `ErrCoroutineDone` if already finished. JIT lowers an anchor-frame `RESUME` to a terminal deopt that runs the threaded resume. |
| `CORO_DONE` | `{}` | `coro → i32` | ◐ | Push `I32(1)` if the handle has finished, else `I32(0)`; does not release the handle. JIT has a native fast path behind an itab guard (reads the `done` byte). |
| `CORO_VALUE` | `{}` | `coro → value` | ◐ | Push the handle's last yielded or returned value and release the handle. JIT has a native fast path behind an itab guard (loads the boxed `value`, retains it, releases the handle). |
| `RESUME` | `{}` | `coro in → coro` | ◐ | Resume a suspended coroutine handle, delivering `in` as the pending `YIELD`'s result, or advance a `types.Iterator` while ignoring `in`. Traps `ErrCoroutineDone` if already finished. JIT lowers an anchor-frame `RESUME` to a terminal deopt that runs the threaded resume. |
| `CORO_DONE` | `{}` | `coro → i32` | ◐ | Push `I32(1)` if the coroutine or iterator has finished, else `I32(0)`; does not release the handle. JIT has a native fast path for `*Coroutine` behind an itab guard and falls back for custom iterators. |
| `CORO_VALUE` | `{}` | `coro → value` | ◐ | Push the coroutine's last yielded/returned value or the iterator's current value, then release the handle. JIT has a native fast path for `*Coroutine` behind an itab guard and falls back for custom iterators. |

## Extensions

Expand Down
8 changes: 8 additions & 0 deletions docs/value-representation.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Heap objects implement `types.Value`.
| `*types.Struct` | `KindRef` | `*StructType` | `Value`, `Traceable` |
| `*types.Map` | `KindRef` | `*MapType` | `Value`, `Traceable` |
| `*types.MapI32`, `*types.MapI64`, `*types.MapF32`, `*types.MapF64` | `KindRef` | `*MapType` | `Value`, `Traceable` |
| `*types.MapIterator` | `KindRef` | `TypeRef` | `Value`, `Traceable`, `Iterator` |
| `*types.Function` | `KindRef` | `*FunctionType` | `Value` |
| `*types.Closure` | `KindRef` | `*FunctionType` | `Value`, `Traceable` |
| `*interp.HostFunction` | `KindRef` | `*FunctionType` | `Value` |
Expand All @@ -152,6 +153,13 @@ slice to `1 + len(Upvals)` and never takes the lazy-nil path.

`STRUCT_GET` and `STRUCT_SET` handle VM-native `*types.Struct` directly and fall back to `*interp.HostObject` for host-supplied values. See [host-integration.md](host-integration.md) for HostObject semantics.

`Iterator` marks heap values that can be consumed by `RESUME`, `CORO_DONE`, and
`CORO_VALUE`. `Next()` advances one item, `Current()` returns the single current
VM value, and `Done()` reports exhaustion. Iterators that hold refs must also
implement `Traceable` and report retained backing refs plus any current ref.
`MapIterator` keeps its source map and current ref key live while it traverses
without building a key array; it yields keys only.

User-defined heap types add no new `Kind`: a custom value implements `types.Value` / `types.Type` and is `KindRef` like every heap object (implement `Traceable` if it holds refs, `io.Closer` to release native resources). `Zero` returns `BoxedNull` for `KindRef`, so a custom type's default is null until an opcode constructs it. Reference the type programmatically via `b.Type(t)` and marshal Go values with `WithConverter` / `ValueMarshaler`; reconstructing a custom type from a textual type string is not yet supported.

Defined scalar values with methods marshal as their underlying primitive unless passed by pointer. Pointer form becomes a `HostObject`; field `0` is reserved as `Value` and exposes the current primitive for ordinary opcodes.
Expand Down
2 changes: 2 additions & 0 deletions instr/opcode.go
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,8 @@ const (
MAP_KEYS

CLOSURE_NEW

MAP_ITER
)

// EXT is the reserved prefix opcode for user-registered extension instructions.
Expand Down
1 change: 1 addition & 0 deletions instr/type.go
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ var types = map[Opcode]Type{
MAP_DELETE: {"map.delete", []int{}},
MAP_CLEAR: {"map.clear", []int{}},
MAP_KEYS: {"map.keys", []int{}},
MAP_ITER: {"map.iter", []int{}},

REF_NEW: {"ref.new", []int{}},
REF_GET: {"ref.get", []int{}},
Expand Down
1 change: 1 addition & 0 deletions instr/type_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ func TestTypeOf(t *testing.T) {
{opcode: MAP_DELETE},
{opcode: MAP_CLEAR},
{opcode: MAP_KEYS},
{opcode: MAP_ITER},

{opcode: REF_NEW},
{opcode: REF_GET},
Expand Down
135 changes: 135 additions & 0 deletions interp/interp_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,36 @@ func (fn callableFunc) Call(ctx uintptr) error {
return fn(ctx)
}

type testIterator struct {
values []types.Value
value types.Value
done bool
}

var _ types.Iterator = (*testIterator)(nil)

func (it *testIterator) Kind() types.Kind { return types.KindRef }

func (it *testIterator) Type() types.Type { return types.TypeRef }

func (it *testIterator) String() string { return "test.iterator" }

func (it *testIterator) Next() bool {
if len(it.values) == 0 {
it.value = types.BoxedNull
it.done = true
return false
}
it.value = it.values[0]
it.values = it.values[1:]
it.done = false
return true
}

func (it *testIterator) Current() types.Value { return it.value }

func (it *testIterator) Done() bool { return it.done }

// vmPoint opts into its own VM representation (a "x,y" string) even though it
// has exported fields and methods that would otherwise route to a HostObject.
type vmPoint struct {
Expand Down Expand Up @@ -2694,6 +2724,65 @@ var tests = []test{
),
values: []types.Value{types.I32(1)},
},
{
program: program.New(
[]instr.Instruction{
instr.New(instr.I32_CONST, 5),
instr.New(instr.I32_CONST, 42),
instr.New(instr.I32_CONST, 1),
instr.New(instr.MAP_NEW, 0),
instr.New(instr.MAP_ITER),
instr.New(instr.CORO_VALUE),
},
program.WithTypes(types.NewMapType(types.TypeI32, types.TypeI32)),
),
values: []types.Value{types.I32(5)},
},
{
program: program.New(
[]instr.Instruction{
instr.New(instr.I64_CONST, uint64(int64(1<<50))),
instr.New(instr.I32_CONST, 42),
instr.New(instr.I32_CONST, 1),
instr.New(instr.MAP_NEW, 0),
instr.New(instr.MAP_ITER),
instr.New(instr.CORO_VALUE),
},
program.WithTypes(types.NewMapType(types.TypeI64, types.TypeI32)),
),
values: []types.Value{types.I64(1 << 50)},
},
{
program: program.New(
[]instr.Instruction{
instr.New(instr.CONST_GET, 0),
instr.New(instr.I32_CONST, 42),
instr.New(instr.I32_CONST, 1),
instr.New(instr.MAP_NEW, 0),
instr.New(instr.MAP_ITER),
instr.New(instr.CORO_VALUE),
},
program.WithConstants(types.String("key")),
program.WithTypes(types.NewMapType(types.TypeString, types.TypeI32)),
),
values: []types.Value{types.String("key")},
},
{
program: program.New(
[]instr.Instruction{
instr.New(instr.I32_CONST, 5),
instr.New(instr.I32_CONST, 42),
instr.New(instr.I32_CONST, 1),
instr.New(instr.MAP_NEW, 0),
instr.New(instr.MAP_ITER),
instr.New(instr.I32_CONST, 0),
instr.New(instr.RESUME),
instr.New(instr.CORO_DONE),
},
program.WithTypes(types.NewMapType(types.TypeI32, types.TypeI32)),
),
values: []types.Value{types.I32(1)},
},
{
program: program.New(
[]instr.Instruction{
Expand Down Expand Up @@ -3587,6 +3676,52 @@ func TestInterpreter_Run(t *testing.T) {
}
})

t.Run("custom iterator resumes through coroutine opcodes", func(t *testing.T) {
for _, mode := range modes {
mode := mode
t.Run(mode.name, func(t *testing.T) {
it := &testIterator{values: []types.Value{types.I32(7), types.I32(11)}}
require.True(t, it.Next())
prog := program.New([]instr.Instruction{
instr.New(instr.CONST_GET, 0),
instr.New(instr.I32_CONST, 0),
instr.New(instr.RESUME),
instr.New(instr.CORO_VALUE),
}, program.WithConstants(it))

i := New(prog, mode.opts...)
defer i.Close()
require.NoError(t, i.Run(context.Background()))
got, err := i.Pop()
require.NoError(t, err)
require.Equal(t, types.I32(11), got)
})
}
})

t.Run("custom iterator reports done", func(t *testing.T) {
for _, mode := range modes {
mode := mode
t.Run(mode.name, func(t *testing.T) {
it := &testIterator{values: []types.Value{types.I32(7)}}
require.True(t, it.Next())
prog := program.New([]instr.Instruction{
instr.New(instr.CONST_GET, 0),
instr.New(instr.I32_CONST, 0),
instr.New(instr.RESUME),
instr.New(instr.CORO_DONE),
}, program.WithConstants(it))

i := New(prog, mode.opts...)
defer i.Close()
require.NoError(t, i.Run(context.Background()))
got, err := i.Pop()
require.NoError(t, err)
require.Equal(t, types.I32(1), got)
})
}
})

t.Run("call starts coroutine and first yield delivers a value", func(t *testing.T) {
// CALL of a coroutine-function (one containing YIELD) produces a Coroutine
// handle instead of return values; coro.value reads the first yielded value.
Expand Down
Loading
Loading