Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .codex/hooks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|MultiEdit|Write",
"hooks": [
{
"type": "command",
"command": "f=$(jq -r '.tool_input.file_path // .tool_input.path // empty'); if [ -f \"$f\" ] && [ \"${f##*.}\" = go ]; then if command -v goimports >/dev/null 2>&1; then goimports -w \"$f\"; elif [ -x \"$HOME/go/bin/goimports\" ]; then \"$HOME/go/bin/goimports\" -w \"$f\"; fi; fi",
"statusMessage": "goimports..."
}
]
}
]
}
}
218 changes: 141 additions & 77 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,125 +1,189 @@
# AGENTS.md

This file provides guidance to AI coding agents (Claude Code, Codex, Cursor, etc.) working in this repository.
This file guides AI coding agents working in this repository (`Claude Code`, `Codex`, `Cursor`, etc.).

---

## Quick Commands

```bash
make test # go test -race ./...
go test -race -run TestFoo ./interp/... # run a single test
make benchmark # go test -run="-" -bench=".*" -benchmem ./...
make lint # goimports -w . && go vet ./...
make coverage # produces coverage.out
make build # builds ./dist/minivm
./dist/minivm # launch the interactive assembly REPL
make test # go test -race ./...
make benchmark # go test -run="-" -bench=".*" -benchmem ./...
make lint # goimports -w . && go vet ./...
make coverage # generate coverage.out
make build # build ./dist/minivm

go test -race ./...
go test -race -run TestFoo ./interp/...

./dist/minivm # interactive assembly REPL
```

---

## Agent Workflow

1. Check `git status --short`; do not overwrite unrelated user changes.
2. Read only the docs listed for the package you will touch.
3. Locate existing tests near the edited package and mirror their style.
4. Update docs when behavior, invariants, commands, or recurring pitfalls change.
5. Run the narrow test first, then `go test ./...` or `make test` when risk is broader.

## Local Hooks

`.codex/hooks.json` runs `goimports` after `Edit`, `MultiEdit`, or `Write` touches a `.go` file, using either `goimports` on `PATH` or `$HOME/go/bin/goimports`. Still run `make lint` before finishing code changes because hooks are best-effort.

## Task Router

| Task | Start here | Usually edit | Verify |
| ---- | ---------- | ------------ | ------ |
| Opcode semantics | `docs/instruction-set.md`, `docs/guides/add-opcode.md` | `instr/`, `interp/threaded.go`, `interp/jit_arm64.go` | `go test ./instr ./interp` |
| Runtime/stack/frame bug | `docs/architecture.md`, `docs/memory-model.md` | `interp/`, `types/` | `go test ./interp ./types` |
| Ref/GC/host function | `docs/memory-model.md`, `docs/value-representation.md` | `interp/host.go`, `interp/threaded.go`, `types/` | `go test ./interp ./types` |
| JIT/ARM64 backend | `docs/jit-internals.md`, `docs/value-representation.md` | `interp/jit*.go`, `asm/`, `asm/arm64/` | `go test ./asm/... ./interp` |
| Optimizer/pass | `docs/pass-system.md` | `analysis/`, `transform/`, `optimize/`, `pass/` | `go test ./analysis ./transform ./optimize ./pass` |
| REPL/CLI | `docs/guides/repl.md` | `cmd/repl/`, `cmd/minivm/`, `instr/parse.go` | `go test ./cmd/repl ./cmd/minivm ./instr` |
| Style-only code change | `docs/coding-patterns.md` | touched package | package tests |

---

## Documentation Index

**Read docs on demand — do not pre-load all of them.** Each entry states exactly when it is relevant.

| Document | Read when… |
|---|---|
| [docs/architecture.md](../docs/architecture.md) | tracing execution flow, debugging across packages, understanding component boundaries |
| [docs/value-representation.md](../docs/value-representation.md) | working in `types/`, passing values through JIT, debugging boxing/unboxing |
| [docs/memory-model.md](../docs/memory-model.md) | writing threaded closures that touch refs, implementing host functions, modifying GC |
| [docs/instruction-set.md](../docs/instruction-set.md) | adding or modifying opcodes, checking JIT coverage, debugging specific instructions |
| [docs/jit-internals.md](../docs/jit-internals.md) | modifying `threaded.go` or `jit_arm64.go`, writing new opcode handlers |
| [docs/pass-system.md](../docs/pass-system.md) | adding optimization passes, modifying `transform/` or `analysis/` |
| [docs/coding-patterns.md](../docs/coding-patterns.md) | writing **any** new code in this repository |
| [docs/guides/add-opcode.md](../docs/guides/add-opcode.md) | adding a new instruction end-to-end |
| [docs/guides/add-architecture.md](../docs/guides/add-architecture.md) | adding a new JIT backend (e.g. x86-64) |
| [docs/guides/repl.md](../docs/guides/repl.md) | using or extending the interactive REPL (`cmd/minivm`, `cmd/repl`) |
Read only what is relevant to the task.

| Document | Read when… |
| --------------------------------------------------------------------- | ------------------------------------------------- |
| [docs/architecture.md](docs/architecture.md) | tracing execution flow, debugging across packages |
| [docs/value-representation.md](docs/value-representation.md) | modifying boxed values, JIT value passing |
| [docs/memory-model.md](docs/memory-model.md) | touching refs, closures, GC, host functions |
| [docs/instruction-set.md](docs/instruction-set.md) | adding or debugging opcodes |
| [docs/jit-internals.md](docs/jit-internals.md) | modifying threaded/JIT compilation |
| [docs/pass-system.md](docs/pass-system.md) | adding optimization or analysis passes |
| [docs/coding-patterns.md](docs/coding-patterns.md) | writing any new code |
| [docs/guides/add-opcode.md](docs/guides/add-opcode.md) | implementing a new instruction |
| [docs/guides/add-architecture.md](docs/guides/add-architecture.md) | adding a new JIT backend |
| [docs/guides/repl.md](docs/guides/repl.md) | extending or debugging the REPL |

---

## Architecture Overview

minivm is a bytecode VM with an adaptive JIT. Every function runs through a two-tier pipeline:
minivm is a bytecode VM with an adaptive JIT.

```
program.Program (bytecode []byte + constants)
```text
program.Program
▼ interp.New()
threadedCompiler → []func(*Interpreter) one closure per byte offset, always
threadedCompiler
[]func(*Interpreter)
▼ Interpreter.Run() — every 128 ticks, count hot-block hits
│ when hits reach threshold (default 4096 ticks):
jitCompiler → native ARM64 replaces closures in-place for hot segments
Interpreter.Run()
├─ executes threaded closures
└─ promotes sampled hot segments to JIT
native ARM64
```

### Package Responsibilities

| Package | Responsibility |
|---|---|
| `program/` | Bytecode + constants container; entry point for program construction |
| `instr/` | Opcode definitions, variable-width encoding/decoding, text formatter (`Format`), text parser (`Parse`/`ParseAll`) |
| `types/` | Value types (`Boxed`, `Function`, `Array`, `Struct`, `String`) and NaN boxing |
| `interp/` | Interpreter state, threaded compiler, JIT driver, host function bridge |
| `asm/` | Virtual-register IR, linear-scan register allocator, executable memory buffer |
| `asm/arm64/` | ARM64 encoder, ABI, caller trampoline |
| `pass/` | Reflection-based pipeline (`Manager`, `Pass[T]`) |
| `analysis/` | `BasicBlocksPass` (shared by JIT and optimizer) |
| `transform/` | `ConstantFoldingPass`, `ConstantDeduplicationPass`, `DeadCodeEliminationPass` |
| `optimize/` | `Optimizer` that wires passes into O0/O1 levels |
| `cmd/repl/` | Interactive assembly REPL (`REPL` type, `Run` loop) |
| `cmd/minivm/` | CLI entry point (cobra root command) |
Hot-segment compilation:

* profile samples record `(function, ip)` every 128 executed instructions
* JIT threshold defaults to 4096 ticks, i.e. 32 profile samples
* compiled native handlers replace threaded closures in-place

---

## Self-Maintenance: Keeping Docs Accurate
## Package Responsibilities

| Package | Responsibility |
| ------------- | ------------------------------------------------------------ |
| `program/` | bytecode + constants container |
| `instr/` | opcode definitions, encoding, parsing, formatting |
| `types/` | boxed values, arrays, structs, strings, NaN boxing |
| `interp/` | interpreter, threaded compiler, JIT driver |
| `asm/` | virtual-register IR, register allocation, executable buffers |
| `asm/arm64/` | ARM64 encoder, ABI, trampolines |
| `pass/` | generic pass pipeline |
| `analysis/` | shared analysis passes |
| `transform/` | optimization transforms |
| `optimize/` | optimization pipeline wiring |
| `cmd/repl/` | interactive REPL |
| `cmd/minivm/` | CLI entrypoint |

When a user reports a mistake or an agent discovers a gap during a task, **consider** updating the relevant documentation. This is a judgment call — not every mistake requires a doc change.
---

### When a doc update is worth it
## Key Invariants

- The same mistake is likely to happen again without a written reminder.
- A command, invariant, or package description in this file is factually wrong or outdated.
- A useful doc exists but is missing from the [Documentation Index](#documentation-index).
- A non-obvious constraint was discovered that no existing doc covers.
These rules cause silent corruption or invalid execution when violated.

### Which file to update
### Runtime

There is no requirement to edit `AGENTS.md` specifically. Update whichever file is the most natural home for the information:
* Heap index `0` is permanently `Null`.
* `release()` must remain iterative, never recursive.
* Threaded closure errors should `panic`; `interp.Run()` recovers and annotates `at=<ip>`.

- **Recurring gotcha or silent bug** → [Key Invariants](#key-invariants) in this file, or the relevant `docs/` page.
- **Wrong or missing command** → [Quick Commands](#quick-commands) in this file.
- **Architectural detail** → `docs/architecture.md` or the relevant doc listed in the index.
- **Coding pattern or style rule** → `docs/coding-patterns.md`.
- **New doc worth consulting conditionally** → add a row to the [Documentation Index](#documentation-index).
### Threaded Compiler

If no existing doc is a good fit, add a minimal note here. If the fix is minor and self-evident, skip the doc update entirely.
* Advance `c.ip` during compile time.
* Advance `f.ip` during runtime execution.
* Missing either causes invalid execution or infinite loops.

### Style rules when editing
### JIT

- Match the terse, imperative style of existing content.
- One entry per distinct fact — do not bundle unrelated changes.
- Document only what is concretely true in the current codebase; no speculative notes.
- After editing, verify the file is valid Markdown (tables aligned, code fences closed).
* JIT handlers must advance `c.ip` before every return path.
* On type mismatch, return `false, false`.
* Never coerce mismatched types.
* Branch terminators return `true, true`.
* Completed JIT segments emit when `count >= emit` (default 4); truncated segments emit only when `count > 4`.

---
### Executable Buffers

## Key Invariants
Always follow this order:

> ⚠️ These are non-obvious rules that cause **silent bugs** when violated. Read before touching any related code.
```text
Unseal → Append → Seal → Call
```

- **Heap index 0 is permanently `Null`** — allocated in `interp.New()`, never free it.
Incorrect ordering crashes on Apple Silicon.

- **Threaded closure: advance `c.ip` at compile time, advance `f.ip` at runtime** — missing either causes wrong execution or infinite loops.
### Optimization

- **JIT handler: call `c.ip++` first, then `return false, false` on type mismatch** — never coerce mismatched types; the second bool signals termination (`true, true` for branch terminators).
* `ConstantFoldingPass` right-aligns folded instructions and pads the left side with NOPs.
* Preserve folded byte ranges until `DeadCodeEliminationPass` compacts bytecode and rewrites branches.
* Threaded NOP handlers absorb consecutive gaps with one runtime dispatch.

---

## Coding Expectations

* Match existing package structure and naming.
* Prefer small, cohesive packages.
* Avoid unnecessary abstractions.
* Keep opcode handlers explicit and predictable.
* Preserve interpreter/JIT behavioral parity.
* Prefer table-driven tests where practical.
* Avoid hidden control flow.

---

- **`Buffer` ordering: `Unseal` → `Append` → `Seal` → `Call`** — wrong order triggers a signal on Apple Silicon.
## Documentation Maintenance

- **`ConstantFoldingPass` pads with NOPs left-aligned** — the threaded NOP handler absorbs the gap at compile time; do not strip this padding.
Update docs when:

- **JIT emits a segment only when `count > 4`** — exactly 4 consecutive JIT-able instructions is not enough; threshold is strictly greater-than.
* an invariant caused a bug
* a command became outdated
* architecture changed
* a recurring pitfall was discovered
* a new important doc should be indexed

- **`release()` is iterative, not recursive** — it uses an explicit stack to walk `Traceable.Refs()`.
Guidelines:

- **Errors in threaded closures should `panic`** — `interp.Run()` recovers and appends `at=<ip>` via `i.error(r)`.
* keep edits terse and factual
* document only current behavior
* avoid speculative notes
* preserve formatting consistency
* verify Markdown after edits
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ minivm gives your Go service a programmable runtime: assemble bytecode, expose G
go get github.com/siyul-park/minivm
```

> Requires Go 1.25+. Zero external dependencies.
> Requires Go 1.25+. The VM core uses only the Go standard library; the CLI and tests use small Go module dependencies.

---

Expand Down Expand Up @@ -136,20 +136,20 @@ minivm runs a **two-tier pipeline** that requires no decisions from you:
startup
bytecode ──────────────────► threaded closures
every 128 iterations:
count hits per basic block
every 128 instructions:
record a function/IP sample
hits reach threshold (default 4096 ticks)
samples reach threshold (default 4096 ticks)
jit compiler runs
emits native ARM64
replaces closures in-place
```

The JIT compiles numeric computation — arithmetic, bitwise ops, comparisons, and type conversions across i32/i64/f32/f64 — to native code. Control flow, function calls, and reference operations continue through the threaded tier. The threaded interpreter itself uses closure dispatch rather than a switch table, so it's fast even before JIT kicks in.
The JIT compiles numeric computation — arithmetic, bitwise ops, comparisons, and type conversions across i32/i64/f32/f64 — to native code. It also handles selected stack operations, locals, constants, `select`, and branch instructions when the current stack shape can be represented by the native segment signature. Function calls, globals, references, and heap-object operations continue through the threaded tier. The threaded interpreter itself uses closure dispatch rather than a switch table, so it's fast even before JIT kicks in.

**What this means in practice:** a compute-heavy loop will be slow for its first ~4096 iterations, then native-speed thereafter. There is nothing to tune unless you want to.
**What this means in practice:** a compute-heavy loop runs in the interpreter for roughly its first 4096 executed instructions, then hot native segments take over. There is nothing to tune unless you want to.

---

Expand Down Expand Up @@ -178,7 +178,9 @@ vm := interp.New(prog,
interp.WithStack(4096), // value stack slots (default: 1024)
interp.WithHeap(512), // initial heap capacity (default: 128)
interp.WithFrame(256), // max call depth (default: 128)
interp.WithThreshold(8192), // ticks before JIT (default: 4096)
interp.WithThreshold(4096), // ticks before JIT (default: 4096)
interp.WithTick(128), // profile sample period (default: 128)
interp.WithEmit(4), // min JIT segment ops (default: 4)
)
```

Expand All @@ -190,8 +192,8 @@ vm := interp.New(prog,
|---|---|
| Threaded interpreter | ✅ |
| AOT optimizer (O1) | ✅ |
| ARM64 JIT — numeric ops | ✅ |
| ARM64 JIT — control flow, calls | 🔲 planned |
| ARM64 JIT — numeric ops, locals, branches | ✅ |
| ARM64 JIT — calls, globals, refs | 🔲 planned |
| x86-64 JIT | 🔲 planned |

---
Expand Down
Loading
Loading