Tags: Nimblesite/osprey
Tags
Promote the example corpus to tests/, add static effect discharge, fi… …x handler-arm debug scopes (#188) ## TLDR Promotes the example corpus into a first-class `tests/` tree where each Default program and its ML twin are proven to print identically against one shared golden, adds static algebraic-effect discharge to the type checker, and fixes six real defects — including handler-arm breakpoints that could never bind, every HTTP program failing to link on Windows, and a required CI job that could not pass on a clean runner. ## Details ### Static effect discharge `[EFFECTS-STATIC-DISCHARGE]` `crates/osprey-types/src/effect_rows.rs` (2,387 LOC, new) computes a least-fixed-point over per-function summaries of required operations, parameter uses and unresolved dynamic calls. Callable provenance is tracked as `Known`/`Unknown`/`Parameter`, and an invocation whose provenance cannot be proven **fails closed** rather than being assumed handled. `crates/osprey-lsp/src/effects.rs` (422 LOC, new) surfaces the same analysis in the editor. ### Test corpus restructure `examples/tested/**` becomes `tests/{core,effects,flavors,framework,interactions,regressions,workflows}/**` under a `.test.osp` / `.test.ospml` convention — 82 Default programs, 78 ML twins, 83 goldens. A Default program and its ML twin share **one** `.expectedoutput`, so the two surfaces are checked against each other rather than each against its own golden, which is what makes a divergence between them a test failure instead of two separately-passing tests. `crates/run_test_corpus.sh` (346 LOC, new) replaces `crates/diff_examples.sh` and `crates/diff_wasm_examples.sh` — one runner across every memory backend and the wasm32 target, instead of two scripts that had drifted. ### Other new functionality - **VS Code test explorer** — `test-explorer-docs.ts`, `test-docs-panel.ts`, `test-profile.ts` add documentation panels and profiling to the test tree. - **Hover fills inferred signature slots** `[LSP-HOVER-INFERRED-SIGNATURE]`, with a new spec section documenting both exceptions (written slots stay as written; unresolved slots are not shown). - `compiler/runtime/wasm_builtins_runtime.c` — builtins for the wasm32-wasip1 target. - `crates/osprey-ast/src/symbol.rs` (200 LOC) plus a shared `walk_each` collection walker in `visit.rs`, replacing three near-identical AST visitors (this is what brought measured duplication to the 5% ceiling in `.deslop.toml`). - `crates/osprey-cli/src/main.rs` drops from 2,596 to 1,552 LOC, moving command logic into its sibling modules. No new dependencies. ### Defects fixed **Breakpoints inside `handle` arms never bound.** A handler arm is emitted as its own LLVM function. `enter_nested_fn` clears the debug scope and nothing reopened one, so `location_id` returned `None` for every instruction lowered from the arm: no `!dbg` anywhere in the arm, and no `DISubprogram` for it. The arm's source lines were absent from the line table entirely — the arm ran, but a debugger had nothing to stop on. Both the direct and the `resume`-using arm paths now open a scope via `Codegen::begin_nested_debug`. The synthetic trampolines (suspend, drive, fn-value forwarders) deliberately keep no scope, so compiler-invented frames stay out of the debugger. **Hover leaked inference internals.** An inferred type still holding a type variable rendered as `fn classify(xs: List<t5>)`. `t5` is the checker's private placeholder — it moves when an unrelated line is edited and means nothing outside the inference run that produced it — so those slots are now left bare, as the author wrote them. **The coverage gate measured something the build never produced.** `osprey-cli`'s 95% floor depends on a wasm end-to-end test that self-skips unless `wasm-ld`, a WASI sysroot and `compiler/bin/libosprey_runtime_wasm.a` are all present — and nothing built that archive, in CI or locally. Measured here: **94.98%** (2782/2929) without it, **96.7%** (2831/2929) with it. A clean tree therefore failed on a coverage number that said nothing about the code under review. Both halves are now explicit: the `ci` job installs the toolchain and builds the archive, and `_test_rust` depends on `_runtime_wasm` so a local `make ci` builds what the gate measures. *Trade-off worth flagging:* a WASI toolchain is now a hard requirement for `make test` on a dev machine. The gate already required it; what changes is the failure mode. Previously you got `osprey-cli FAIL: 95.0% < 95%` and went looking for the test you broke. Now `_runtime_wasm` fails first with `ERROR: no WASI sysroot found. Install it with 'brew install lld wasi-libc'` — the same requirement, stated where it can be acted on. **`make test` could silently test a stale compiler.** The VS Code extension resolves its *bundled* `bin/<os>-<arch>/osprey` ahead of PATH, so a bundle left by an earlier build shadowed the binary `_test_vscode_extension` had just staged. CI never sees it — a fresh checkout has no `bin/` and falls through to PATH — which makes it precisely the class of failure a developer cannot reproduce from a green CI run. `_test_vscode_extension` now depends on `_vsix_bundle`. **Every HTTP or WebSocket program failed to link on Windows.** The runtime's `socket`/`connect`/`select`/`accept`/`closesocket` calls compile fine but resolve to `__imp_*` import stubs that only `ws2_32` supplies, so each of the 12 HTTP programs in the corpus died at *link* time with `undefined reference to __imp_socket` while every non-HTTP program linked clean. `WSAStartup` was already handled in `compiler/runtime/http_shared.c`; only the import library was missing. It stayed hidden because Windows CI ran two non-HTTP smoke programs until this branch started running the whole corpus there — the first Windows run on this branch was 148 passed / 12 failed, exactly the HTTP set. `link_args()` now appends `-lws2_32` when a program uses HTTP, ordered after the archive that references those symbols. **The GC corpus could not survive a runner.** `spawn` puts a second allocating thread on the heap, which latches the tracing collector off for the rest of the process by design ([MEM-FIBER-ISOLATION], `compiler/runtime/memory_gc.c`) — so everything allocated after that point is retained until exit. `cpu_profiling_demo.test.osp` drove `fib(36)`/`fib(37)` through that path: measured at over **6 GB resident and still climbing after 240s** under `--memory=gc`, against **3s** under the default backend. Two of those overlapping a 16 GB runner is why both required jobs died three times at the new GC step with exit 143 and *"The runner has received a shutdown signal"*, ~3 minutes in and with no output to say where. The demo now runs `fib(28)`/`fib(29)` — **430 MB and 1.16s** under GC — and keeps the shape it exists to demonstrate: three distinct fibers, `fib` self-time dominant, one napper blocked in `sleep`. No assertion changed; its checks were always on `fib(10)`/`tunedForTest(10)`. Also: `test_output.txt`, which the file-I/O regression test writes into the working directory, is no longer committed — it is now gitignored. ## How Do The Automated Tests Prove It Works? **Handler-arm debug scopes.** `a_breakpoint_inside_a_handler_arm_body_has_a_line_to_bind_to` (`crates/osprey-codegen/src/lib.rs`) compiles a `handle`d program with debug info and asserts the arm's `DISubprogram` is emitted and that `DILocation(line: 5,` — the arm's own `let` — reaches the line table. On the pre-fix compiler it fails with *"the arm function needs its own subprogram to be a debuggable scope"*. A second assertion covers the `resume`-using arm through its separate emitter; each fix was individually reverted to confirm its own assertion fails without it, so neither assertion passes vacuously. Against `tests/effects/errors/direct_recovery.test.ospml` — the file the bug was reported on — line 275 went from **0** debug locations to **57**. **Inferred hover signatures.** `an_inferred_signature_fills_resolved_slots_and_leaves_generic_ones_bare` (`crates/osprey-lsp/src/hover.rs`) asserts `fn area(r: int)` is filled from inference while `fn classify(xs) -> int` stays bare, pinning both directions: the feature still fills resolved slots, and a type variable never reaches a tooltip. **Effect discharge.** `effect_rows_tests.rs` (864 LOC) and `effect_rows_expr_tests.rs` (471 LOC) cover the row- and provenance-carrying expression forms. Four of them build `Expr::Pipe`, `Expr::MethodCall` and `Expr::Update` nodes by hand, because both surface lowerers desugar `x |> f` and `t.m(..)` into plain calls — those checker arms had never executed under any test (lcov hit-count 0) despite being live fail-open risks. **Windows linking.** The Windows job now builds and runs the whole corpus rather than two smoke programs, so the 12 HTTP programs are link-checked on every run — the failure this fixes would reappear as 12 red suites, not silence. Verified green on this branch. **GC memory.** The GC corpus step *is* the regression guard: reintroducing an unbounded fiber workload takes the runner down again, which is exactly what it did three times here. Peak resident measured per suite across the full `--memory=gc` corpus is now **198 MB** for the worst one, against >6 GB before. **Profiler end-to-end.** `_test_vscode_extension` sits after `_conformance-gc` in `make test`, so its new profiling tests had never actually run in CI. On their first real run the sampler collected nothing on Linux: the fixture spun 2,000,000 iterations, which at the 997 Hz sample rate is ~20 ms of sampled CPU — 19 samples on a fast dev machine, few enough that the profiler's own report appends *"run longer for confidence"*, and zero on a busier runner. Measured yield is 2M → 19, 20M → 210, 60M → 623; the fixture now spins 60,000,000 for a ~30× margin at about half a second of CPU. The assertion did not move — `applied[0] > 0`, *"the profile really collected samples"*, is as strict as it was. **The corpus itself.** `make ci` runs 160 suites under each memory backend and again on wasm32, comparing stdout byte-for-byte against the shared goldens, with per-crate coverage floors from `coverage-thresholds.json` enforced on top.
Website + docs, ML/LSP intelligence, memory-backend hardening, GitHub… … Action spec (#181) Docs website refresh, ML-flavor & LSP intelligence expansion, memory-backend conformance hardening, and the Osprey GitHub Action spec (0033) + plan (0015). CI fully green.
v0.13.0 Documentation website launch, ML-flavor parser/lowering expansion, LSP language-intelligence overhaul (context-aware completion/hover/ML rendering), GC/ARC memory-backend conformance gates with a zero-leak bar, and codegen/ C-runtime work. Default and ML flavors emit byte-identical IR.
Swappable memory backends (Perceus ARC + tracing GC), cross-language … …benchmark suite, and runtime hardening (#171) ## TLDR Adds a swappable memory-management layer with a **Perceus ARC** reference-counting backend and a conservative **tracing GC** backend alongside the default leak-everything allocator, lands a 22-case cross-language benchmark harness that measures all three, and ships a VS Code profiler/test-explorer plus the Talon Bank modules showcase — with the C runtime tightened under a stronger portable lint set and new unit suites. ## Details **Swappable memory backends** (`--memory=default|gc|arc`, spec 0018 [MEM-BACKENDS]) - Codegen names only `osp_alloc*` / `osp_retain` / `osp_release` / `osp_collect`, so the manager is chosen at link time (three archive variants per runtime). The emitted IR is backend-agnostic. - **Perceus ARC** (`compiler/runtime/memory_arc.c`): a 16-byte header `{meta, rc, size}`, a size-classed recycling free-list, and a layout word (kind + pointer bitmask) driving the drop walk. A new `OSP_MEM_MASK_DIRECT` kind lets codegen prove a struct's managed fields are ARC bodies or NULL, so the drop path skips the registry probe (was ~52% of binarytrees CPU); nullary variants (e.g. `Leaf`) are interned as immortal singletons; constructors allocate through `osp_alloc_tagged_noinit` to skip the drop-safety pre-zero. Live-object leak accounting is now gated behind `OSPREY_ARC_DEBUG` so a normal run skips it entirely. - **Tracing GC** (`compiler/runtime/memory_gc.c`): conservative non-moving mark & sweep with an adaptive collection budget; the alloc prologue was de-duplicated and read-only diagnostics (`osp_gc_live_objects` / `osp_gc_collections`) added. - The recycling free-list is factored into a shared header `compiler/runtime/memory_pool.h`. - Codegen (`osprey-codegen`): `MetaField::PtrDirect`, `proven_heap_name`, extern-return poisoning, and interned nullary singletons emit the new layout metadata. **Cross-language benchmark suite** (`benchmarks/`) - 22 cases (ackermann … wordfreq), each implemented in Osprey plus C, Rust, C#, Dart, OCaml, Haskell. `run.sh` builds every impl, times with hyperfine, records peak RSS, and `report.py` renders `results.{html,json}` + bakes the website `/benchmarks` tables. - Measured standing of **osprey-arc**: beats C# on peak memory on **22/22** cases; beats C# on CPU on **13/22**; at or below the field median on **14/22**. On `binarytrees` (pure allocation churn) it runs 216 ms at **2.9 MB RSS** vs C#'s 122 ms at 16.4 MB — above the field median and faster than hand-written C and Rust, though C#/Dart's generational bump-alloc still win that case on CPU. **VS Code extension**: profiler panel (flame graph, heat-line decorations, summary) and a native test explorer, with harnessed unit + e2e suites. **Talon Bank modules showcase** (`examples/projects/modules`): a multi-module Osprey app serving an HTTP JSON API + web UI, exercised by native `osprey test` suites and a Playwright e2e journey. **C runtime hardening** - A shared `WARN` / `WARN_MAX` lint core added to `-Werror` across every C recipe: `-Wshadow -Wpointer-arith -Wvla -Wundef -Wredundant-decls -Wcast-qual -Wcast-align -Wold-style-definition -Wbad-function-cast` (+ `-Wstrict-prototypes -Wwrite-strings` on the shipped archives). The set is the gcc∩clang portable intersection (a clang-only spelling was deliberately excluded so `make _runtime` stays green under the gcc-based CI/Docker builder). Five pre-existing offenders fixed (const-correct `http_method_to_string`, function-cast hoists in `memory_gc`/`profiler_runtime`, a K&R prototype). - New suites: `memory_gc_tests.c` (reachability, reclamation, interior-pointer marking, shim allocators, multithread-disable latch) and `memory_pool_tests.c` (cap rounding, LIFO reuse, size-class isolation, retention ceiling). ## How Do The Automated Tests Prove It Works? - **Differential golden harness** (`crates/diff_examples.sh`): `PASS=148 FAIL=0 NOEXP=0`, `FC_OK`, byte-identical stdout under the default backend **and** `--memory=gc` **and** `--memory=arc` — the ARC run reports `ARC_LEAKY=0` (zero language values leaked, the [GC-ARC-PERCEUS] gate). - **C runtime unit tests** (`make _test_c_runtime`), all under the hardened flags: `memory_arc` 600 975 assertions, **`memory_gc` 40 259 (new)**, **`memory_pool` 8 222 (new)**, `list_runtime` 5 146 191, `map` 405 020. `memory_gc_tests` runs with `OSPREY_GC_DEBUG` showing real collections fire (then freeze after the multithread latch). - **Rust workspace**: `cargo test --workspace` — 313 tests pass; `cargo clippy --workspace --all-targets -- -D warnings` clean (pedantic); `cargo-llvm-cov` reports all crates meet their `coverage-thresholds.json` gates; `cargo fmt --all --check` clean. - **Bank**: `make bank-test` (3 native `osprey test` suites) and `make bank-e2e` (17 Playwright journeys) pass, driving the compiled runtime's HTTP server end-to-end. - **Profiler**: `make _test_profiler` → `PROFILER-E2E-OK`. - **Web-compiler Docker** (the gcc CI compiler): container build + `test.sh` compile-and-run of `osprey_mega_showcase.osp` matches expected output byte-for-byte. - **Benchmarks**: `make bench` produces the committed `benchmarks/results/*` and the baked website tables from the numbers above. - **deslop** duplication gate (`make lint`) holds after extracting the shared pool.
Generics & variance, algebraic-effect handlers, and documentation com… …ments (#167) ## Summary Large feature branch landing three language subsystems plus tooling/CI hardening. All CI checks were run locally and pass (see below). - **Generics & variance** — `crates/osprey-types/src/variance.rs`, `crates/osprey-ast/src/generics.rs`, `crates/osprey-codegen/src/effect_generics.rs`; spec `docs/specs`/plan `0015-generics-and-variance.md`. New must-reject cases in `examples/failscompilation/variance_*` and `generic_effect_*`. - **Algebraic effects & handlers** — `crates/osprey-ast/src/resume.rs` and handler/resume codegen; plan `0016-algebraic-effects-and-handlers.md`. Reject cases `multishot_resume_rejected`, `resume_in_arm_lambda`. - **Documentation comments** — flavor-neutral doc model `crates/osprey-ast/src/doc.rs` + parser `crates/osprey-syntax/src/docparse.rs`; spec `0026-DocumentationComments.md`, plan `0018-documentation-comments.md`. - **New tested examples** — `fiber/fiber_determinism`, `fiber/fiber_exact_replica`, `basics/json/json_document_query` (`.osp`/`.ospml` twins), registered in the example manifest. - **Duplication gate** — extracted a shared `osprey_ast::walk_each` collection-walker (backing the freevars/effects/analysis visitors), a `store_tag` helper in `aggregate.rs`, `nested`/`map_of_kind` lowering helpers; `.deslop.toml` gate held at 6.10% with genuine dedup (measured 6.06%). ## Test plan (run locally, in CI order) - `cargo fmt --all --check`, `cargo clippy --workspace --all-targets -- -D warnings` — clean - `cargo build --release --workspace`, `cargo test --workspace` — 0 failures - Differential golden harness — `PASS=142 FAIL=0 NOEXP=0`, `FC_OK` - `npm run test:shipwright` — manifest valid - `make lint` — deslop `6.10% (ok)`; `make test` — all crates over coverage thresholds, extension 95.13%; `make build` - Example smoke (comprehensive_math, feature_omnibus) — pass - WASM: `_runtime_wasm` + compile/validate/smoke + `diff_wasm_examples.sh` — `PASS=47 FAIL=0 SKIP=26` - Website Playwright E2E — `76 passed` - Web-compiler `test.sh` — pass (run against a locally-started server; the Docker daemon was unavailable on the prep host, so the container-packaging layer will be exercised by CI) > Note: `ci-windows.yml` is Windows-only and was not runnable on the macOS prep host; the workspace build/clippy/test it depends on all pass here.
ML flavor front end, formatter, effects, modules, and docs/website ov… …erhaul (#165) <!-- agent-pmo:74cf183 --> ## TLDR Adds a second **ML (offside/curry-by-default) surface** to Osprey — parser, lexer, lowering, formatter, LSP, and VSCode support — proven byte-for-byte equivalent to the Default flavor, alongside a full spec/docs/website overhaul and a browser WebAssembly SQL demo. ## Details **ML flavor frontend (`crates/osprey-syntax/src/ml/`)** — a complete indentation-delimited, curry-by-default front end that lowers to the same AST as the Default flavor: - `lexer.rs` / `token.rs` — offside-rule layout lexer (INDENT/DEDENT, newline-terminated strings). - `parser.rs` / `cst.rs` — ML grammar: whitespace application (`f x`), `\x => e` lambdas, `:=` mutation, `=` binding, layout blocks — no `fn`/`let`/braces. - `lower.rs` (1730 LOC) — lowers the ML CST to the shared AST. - `crates/osprey-syntax/src/default/` — the existing brace-flavor parser moved under `default/` so the two surfaces sit side by side; `strings.rs` factored out as shared interpolation lexing. - `.osp` → Default, `.ospml` → ML, selected by extension; a flavor marker conflict is a hard error. **Formatter (`crates/osprey-fmt/` + `osprey fmt`)** — new crate with brace (`brace.rs`) and layout (`layout.rs`) formatters plus a shared scanner; wired into the CLI as `osprey fmt` (stdin/dir-recursive/check/rewrite modes) and into `make fmt`. **Algebraic effects / first-class handlers** — `compiler/runtime/effects_runtime.c` (+250) and `osprey-codegen/src/effects.rs` (+515) extend effect resumption and make handlers first-class values. **Modules & namespaces** — spec `0025` plus supporting type-checker and lowering changes. **LSP / VSCode** — `osprey-lsp` gains flavor-aware diagnostics/features; new `osprey-ml.tmLanguage.json`, ML snippets, and `language-configuration-ml.json`. **Website & docs** — every `website/src/spec/*` chapter revised; new chapters `0023 Language Flavors`, `0024 ML Flavor Syntax`, `0025 Modules and Namespaces`; mirrored under `docs/specs/`. `/docs/` index gains an inline **Flavors** explainer (one language, two ways to write it). New **`/wasm/`** page: an Osprey program compiled to `wasm32-wasip1` seeds a real in-browser SQLite (sql.js) database with a tabbed query/add-data/source UI, and both the Osprey and SQL are Prism-highlighted (`js/wasm-studio.js`). Playground gains a Default⇄ML flavor toggle. **Examples** — most `examples/tested/*` programs gain a byte-identical `.ospml` twin; several `.osp` files were de-annotated to lean on inference. > Cleanup note: this branch also carries `scratchpad/` working files and `*/.claude/settings.local.json` that should probably be dropped before merge. ## How Do The Automated Tests Prove It Works? `make ci` (lint + test + build) passes end-to-end, exit 0. Specific coverage: - **`cross_flavor_ir_equiv.rs`** — `every_ml_example_has_a_default_twin` and `ml_and_default_twins_emit_identical_ir` compile each `.ospml`/`.osp` pair and assert the emitted LLVM IR is identical, proving the ML surface lowers to the same program. - **`cross_flavor_equiv.rs`** — `ml_multiparam_equals_default_explicit_curry` / `ml_uncurried_tuple_equals_default_multiparam` pin ML currying semantics against the Default flavor. - **`corpus.rs`** — `default_examples_format_idempotently` and `ml_examples_format_idempotently` prove `osprey fmt` is a fixpoint over the whole example corpus (format twice == format once). - **`cli_e2e.rs`** — `fmt_*`, `flavor_marker_conflict_exits_two`, `check_*`, and `llvm_emits_ir_and_rejects_ill_typed` exercise the `fmt`/`check`/`llvm` CLI surface end-to-end including error exit codes. - **`no_needless_main.rs`** — `no_example_wraps_a_trivial_program_in_main` enforces the style rule across examples. - **`ml_coverage.rs`** (823 LOC) — parser/lowering unit coverage for the ML front end. - **Differential harness** (`diff_examples.sh` / `diff_wasm_examples.sh`) — every `.osp`/`.ospml` in `examples/tested/` runs and must match its `.expectedoutput` byte-for-byte. - **Website E2E (`website/tests/interactions.spec.js`)** — `wasm demo seeds a browser SQLite db and runs queries` boots the wasm module, asserts `#banner` reaches "Database ready", the query tab auto-renders a result table with highlighted SQL tokens, the source tab shows highlighted Osprey, and adding a row then re-querying reflects the new data; `playground flavor toggle swaps the sample between .osp and .ospml` verifies the Default⇄ML toggle. - **VSCode extension suite** — 78 passing, coverage 90.94% (≥89% threshold).
feat: wasm32 browser target — wasm example tour, wasm-serve, benchmar… …k refresh (#154) ## Summary Browser-targeted WebAssembly support, plus the developer ergonomics and example coverage to go with it. - **wasm example as a language tour** — `examples/wasm/hello.osp` expanded from a hello-world into a full wasm-safe walkthrough of the language (pattern matching, functions, string interpolation, etc.), with `.expectedoutput` and the prebuilt `hello.wasm` regenerated to match. - **Make targets** — VSIX targets scoped to `osprey` only, and a new `wasm-serve` dev server target for running the wasm example in the browser. - **Benchmarks** — restored the full 22-case run (including the GC backend) and refreshed the published results/tables. ## Test plan - [ ] `make ci` (lint + test + build) - [ ] Differential example harness passes (`examples/wasm/hello.osp` matches `.expectedoutput` byte-for-byte) - [ ] `make wasm-serve` runs the example in a browser
chore(benchmarks): restore full 22-case run (incl. GC backend) Rescued from the pre-merge auto-stash: a complete benchmark run across all 22 cases and the osprey-gc backend that was uncommitted in the working tree (the committed snapshot held only the listops case).
feat: wasm32 target, source-level debugger, CSPRNG builtins, GC backe… …nd + non-integer benchmarks (#153) ## TLDR Adds a WebAssembly (`wasm32-wasip1`) compile/run target, a source-level native debugger (DWARF + lldb-dap) with a new `osprey-debug` crate, three CSPRNG/stdin builtins (`random`/`randomBelow`/`input`), a tracing GC backend, and four non-integer benchmark cases, while consolidating the example corpus to a top-level `examples/` directory. ## Details ### WebAssembly target - New `osprey --target=wasm32` path producing a `wasm32-wasip1` command module, driven by a dedicated `crates/osprey-cli/src/wasm.rs` backend: it injects a `__main_void` entry thunk into the IR, lowers to a wasm object via `clang --target=wasm32-wasip1`, and links with `wasm-ld` against a WASI sysroot's `crt1-command.o`, the portable `libosprey_runtime_wasm.a` archive, and `-lc`. WASI sysroot/lib-dir and toolchain commands are discoverable/overridable (`OSPREY_WASI_SYSROOT`, `OSPREY_WASM_CC/LD/RUN`); `--run` executes under wasmtime. - New CLI flags `--target=native|wasm32`, `--memory=default|gc`, `-o <path>`, and `--debug`, with iterator-based arg parsing, `parse_target`/`parse_memory` validators (reserved `arc` explicitly rejected), and an `output_path` resolver. `--debug` is rejected for wasm. - ILP32 width-stability fixes [WASM-TARGET-WIDTH / BUILTIN-STRING-LENGTH]: string length/concat now call the runtime's i64-returning `osp_strlen` instead of libc `strlen`; int formatting uses `%lld`; and `coerce_return` repacks an existing Result to the declared success-slot inner type via new `result::repack_to_inner` so disc/errmsg offsets agree across targets. - New WebAssembly Target spec (`docs/specs/0022-WebAssemblyTarget.md` + website mirror), a runnable `examples/wasm/` demo (hello.osp, a shared WASI preview1 browser shim, bundler-free `index.html`), and Node/browser smoke scripts (`scripts/wasm-smoke.mjs`, `scripts/wasm-browser-smoke.mjs`). ### Source-level debugger - New `crates/osprey-debug` crate of language-agnostic primitives: `DebugSource` (path split into filename/directory with NO-PLACEHOLDER fallbacks, reconstructed via `path()`) and `DebugBuild` (OFF/ON, `opt_flag` defaulting to `-O0` for debug, `native_driver_flags` `-g`/`-fno-omit-frame-pointer`). It is depended on by both `osprey-codegen` and `osprey-cli`. - LLVM/DWARF debug-info emission in codegen: new `compile_program_debug(program, DebugSource)` entry point and `CodegenOptions { debug_source }` switch. When enabled, codegen emits `source_filename`, `!llvm.dbg.cu`/`module.flags`/`ident`, a `DICompileUnit`/`DIFile`, `DIBasicType`s, per-function `DISubprogram`, `DILocalVariable`s, `DILocation`s, and `@llvm.dbg.value` intrinsics, tagging instructions with `, !dbg !<id>`; Dwarf version is host-dependent (4 on macOS, 5 elsewhere). Source `Position` is threaded through statement/function lowering, and `SavedFn` snapshots/restores debug scope so nested functions keep correct scopes. - AST change: `Stmt::Expr(Expr)` becomes the struct variant `Stmt::Expr { value, position }`, updated across codegen, syntax lowering, types, and the LSP (`walk_stmt_body` now matches `Stmt::Expr { value, .. }`). - VS Code extension: the no-op debug adapter is replaced with a real `DebugAdapterExecutable` backed by lldb-dap. `resolveDebugConfiguration` saves the document, compiles a native debug binary via `osprey <src> --debug --compile -o <out>`, and launches it; new lldb-dap resolution helpers (launch config → `osprey.debug.lldbDapPath` setting → PATH → `xcrun -f lldb-dap` → common LLVM paths), an `osprey.debug` command rebound to F5, an Osprey Debug activity-bar TreeView (`client/src/debug-panel.ts`), a `breakpoints` contribution, and an expanded launch schema. - New debugger specs/plans: `docs/specs/0021-Debugger.md` and `docs/plans/0012-osprey-debugger.md` (749 lines); LSP/editor spec `0020` gains a Debugger Integration section with Mermaid diagrams. ### CSPRNG builtins + input - New runtime unit `compiler/runtime/random_runtime.c` implementing `random` (`osp_random`, non-negative int63), `randomBelow(n)` (`osp_random_below`, unbiased rejection sampling, `-1`/Error on `n<=0`), and `input()` (`osp_input`, one stdin line, `""` at EOF). Entropy via `arc4random_buf` or `getrandom(2)` with `/dev/urandom` fallback. - Wired end-to-end: `extern_call.rs` adds a `Ret::Str` return kind plus the three signatures, `osprey-types/builtins.rs` registers their types, and builtin docs/website function pages (`random.md`, `randombelow.md`) are added. `input()` is redocumented as returning `string` (was `int`). ### Tracing GC backend - New conservative, non-moving mark & sweep GC (`compiler/runtime/memory_gc.c`) over the C stack, callee-saved registers, and data/BSS segments, exposing the swappable `@osp_alloc` ABI selected by `--memory=gc` (retain/release become no-ops). It adds managed value-container allocators (`osp_gc_malloc/calloc/realloc/free`) and a force-included `osp_gc_shim.h` that redirects malloc/free in list/map units only, leaving the default backend byte-identical. The CLI's `link_args` swaps in `_gc`-suffixed runtime archives at link time; IR is unchanged [MEM-BACKENDS]. Documented in spec `0018` and plan `0011-arc-gc-implementation.md`. ### Benchmarks (non-integer + randomized input) - Four new data-structure cases each shipping five language ports (.osp/.c/.hs/.ml/.rs) over a shared Park-Miller MINSTD generator: `wordfreq` (200k tokens into a HAMT Map), `textstats` (per-token string metrics), `listops` (persistent List, a second memory benchmark), and `exprtree` (recursive Expr union eval). - `run.sh` adds an `osprey-gc` column and three wasm columns (osprey/rust/c, run via wasmtime), toolchain probes, wasm-incompatible skips vs hard native failures, and constant (MODE0) vs random-seed (MODE1, `BENCH_RANDOM=1`) stdin input modes; `report.py` extends ORDER/LABEL and switches Osprey checks to `lang.startswith('osprey')`. Regenerated results show the GC reclamation (e.g. listops 2.6 GB → 3.78 MB). ### Examples reorganization, build & CI - The entire `compiler/examples/` tree (~258 files) is consolidated to a top-level `examples/` directory; all path references are updated across the CLI tests, codegen/types doc-comments, CI workflows, Makefile, docs, and website. - Makefile gains a public `wasm` target, `_runtime_wasm` (cross-compiles a portable C-runtime subset to `libosprey_runtime_wasm.a`), GC archive builds (`libfiber_runtime_gc.a`/`libhttp_runtime_gc.a` via the shim), and a `_conformance-gc` target asserting byte-identical GC output. The `osprey-debug` crate is added to the workspace with a 95% coverage threshold. - CI gains a `wasm` job (installs clang/LLVM/lld/wabt + wasi-sdk v24, builds native+wasm runtimes, compiles `examples/wasm/hello.osp`, runs `wasm-validate`, Node WASI and browser-shim smoke runs, then the wasm golden suite gated on FAIL=0/NOEXP=0). ## How Do The Automated Tests Prove It Works? - `cargo test --workspace`: 315 tests pass. New codegen unit test `debug_compile_emits_source_level_metadata` asserts the emitted IR contains `source_filename`, `!llvm.dbg.cu`/`module.flags`, `DICompileUnit`/`DIFile`, the host-correct Dwarf Version, `DISubprogram` for `add` and `main`, `DILocalVariable`, `@llvm.dbg.value`, a `DILocation`, and `, !dbg !` instruction tags. New `wasm.rs` tests cover the entry-thunk injection, clang/wasm-ld argv shape, sysroot/lib-dir resolution, env-stubbed full-driver build/run, and a toolchain-gated end-to-end build asserting the `\0asm` magic header; `main.rs` tests cover target/output parsing, `output_path` defaults, debug-wasm rejection, and gc archive selection. `osprey-debug` has three tests covering `DebugSource` path splitting/fallbacks and `DebugBuild` flag selection. - Differential golden harness `crates/diff_examples.sh`: PASS=48 FAIL=0 NOEXP=0 with the failscompilation must-reject ratchet holding (FC_OK), on both the default and GC backends (GC output is byte-identical, verified by `make _conformance-gc`). - WebAssembly golden harness `crates/diff_wasm_examples.sh` + `make wasm`: PASS=30 FAIL=0 SKIP=18 (skips are non-portable features); `wasm-validate`, Node WASI, and browser WASI smoke runs all pass. - VS Code extension suite: 76 passing, including the new "Osprey VSIX Debugger E2E" and "Osprey Debugger E2E Workflows" suites running the real lldb-dap adapter (conditional breakpoints, watch evaluation, step in/out/over, ordered breakpoints, scope/local inspection, stopOnEntry); extension coverage 90.84% (>= 89%) and the shipwright manifest is valid. - `cargo fmt --check` and `clippy --workspace --all-targets -D warnings` are clean. Per-crate Rust coverage all meets thresholds: osprey-cli 95.5%, osprey-codegen 95.6%, osprey-debug 100%, osprey-lsp 99.1%, osprey-types 98.7%, osprey-ast 97.1%, osprey-syntax 95.6%, osprey-runtime-sys 100%. - Website Playwright E2E: 68 passing; Docker web-compiler test passes (output matches oracle). `make lint`, `make test`, and `make build` all exit 0.
PreviousNext