moon-pprof

A profiling toolkit for MoonBit that builds the same code across native / wasm-gc / wasm / js backends and normalises every run into the pprof format.

Japanese version: README-ja.md.

The fastest path is docs/quick-start.md — install the CLI, profile the bundled sample wasm, read the summary in under a minute. For the pprof / Chrome trace / Speedscope / folded stack conversion matrix, see docs/profile-formats.md.

If you just want the CLI (any wasm → profile / summary / cpuprofile2pprof / firefox2pprof):

# cargo from crates.io (requires rustc 1.80+ and `protoc` on PATH)
cargo install moon-pprof --locked

# nix (build-time deps live inside the flake)
nix run github:mizchi/moon-pprof -- --help
nix profile install github:mizchi/moon-pprof           # persistent install

If you want to track main instead of the published release, swap the cargo line for cargo install --git https://github.com/mizchi/moon-pprof moon-pprof --locked.

moon-pprof bench is the only subcommand that needs external moon / node / samply at runtime — see the Quickstart below, or nix develop to pull them all in.

The MoonBit-facing parts:

• A single moon-pprof CLI for profile / summary / bench (plus cpuprofile2pprof / chrometrace2pprof / pprof2chrometrace / pprof2folded / folded2pprof / pprof2speedscope / speedscope2pprof / firefox2pprof converters).
• Demangles MoonBit symbols and lines up all four backends in the same pprof schema.
• baseline ↔ patched comparison workflow for upstream PR experiments (patched-toolchain / patched-mooncakes / moon-pprof bench).

The internal libraries are MoonBit-agnostic. The Rust crates firefox-to-pprof / cpuprofile-to-pprof / chrome-trace-to-pprof / pprof-to-chrome-trace / pprof-stack-formats / wasmtime-guest-pprof work unchanged for AssemblyScript / Rust / Zig wasm too. → Details

Inside nix develop, moonbit-overlay brings in moon; the shell also has Node.js, Rust, wasmtime, samply, wabt, protobuf, and graphviz. (go is included only for go tool pprof visualisation — there's no Go code in this repo.)

nix develop
cargo build --workspace --release
mkdir -p .bin && cp \
  target/release/moon-pprof \
  target/release/http-baseline-server \
  runners/patched-toolchain \
  runners/patched-mooncakes .bin/
chmod +x .bin/patched-toolchain .bin/patched-mooncakes
npm install

Capture your first profile:

npm run build:wasm-gc && npm run profile:wasm-gc     # → wasm-gc.pb.gz
.bin/moon-pprof summary wasm-gc.pb.gz                # Top-N in the terminal
go tool pprof -http :8000 wasm-gc.pb.gz              # browser UI

--mem-pattern <regex> overrides the summary mem-mgmt classifier (default is MoonBit's). moon-pprof bench supports two orthogonal baseline/patched axes: --baseline-moon / --patched-moon (core toolchain swap) and --mooncakes-baseline / --mooncakes-patched (registry dep swap).

moonbitlang/core patches (rewrite the core under ~/.moon) — reproducing the bigint PR:

.bin/patched-toolchain init
.bin/patched-toolchain apply notes/pr-drafts/01-bigint-mul-single-limb/patch.diff
.bin/patched-toolchain rebundle
.bin/moon-pprof bench --workloads bigint_ops,bigint_square --runs 3

moonbitlang/x patches (rewrite a registry dep) — reproducing the uuid PR:

.bin/patched-mooncakes init bench-x
cp -r /tmp/moon-pprof-mooncakes/bench-x /tmp/moon-pprof-mooncakes/bench-x.patched
( cd /tmp/moon-pprof-mooncakes/bench-x.patched/moonbitlang/x \
  && patch -p1 < $(pwd)/notes/x-pr-drafts/04-uuid-tostring-inplace/patch.diff )

.bin/moon-pprof bench \
  --bench-dir bench-x \
  --backends native,wasm-gc,js \
  --workloads uuid_parse \
  --mooncakes-baseline /tmp/moon-pprof-mooncakes/bench-x \
  --mooncakes-patched /tmp/moon-pprof-mooncakes/bench-x.patched \
  --runs 3

→ The markdown table shows native -64% / wasm-gc -45% / js -39%.

Every path routes mangled names (e.g. _M0FP26mizchi5bench9ackermann) through the shared demangler to land as mizchi::bench::ackermann in the final pprof.

npm run build:wasm-gc && npm run profile:wasm-gc

moon build --no-strip --target=wasm-gc keeps function names in the wasm; moon-pprof profile --wasm-gc loads it with Config::wasm_gc(true)

• wasm_function_references(true) + wasm_reference_types(true) and samples via GuestProfiler on every epoch tick, then converts to gzip'd pprof. Host imports (spectest.print_char / wasi fd_write) come from the moonbit-wasm-host crate in one call.

If you want the wall-time that V8's inline cache hands you, the V8 path is still around:

npm run profile:wasm-gc:v8   # legacy Node V8 inspector path (for comparison)
# or:
.bin/moon-pprof bench --backends wasm-gc --wasm-gc-via-v8 ...

wasmtime (Cranelift baseline) and V8 (inline-cache-armed) produce different self-time distributions for the same wasm-gc binary. The hot-path topology agrees (which functions matter), the absolute numbers don't — treat them as separate signals.

Note: wasm-gc allocations go through wasm GC instructions (struct.new etc.), so the --mem-pattern mem-mgmt classifier doesn't see them. To track GC overhead you'd need instrumentation at the wasm-instruction level (out of scope for now).

npm run build:js && npm run profile:js

MoonBit's JS backend emits mangled symbols verbatim as JS function names (function _M0FP26mizchi5bench3fib(n) {...}). Node's inspector picks them up directly, so the same converter as wasm-gc applies.

npm run build:js && npm run profile:js:heap
go tool pprof -alloc_space -http :8000 js-heap.pb.gz

Drives Node's HeapProfiler.startSampling (V8 sampling allocation profiler), writes a .heapprofile, and moon-pprof heapprofile2pprof turns it into a pprof with two sample types: alloc_objects (count) and alloc_space (bytes). Demangling reuses moonbit-demangle, so mizchi::bench::mandel__sum appears as the top allocator just like in the CPU view.

V8's default sampling interval is 16 KiB. Pass node runners/v8/run-js-heap.mjs … --interval <bytes> to tighten or loosen it.

moon-pprof summary understands heap sample types for terminal top-N; use go tool pprof for the full GUI / source views.

# wasm (non-gc)
npm run build:wasm
.bin/moon-pprof memprofile bench/_build/wasm/release/build/cmd/main/main.wasm \
  --out wasm-mem.pb.gz --trace-out wasm-alloc.trace.json

# wasm-gc
npm run build:wasm-gc
.bin/moon-pprof memprofile bench/_build/wasm-gc/release/build/cmd/main/main.wasm \
  --out wasm-gc-mem.pb.gz

go tool pprof -alloc_space -http :8000 wasm-mem.pb.gz

Same subcommand handles both backends. It rewrites the wasm with walrus to add a host import moonbit_profile.alloc_hook(size: i32) and inject calls to it at every allocation point:

• wasm (non-gc): prepend local.get 0; call $hook to the body of $moonbit.malloc. $moonbit.gc.malloc internally calls $moonbit.malloc(n+8), so this one wrap covers raw and refcount-managed allocations.
• wasm-gc: rewrite every alloc opcode. Static-size cases (struct.new, struct.new_default, array.new_fixed) get a i32.const <size>; call $hook prefix. Dynamic-size cases (array.new, array.new_default, array.new_data, array.new_elem) save the length to a scratch local, push len * elem_size, call the hook, push the length back, then run the original opcode.

At run time the hook captures the current wasm call stack with wasmtime::WasmBacktrace::force_capture and accumulates (stack → (count, bytes)). The pprof's drop_frames field hides moonbit.malloc / moonbit.gc.malloc so user code is the visible leaf.

--trace-out <trace.json> also writes Chrome trace-event JSON with an allocation counter timeline (bytes / objects) plus per-sampled allocation instant events. This is allocation activity from the hook, not a true runtime GC pause trace.

The two backends agree on totals (e.g. ~24.9 MB on the bigint_square workload), but the shape differs:

• wasm non-gc attributes ~99% to the runtime helper moonbit.i32_array_make_raw (the real malloc site), with user functions visible via cum%.
• wasm-gc attributes directly to Add::add (9 MB) / Shl::shl (6 MB) / BigInt::split (4 MB) etc, because the alloc opcode lives at the user site, not in a runtime helper.

Caveat: wasm-gc sizes are a field-sum proxy, not the exact wasmtime GC heap consumption — alignment, object headers, and ref-slot sizing are engine-defined. The totals are an attribution signal, not a precise byte count.

The host hook calls WasmBacktrace::force_capture on every alloc by default, which dominates wall time on heavy workloads — a 50-iteration parse of a 197 KB JSON triggers ~13 M allocs and takes ~8 minutes per run. Pass --sample-rate <N> to capture a stack only every Nth alloc; the sample is then credited with N × size bytes so totals stay comparable.

# baseline + patched + diff in ~45 s (vs ~15 min without sampling)
moon-pprof memprofile baseline.wasm --sample-rate 100 --out base.pb.gz
moon-pprof memprofile patched.wasm  --sample-rate 100 --out patched.pb.gz
moon-pprof summary --diff base.pb.gz patched.pb.gz

Empirically, on JSON-parse-scale workloads (~13 M allocs) --sample-rate 100 matches the unsampled top-site bytes to within 0.1 % and the total within ~1 %. On smaller workloads (~100 k allocs, e.g. bigint) the total can drift by ~20 % at --sample-rate 100 because the law of large numbers hasn't kicked in — drop to --sample-rate 10 (still ~2× faster) or 1 (exact).

The sampler is deterministic 1/N, not Poisson: rerunning the same wasm at the same rate produces byte-identical pprof output, but a workload that allocates A,B,A,B,… correlated with the sampling stride will be biased. For exact answers, use --sample-rate 1 on a shorter iteration count.

npm run build:native
.bin/moon-pprof memprofile-native \
  bench/_build/native/release/build/cmd/json_parse/json_parse.exe \
  --out native-mem.pb.gz --sample-rate 100
go tool pprof -alloc_space -http :8000 native-mem.pb.gz

# retained records at exit (see coverage limitations below)
.bin/moon-pprof memprofile-native \
  bench/_build/native/release/build/cmd/json_parse/json_parse.exe \
  --retained --out native-retained.pb.gz --sample-rate 1
go tool pprof -inuse_space -http :8001 native-retained.pb.gz

MoonBit's --target native backend statically links mimalloc into every binary, and the user code's moonbit_malloc(size) macro inlines to libc_malloc(...) → mimalloc's local symbol. That bind is resolved at link time, so DYLD_INSERT_LIBRARIES / LD_PRELOAD of malloc never sees moonbit allocations (verified by counting: on a 197 KB JSON × 50-iter workload, libc-malloc interpose catches ~190 events, all from libsystem startup).

moon-pprof memprofile-native works around this by patching the generated C and recompiling:

1. From the .exe path, find the moon project root (containing moon.mod or moon.mod.json) and grab the cc command that built it via moon build --target native --release --dry-run.
2. Wrap moonbit_malloc_inlined in <cmd>.c to call __moon_pprof_alloc_hook(size) while preserving the original allocator body, including reference-count initialization, allocator selection, and GC safe points. With --retained, call __moon_pprof_alloc_ptr_hook(ptr, size) after allocation and wrap the existing moonbit_free macro (including header-defined macros) to observe frees without changing its behavior or evaluating arguments twice.
3. Compile a bundled native_alloc_hook.c (uses backtrace(3) + dladdr(3)) with the same cc flags.
4. Re-run the original cc command with the patched .c + hook .o, output to <cmd>.memprof.exe alongside the original.
5. Run <cmd>.memprof.exe. The hook writes a raw stream of {bytes, symbols} records to a tempfile.
6. moon-pprof aggregates and emits gzip'd pprof with sample types alloc_objects/count + alloc_space/bytes, or with --retained inuse_objects/count + inuse_space/bytes. drop_frames hides the hook, mimalloc, and runtime helpers so user code is the leaf.

Same --sample-rate <N> semantics as the wasm path. At rate 1 every allocation through the instrumented generated-C helper is recorded. This is not total process allocation: object headers, allocator overhead, and runtime-internal allocations (including some arrays/strings) are excluded. Retained profiles also miss frees performed inside the linked runtime, so remaining records can overcount live objects even at rate 1. Do not interpret them as proof of a leak; compare with RSS and object-lifetime evidence. Larger rates additionally sample and scale the captured bytes/counts. On the JSON parse workload, --sample-rate 1 takes ~41 s and --sample-rate 100 takes ~580 ms (~70× faster) with matching top-site attribution.

Works on macOS (Mach-O) and Linux (glibc ELF — the relink appends -rdynamic -ldl -lpthread so dladdr can see MoonBit's static symbols and the hook can lock its mutex). A Docker-based reproducer lives at notes/linux-memprofile/ — bash notes/linux-memprofile/run.sh builds an aarch64 Linux image with the MoonBit toolchain and runs an end-to-end check.

npm run build:wasm && npm run profile:wasm

The Rust API equivalent of wasmtime CLI's --profile=guest: run the wasm at Cranelift JIT speed, bump engine.increment_epoch() from a side thread, and let GuestProfiler::sample fire inside epoch_deadline_callback. The Firefox JSON output is then folded into pprof + gzip by the firefox-to-pprof crate. Host imports come from moonbit-wasm-host.

npm run build:native && npm run profile:native

samply records an OS-level sampling profile in Firefox Profiler format. --unstable-presymbolicate produces a .syms.json sidecar with per-symbol info; moon-pprof firefox2pprof --source samply --syms <sidecar> then converts to pprof (inline frames expanded). RVA → enclosing-symbol binary search is handled by firefox-to-pprof::samply::SamplySymsResolver.

The Rust crates and the npm package are independently usable from external projects.

All ten crates are on crates.io. Pick the ones you need:

[dependencies]
moonbit-demangle      = "0.1"
firefox-to-pprof      = "0.1"  # generic: samply / wasmtime JSON → pprof
cpuprofile-to-pprof   = "0.1"  # generic: V8 .cpuprofile → pprof
chrome-trace-to-pprof = "0.1"  # generic: Chrome trace-event V8 ProfileChunk → pprof
pprof-to-chrome-trace = "0.1"  # generic: pprof → synthetic Chrome trace-event V8 ProfileChunk
pprof-stack-formats   = "0.1"  # generic: pprof ↔ Speedscope, pprof ↔ folded stacks
heapprofile-to-pprof  = "0.1"  # generic: V8 .heapprofile → pprof
perf-to-pprof         = "0.1"  # generic: Linux `perf script` text → pprof
wasmtime-guest-pprof  = "0.1"  # generic: drop into a wasmtime app
moonbit-wasm-host     = "0.1"  # registers the moonbit wasm host imports in one call

use moonbit_demangle::demangle;
assert_eq!(demangle("_M0FP26mizchi5bench9ackermann"), "mizchi::bench::ackermann");

import {
  moonbitWasmImports,
  autoStubMissing,
} from "@mizchi/moonbit-wasm-host";

The pprof / firefox / cpuprofile / chrome-trace / demangle utilities have moved to Rust crates. From the CLI use moon-pprof cpuprofile2pprof / moon-pprof chrometrace2pprof / moon-pprof pprof2chrometrace / moon-pprof pprof2speedscope / moon-pprof speedscope2pprof / moon-pprof pprof2folded / moon-pprof folded2pprof / moon-pprof firefox2pprof. What stays on the npm side is just the host import (spectest.print_char / WASI fd_write) used to run a MoonBit wasm under Node V8.

The library layer is not MoonBit-specific. If you want to pprof-profile a Rust / AssemblyScript / Zig wasm:

Rust (run via wasmtime + Cranelift JIT):

use wasmtime::{Engine, Linker, Module, Store};
use wasmtime_guest_pprof::{ProfileSession, ProfilerHost, ProfilerHostExt as _, TakeProfileSession};

// Skip the MoonBit-only bits:
// - replace `moonbit-demangle` with your own demangler (or identity)
// - skip `moonbit-wasm-host`, register your app's own host imports
//
// `wasmtime-guest-pprof` + `firefox-to-pprof` work unchanged.

firefox-to-pprof::Builder::demangle_with() swaps the symbol decoder:

let bytes = firefox_to_pprof::Builder::new(&profile, frames, samples)
    .demangle_with(|s| my_demangle(s))   // any language is fine
    .encode()?;

Node / V8 .cpuprofile:

CLI:

moon-pprof cpuprofile2pprof --no-demangle in.cpuprofile out.pb.gz

As a library, the cpuprofile-to-pprof crate:

use cpuprofile_to_pprof::{Builder, CpuProfile};
let profile: CpuProfile = serde_json::from_slice(&bytes)?;
let out = Builder::new(profile)
    .demangle_with(|s| s.to_string())  // disable moonbit demangle
    .encode()?;
std::fs::write("out.pb.gz", out.encoded)?;

summary mem-mgmt classifier:

moon-pprof summary --mem-pattern <regex> (or $PPROF_SUMMARY_MEM_PATTERN) replaces the MoonBit-specific defaults (moonbit_drop_object etc.) with any runtime-primitive regex.

Cargo.toml                              ← Rust workspace
package.json                            ← npm workspace (workspaces: packages/*)

crates/                                 published libraries (Rust)
├── moonbit-demangle/                   mangled symbol → readable name
├── moonbit-wasm-host/                  moonbit wasm host imports (spectest / WASI)
├── firefox-to-pprof/                   Firefox Profiler JSON → pprof (generic)
├── cpuprofile-to-pprof/                V8 .cpuprofile → pprof (generic)
├── chrome-trace-to-pprof/              Chrome trace-event V8 ProfileChunk → pprof (generic)
├── pprof-to-chrome-trace/              pprof → synthetic Chrome trace-event V8 ProfileChunk (generic)
├── pprof-stack-formats/                pprof ↔ Speedscope, pprof ↔ folded stacks (generic)
├── heapprofile-to-pprof/               V8 .heapprofile → pprof (generic)
├── perf-to-pprof/                      Linux perf script text → pprof (generic)
└── wasmtime-guest-pprof/               wasmtime GuestProfiler driver + pprof (generic)

packages/                               published library (npm)
└── moonbit-wasm-host/                  @mizchi/moonbit-wasm-host (Node V8 host imports for moonbit wasm)

runners/                                CLIs / binaries
├── moon-pprof/                         Rust unified CLI
├── http-baseline-server/               Rust (axum + tokio), k6 baseline
├── patched-toolchain                   bash, ~/.moon snapshot / patch / rebundle
├── patched-mooncakes                   bash, .mooncakes/ snapshot / patch / restore
└── v8/                                 Node V8 inspector paths
    ├── run-wasm-gc.mjs                 wasm-gc under V8 (--via-v8)
    └── run-js.mjs                      js under V8
                                        (.cpuprofile → pprof: moon-pprof cpuprofile2pprof;
                                         Chrome trace JSON → pprof: moon-pprof chrometrace2pprof;
                                         pprof → Chrome trace JSON: moon-pprof pprof2chrometrace;
                                         pprof ↔ Speedscope / folded: moon-pprof pprof2speedscope / speedscope2pprof / pprof2folded / folded2pprof;
                                         samply / wasmtime guest JSON → pprof: moon-pprof firefox2pprof)

bench/                                  MoonBit bench workloads (ackermann / fib / mandel)
bench-async/                            moonbitlang/async investigation (coroutine / HTTP server)
bench-x/                                moonbitlang/x investigation (uuid / base64 / encoding / ...)
notes/                                  investigation logs + upstream-PR materials

bench/bench.mbt contains three CPU-bound workloads invoked from bench/cmd/main/main.mbt:

• ackermann(3, 10) — deep recursion
• fib(32) — classic recursion
• mandel_sum(160, 500) — nested loop + floats

Same code across every backend.

bench-async/ (moonbitlang/async investigation) and bench-x/ (moonbitlang/x investigation) carry their own workloads. See notes/async_investigation.md and notes/x_investigation.md for context.

notes/ holds the patch experiments derived from profiles, plus upstream-PR drafting material.

• Memory profiling: js, wasm, wasm-gc, and native (macOS + Linux). moon-pprof heapprofile2pprof converts a Node V8 sampling allocation profile (see Memory profiling (js)). moon-pprof memprofile instruments allocation sites in either wasm backend (see Memory profiling (wasm and wasm-gc)) — wasm wraps moonbit.malloc, wasm-gc rewrites every struct.new / array.new* opcode. wasm-gc sizes are a field-sum proxy. moon-pprof memprofile-native patches the generated <cmd>.c and relinks via the project's own cc command so the per-alloc hook fires inside moonbit_malloc_inlined — bypassing the statically-linked mimalloc that would otherwise hide allocations from LD_PRELOAD / DYLD_INSERT_LIBRARIES (see Memory profiling (native)).
• The demangler is heuristic. Impl / method / generic markers (_M0I…, _M0M…, GsE/GuE suffixes) are only partially decoded — for example, the core:: prefix on stdlib methods is dropped.
• The llvm backend (moon build --target=llvm) hasn't been verified — a MoonBit-side build error blocks it.
• Native profiling on Linux (samply-equivalent) depends on the environment. A perf → pprof converter (perf-to-pprof crate) is on the TODO list in notes/pprof_mbt_roadmap_v2.md.

Apache-2.0