Eclipse: In-process wasm TeX engine - #152

The sparse/selective checkout fought the web2c build system (it expects
the whole tree, e.g. doc/Makefile.in). Switch to a plain shallow full
clone — simpler and matches what the build actually needs.

Phase 0 milestone: pristine TeX Live 2026.1 pdftex source cross-compiles
to WebAssembly via Emscripten and runs (pdfTeX 3.141592653, kpathsea
6.4.2) — no third-party build harness, no engine-source patches.

Captures the recipe: emconfigure --build/--host, disable luajit (its
32-bit native buildvm can't cross-build on arm64), trim CONF_SUBDIRS to
pdftex's libs, native otangle for the WEB->C bootstrap.

This is the node-targeted wasm; the standalone-wasm + wasm2c step for
texpresso integration is next (Phase 1).

Rework the pdftex wasm build to standalone WASI with no JS runtime or JS
compat. SUPPORT_LONGJMP=wasm moves setjmp/longjmp into wasm exception
handling, eliminating the JS invoke_* trampolines. Result: 22 imports
(WASI + native syscalls), zero invoke_/_js.

Flags applied at configure time (not make-override, which would leak
emcc -s flags into native build-tool compiles). EMCC_SKIP_SANITY_CHECK
trims per-invocation overhead.

wasm2c consumes the output with --enable-exceptions (default rejects the
wasm EH tag section).

Phase 1 core: pristine pdftex runs as a native arm64 binary via wasm2c
with no node/JS. src/engine-wasm/wasm_host.c implements the 22 wasm
imports (WASI fd/args/environ/clock/proc_exit + env.__syscall_* getcwd/
faccessat/readlinkat/etc.) against native POSIX; shell escape and
networking stubbed. scripts/build-wasm2c-pdftex.sh runs wasm2c
(--enable-exceptions) and links the generated module + wabt wasm-rt +
host into pdftex-native.

Verified: ./pdftex-native --version -> pdfTeX 3.141592653, kpathsea 6.4.2.

Switch the wasm build to non-standalone so filesystem syscalls
(openat/stat/lstat/newfstatat/fcntl/ioctl) are emitted as imports the
host implements — the VFS hook. SUPPORT_LONGJMP=wasm keeps setjmp in
wasm EH, so no JS invoke_* trampolines despite non-standalone.

wasm_host.c now implements the full 33-import surface: WASI fd/environ,
the file syscalls (with Linux->host O_* flag and AT_FDCWD translation and
emscripten struct-stat marshalling), time/abort/exit/resize_heap, and the
__main_argc_argv entry (argv built in wasm memory via emscripten_stack_alloc).

Verified end-to-end, JS-free: pdftex-native --ini reads a .tex and writes
a valid doc.dvi (1 page) with all file I/O routed through the host.

Known follow-ups: create-mode bit translation; kpathsea search path / the
texmf.cnf EACCES-vs-ENOENT warning on absolute probes.

Add the wasm-opt --asyncify pass to the build so the wasm call stack can
be spilled into linear memory at suspend points — the prerequisite for
capturing full execution state (stack included) with a linear-memory COW
snapshot, since wasm2c keeps the wasm stack on the native C stack.

Verified the asyncified engine still typesets normally (asyncify is
dormant until the host drives start_unwind/rewind). Exposes the
asyncify_{start,stop}_{unwind,rewind} + get_state controls.

Run the wasm2c engine on a dedicated fixed-address stack via ucontext,
and yield back to the host from inside fd_read (swapcontext). Proven:
the engine suspends mid-read and resumes to complete a typeset. This
replaces asyncify entirely — no wasm instrumentation, no size bloat, no
asyncify ABI, no malloc-less buffer problem. Drop the asyncify build pass.

Next: COW-snapshot the linear memory + engine stack at the suspend point
for rollback (mprotect dirty-tracking).

…ven)

Snapshot the full engine execution state without fork or asyncify:
linear memory + the coroutine stack + register context + open-fd
positions. TEXPRESSO_SNAPSHOT_TEST suspends at the first read, runs a
typeset to completion (run A), rolls back, and replays from the snapshot
(run B). Both runs emit byte-identical output -> SNAPSHOT ROLLBACK PASS.

Also fix openat: emscripten passes a *pointer* to the variadic args, so
the create mode must be read from wasm memory (it was used as the value,
creating output files 0040/read-only, which broke the write reopen).

Full-copy for now; the memory is mmap fixed-base so mprotect dirty-
tracking drops in later. Update WASM-ENGINE.md: asyncify is out, the
coroutine stack is the snapshot mechanism.

Replace the full-copy heap snapshot with mprotect dirty-tracking: snapshot
marks the linear memory read-only (O(1)); a SIGSEGV/SIGBUS handler saves
each page's original bytes on first write and re-marks it writable; rollback
copies back only the dirtied pages. A full typeset dirties 18/1084 pages
(~1.7%), and rollback still replays byte-identically (SNAPSHOT ROLLBACK PASS).

The engine stack + register context + fd positions stay full-copy (small).
Base is fixed (WASM_RT_USE_MMAP) so the protected region is stable.

…shim

xetex.wasm (24 MB) now cross-compiles under emscripten with freetype2,
harfbuzz, graphite2, teckit and icu, non-standalone for wasm2c, no asyncify.

xetex requires fontconfig off-Mac and there is no wasm fontconfig, so link a
minimal FreeType-backed shim (src/engine-wasm/fontconfig-shim): it implements
only the enumeration subset xetex uses (FcFontList + metadata accessors, no
matching/cache/fonts.conf), driven by a font-path manifest. API/ABI-compatible
with real fontconfig.

scripts/build-wasm-xetex.sh handles the emscripten cross-build quirks:
- harfbuzz in-source -Werror pragmas -> -DHB_NO_PRAGMA_GCC_DIAGNOSTIC_ERROR
- ICU native data-tool build (emmake leaks CC=emcc) -> force CC=cc for icu-native
- ICU has no emscripten platform config -> map mh-unknown to mh-linux
- freetype's own build drops our flags -> pass CFLAGS into its configure
- trim CONF_SUBDIRS + MAKE_SUBDIRS to xetex's libs (no lua53/luajit)
- build the xetex target specifically (building "all" pulls lua53)

… XDV

Generate both engines with `wasm2c -n engine` so a single wasm_host.c drives
pdftex and xetex (import symbols are keyed on env/wasi, not the engine). Add
build-wasm2c-xetex.sh; rename the pdftex wasm2c output to engine.c/.h.

Host additions for xetex:
- file-backed _mmap_js via the engine's emscripten_builtin_memalign + pread
  (ICU/freetype map files; ICU has no fallback), _munmap_js no-op
- weak fallback for emscripten_builtin_memalign so pdftex (no mmap) still links
- TEXPRESSO_TRACE gate for openat/mmap logging

xetex.wasm linked -sALLOW_MEMORY_GROWTH=1 (xmallocs ~72 MB at startup).

Result: xetex-native runs a full pass and writes XDV through the host I/O;
pdftex snapshot/rollback still PASS. Remaining blocker: ICU common data is
statically linked but not reached at runtime (ucnv_open -> U_FILE_ACCESS_ERROR);
next step is loadable-archive ICU data via ICU_DATA. See WASM-ENGINE.md.

Two fixes make xetex actually render glyphs:

1. ICU data: the statically-linked data entry point is never consulted at
   runtime under wasm2c (ucnv_open -> U_FILE_ACCESS_ERROR). Build ICU data as a
   loadable archive (--with-data-packaging=archive -> icudt78l.dat) and load it
   at runtime from ICU_DATA via the host's file mmap. xetex.wasm drops 24 MB ->
   3.3 MB (no embedded data). TeX Live's icu kludge wants a libicudata.a in
   lib/; in archive mode fall back to copying the stubdata there.

2. environ passthrough: environ_get/environ_sizes_get returned an empty
   environment, so the engine's getenv("ICU_DATA") was NULL. Pass the host
   environment through.

Result: `\font\x="[/abs/font.ttf]"` renders glyphs into the XDV (full pipeline:
engine + ICU + freetype + harfbuzz + fontconfig shim + host file I/O + mmap).
Run with ICU_DATA=<dir with icudt78l.dat>. pdftex snapshot/rollback still PASS.

Third engine on the pipeline. luatex compiles to wasm (4.35 MB) with PUC Lua
(lua53); needs neither ICU nor fontconfig. Libs: zlib libpng freetype2 graphite2
harfbuzz pplib zziplib lua53. kpathsea is a top-level build subdir (top make,
not make -C texk).

Same unified wasm_host.c drives it (wasm2c -n engine). Added the 28 imports
luatex needs beyond xetex: real chdir/dup/dup3/fstat64/linkat/mkdirat/symlinkat/
utimensat/fd_sync, mktime_js, emscripten_get_heap_max; luasocket networking and
DNS stubbed (typesetting never needs a socket); sighandler/keepalive no-ops.

Proven: luatex-native executes \directlua (Lua 5.3), kpse.var_value resolves
paths, --version works. Follow-up: bare `--ini \shipout` aborts in kpathsea
(output open returns NULL with no format/backend loaded); needs a format or the
texpresso driver. pdftex + xetex still pass (snapshot/rollback, render).

Time sources (clock_time_get, emscripten_date_now/get_now) now return a fixed
epoch (from $SOURCE_DATE_EPOCH, else read once at startup) instead of the live
host clock. Constant for the process, so any mid-document time read (luatex
os.time, \pdfcreationdate, ...) replays identically after a COW rollback.
Monotonic clock -> 0. gmtime/localtime/mktime stay pure (deterministic given
input). pdftex snapshot/rollback still PASS.

Add wasm_host.h: a POSIX-shaped wasm_io_ops vtable (openat/read/write/lseek/
close/fstat/statat/accessat) + wasm_host_set_io(). The host's core file imports
now call g_io->* instead of native POSIX directly. Default backend is a native
passthrough (used by the standalone ENGINE-native binaries); texpresso will
install a state.c-backed impl that serves file content from fileentry buffers
(no native FS, so snapshots capture all I/O state).

This is the seam the in-process wasm engine backend hangs off. No behavior
change: all three engines still pass (pdftex snapshot rollback, xetex renders,
luatex runs Lua). Peripheral mutation ops (unlink/rename/mkdir/...) stay native
for now; texpresso's impl can extend the vtable as needed.

…mfs)

Add an in-memory io backend (TEXPRESSO_MEMFS): opened files are served from
memory — inputs lazily read from disk once then served from buffers, writes
captured to memory and never written to disk (fds 0/1/2 pass through so engine
messages still show). Proven: xetex renders entirely from memory and the
captured XDV is byte-identical to a native-FS run, with no output file touching
disk. This is exactly what the texpresso state.c backend will do.

Also fix _mmap_js to pread via the io_ops backend (lseek+read) instead of native
pread, so file mmap (ICU data, fonts) works when fds are VFS handles rather than
native fds. pdftex snapshot + luatex Lua still pass.

…ocess use

Extract the engine lifecycle into a reusable API so texpresso can drive the
wasm engine in-process (its own main would otherwise clash with the host's):
  wasm_engine_init/run/exited/request_yield/snapshot/restore/defer_close/shutdown
The module instance + env/wasi become file-scope statics (single in-process
engine). snapshot_take/restore use g_mod and route fd-position save/restore
through g_io (native for standalone; texpresso's log owns it for state.c).

The standalone main + the memfs test backend are now guarded by
WASM_HOST_NO_MAIN, so texpresso links the host as a library (verified: compiles
with no main). Standalone unchanged: pdftex snapshot PASS, luatex Lua, xetex
render, memfs proof all still work.

New src/frontend/engine_wasm.c implements the txp_engine vtable driving a stock
TeX engine compiled to wasm2c *in-process* (no fork, no IPC). File syscalls are
answered from texpresso's VFS (state.c fileentry buffers) via the wasm_io_ops
seam: openat/read/write over entry_data()/saved.data, with writes fed to
incdvi/synctex/editor exactly like engine_tex.c's answer_query. Reuses
incdvi/synctex/state and the render/getter methods verbatim.

First-cut replay model: any edit -> reset VFS to the restart mark + re-instantiate
the module + replay the whole document (wasm_engine_init is now re-callable).
Correct but not incremental; the coroutine+COW incremental rollback (already
proven in the host) is Phase 3.

Compiles clean against the frontend/mupdf/host headers. Not yet linked into the
texpresso binary (Makefile + main.c selection next). Standalone engines
unaffected: snapshot rollback still PASS.

src/frontend/Makefile: when WASM_ENGINE_DIR is set, add engine_wasm.o and link
the chosen wasm2c engine (engine.o + wabt runtime) plus a library-mode
wasm_host_txp.o (-DWASM_HOST_NO_MAIN; compiled with plain cc so the frontend's
-I. doesn't shadow the wasm2c engine.h). -DHAVE_WASM_ENGINE threads through.
main.c: with HAVE_WASM_ENGINE, TEXPRESSO_WASM=1 selects txp_create_wasm_engine
instead of the fork engine. Normal build (no WASM_ENGINE_DIR) unchanged.

Verified: texpresso links clean with the wasm xetex engine (no undef/dup syms).
Build with: make -C src/frontend WASM_ENGINE_DIR=../../engines/build-wasm2c-xetex

Bring-up fixes to the engine command line built by engine_wasm.c:
- argv[0] = a real existing path (the engine_path) so the engine's kpathsea can
  lstat its SELFAUTOLOC (a fake name failed: "Can't get directory of program").
- -ini: build from primitives, no preloaded .fmt (else it hunts for
  texpresso-xetex.fmt and aborts).
- -no-pdf: emit XDV for incdvi, not PDF (the PDF path failed to open output).
- prefix the main file with "./" so kpathsea opens it directly (bare relative
  names aren't found without a texmf.cnf "." path).

Result: `texpresso -test-initialize` with TEXPRESSO_WASM=1 runs stock xetex
in-process via wasm2c, reads the .tex from state.c's VFS, captures the XDV, and
incdvi reports 1 page -- the whole engine->render pipeline works with no fork,
no subprocess, no patched engine.

(-no-pdf/-ini are xetex/bring-up specific; real LaTeX docs need a format +
texmf, a later step. Engine cmdline will be parameterized per engine.)

texmf integration for the in-process wasm engine:
- scripts/build-wasm-fmt.sh: build xelatex.fmt with the wasm xetex against the
  system TeX Live (found via kpsewhich), cached in engines/wasm-fmt/. One-time
  artifact, like the wasm binaries.
- engine_wasm.c: wasm_setup_texmf() points the engine's kpathsea at the system
  tree by overriding TEXMFROOT/DIST/LOCAL/SYSVAR/CNF from kpsewhich (its
  SELFAUTO* derivation lands on our binary, not the tree). When TEXPRESSO_WASM_FMT
  is set it exports TEXFORMATS and loads the format (-fmt=xelatex) for real
  LaTeX; otherwise falls back to -ini (raw primitives).

Result: `texpresso` with TEXPRESSO_WASM=1 + TEXPRESSO_WASM_FMT runs stock xetex
in-process, loads xelatex.fmt, reads article.cls/fonts from system texmf, and
produces a 1-page XDV for `\documentclass{article}...$E=mc^2$...` — real LaTeX,
no fork, no subprocess.

Follow-ups: per-session snapshot after format load (avoid re-undump per edit);
EACCES-vs-ENOENT probe noise in the io layer; bundle texmf for portability.

Snapshot the engine once at the main document's first read (before its
bytes enter TeX's line buffer) and restore it on each edit, so replay
re-reads the edited document and re-typesets the whole preamble+body with
the format still loaded — skipping module init + format undump per edit.

Host (wasm_host.c): snapshot_restore now re-arms the COW instead of
freeing, so one snapshot can be restored repeatedly; add snapshot_discard
and wire it through init/shutdown.

Frontend (engine_wasm.c): arm the snapshot yield on the doc's first read,
take it in do_run, and add do_replay (log_rollback + restore + fd restore
+ resume). engine_step replays on dirty. Env-gated self-test: mode 1
(unchanged doc -> same page count) and mode 2 (injected \clearpage ->
+1 page) both PASS, proving edited content is re-read through the restore.

Known follow-up: the .aux read-back at \enddocument aborts on replay
(kpathsea's "." directory db does not resurface doc.aux even after the
preamble re-runs). Rendered pages are correct — they ship before \end —
but the run logs an Emergency stop; cross-reference convergence needs the
kpathsea-db-on-replay fix.

The shadow-stack pointer (wasm global 0) lives in the module struct, not
in linear memory, so the COW layer never captured it — restore left it at
the first run's final value. Capture it in snapshot_take and restore it in
snapshot_restore. Verified the linear-memory restore is byte-identical to
the snapshot (0 bytes differ), so the snapshot is now complete.

Also corrects a do_run/wio_openat comment: re-running the preamble does NOT
by itself resolve the .aux read-back on replay (that divergence is a
host-input issue under investigation, not a memory-snapshot gap).

fill_estat zeroed the whole struct, so every VFS-backed stat reported
st_ino=0. The engine's (unpatched) kpathsea keys directory identity on
st_ino/st_dev; with a zero inode it dropped "." from its search path on
replay, so the .aux written earlier in the same run was not found at
\enddocument and the run aborted with an Emergency stop (the fork engine
avoids this because its patched protocol only exposes size+mtime).

The first run mostly hit real disk (real inodes) while the replay hit
cached fileentries (ino=0) — that inconsistency is what broke the replay
specifically. Base fill_estat on the cached real fs_stat (stable
ino/dev/mode/mtime, matching the first-run disk stat) and only override
the size; synthesize a stable inode from the entry pointer for
in-memory-only files. Incremental replay now completes cleanly and the
.aux read-back / cross-reference convergence works.

The TEXPRESSO_WASM_REPLAY_TEST modes were scaffolding to validate
snapshot/restore/replay headlessly before the mechanism was proven. Now
that incremental replay and re-instantiate both complete cleanly, remove
the test code from the engine. Snapshot correctness is covered by the
standalone host A/B test (TEXPRESSO_SNAPSHOT_TEST); a proper editor-
protocol-driven edit-replay test is the right permanent coverage.

Split the engine-specific configuration out of the driver. engine_wasm.c
is now engine-agnostic and exposes txp_wasm_engine_create(profile, ...);
each engine is one file (engine_wasm_<name>.c) providing a
wasm_engine_profile (format name, extra argv, ICU need). The public
txp_create_wasm_engine picks the profile from the engine path.

xetex is validated; pdftex/luatex profiles are provided against the same
contract but not yet exercised (no .fmt built for them here). The profile
also sets ICU_DATA from the format dir, so launching xetex needs only
TEXPRESSO_WASM_FMT. Adding an engine is a new profile + its wasm2c object.

A fresh clone could not build: engines/ is gitignored, and the only way to
get an engine was the full emscripten + wabt + TeX Live source toolchain.
The packager already advertised a fetch script that did not exist.

Add the tier 2 path. scripts/fetch-wasm-engine.sh downloads a prebuilt
bundle — engine.c/.h from wasm2c, the vendored wabt runtime, and ICU data —
pinned per engine so one can be re-cut without disturbing the others, and
`make fetch-engine ENGINE=<eng>` drives it. Tier 1 (build everything from
pinned upstream sources) is now `make engine-source`. Neither runs during
`make all`: a build should not reach the network on its own.

Two changes make a bundle usable:
  - src/frontend/Makefile compiles $(WASM_ENGINE_DIR)/%.o from %.c, since a
    bundle ships portable C rather than objects.
  - WABT_INC falls back to the bundle, which vendors wasm-rt.h. It was
    hardcoded to `brew --prefix wabt`, so no bundle could build on Linux.
A missing engine now names both ways to get one instead of failing with an
opaque "No rule to make target engine.o".

ICU data moves next to the engine: it is built from the ICU inside our wasm,
so it belongs to the engine, not to the format directory it happened to
share. locate_icu_data finds it beside the engine or beside the binary, which
also suits a relocatable install. Formats stay unshipped.

Verified end to end: unpacked the bundle in a scratch directory and built
texpresso against it in 1m17s with no wabt and no wasm2c present; the result
typesets. Full suite still passes.

We deliberately ship no .fmt, so a fetched engine had no format at all and
fell back to -ini — bare primitives, where \documentclass is undefined. That
made every non-developer install unusable and blocked shipping binaries.

A format cannot be skipped. latex.ltx ends with \dump, so loading the LaTeX
kernel *is* format generation: TeX cannot load it and typeset in one pass.
Measured on xetex here: 377 ms with a format, 10.4 s re-reading the kernel —
and the latter produces nothing, because the run dumps and exits.

So generate it locally instead of shipping ours. A .fmt is a memory image of
the engine after loading latex.ltx; shipping one would bake our TeX Live's
kernel into everyone's setup, to be used against packages from theirs.
ensure_format() looks for a format beside the binary (a dev tree that ran
build-wasm-fmt.sh keeps working unchanged), then in a per-user cache keyed by
engine and by the identity of latex.ltx itself — path, size and mtime — so a
TeX Live upgrade regenerates rather than silently reusing a stale kernel.

Generation runs the engine once in INITEX mode against native I/O, since the
dump has to reach the real filesystem rather than texpresso's in-memory VFS;
wasm_host_use_native_io() switches back for that one invocation.

Verified on all three engines with no format present: each generates one,
caches it, and typesets; a second run reuses the cache without regenerating,
and matches what the dev-built format produces.

luatex additionally warns once about fonts on the very first run, because
luaotfload cannot scan system font directories — getdents64 has been a stub
since the first host commit. Steady state is clean; recorded in
eclipse-roadmap.md.

unlinkat, renameat, mkdirat, rmdir, symlinkat and linkat called the host
syscall directly, so the engine could delete or rename real files even though
every read and write was being served from texpresso's in-memory VFS. Worse,
those effects sat outside the snapshot: a rollback could not undo them.

Refuse them with EROFS unless the plain native backend is installed — the
standalone binaries and the one-off format dump, which legitimately write to
disk. Typesetting is unaffected; the only mutation TeX attempted under
texpresso was removing .synctex.gz, which was already failing anyway.

Making them VFS-aware instead would mean adding the ops to wasm_io_ops and
defining delete/rename semantics for an in-memory filesystem that has
neither. Refusing is the honest behaviour until something needs more.

Verified: a canary file beside the document survives a compile, output is
unchanged, format generation still writes through the native path, and the
full suite passes.

One release, engines-2026.1-1, carrying all three engines. They are a matched
set rather than independently versioned artifacts: engine.c and wasm_host.c
share an ABI — the w2c_engine struct and the syscall import set — so a bundle
from one revision against a host from another fails to link or, worse,
misbehaves quietly. A single tag makes that coupling explicit. Checksums stay
per asset so a corrupted download still fails loudly.

Verified against the live release: xetex and pdftex both fetch, verify,
unpack and rebuild; a deliberately wrong checksum is caught and reported.
Full suite passes against the fetched engine.

engines/ is gitignored, so `make all` had nothing to link. Fetch the bundle
and cache it, keyed on the fetch script (which carries the tag and checksums).

Verification used `shasum -a 256`, macOS-only; prefer sha256sum. Add curl to
the minimal arch and fedora images.

Second job builds pdftex and luatex and typesets with each: their w2c_engine
structs differ, and wasm_host.c once failed to compile against luatex for a
week unnoticed. One platform — the ABI does not vary by OS.

Objects were not rebuilt when a header changed. Removing a field from
state_t therefore left stale objects disagreeing about its size, and the
mismatch corrupted the heap at runtime instead of failing to build — the
tests aborted inside malloc, far from the cause.

Add -MMD -MP and include the generated .d files.

… worked

`make dev` and `make debug` delegated to src/Makefile, which defines neither
and still ran `$(MAKE) -C engine` against the directory deleted in 9044d8a.
texpresso-dev.so could not link either — it omitted the wasm objects and
libcommon — and texpresso.so had no rule at all.

Hot-reload cannot work now regardless: the engine's memory and globals live in
this process, which a frontend reload does not touch. should_reload_binary was
already hardwired false, so the reload flag was never set and the driver loop
provably ran once. Drop it, loader.c, and src/Makefile.

Keep proxy.c and its target — it still builds and Emacs
texpresso-connect-debugger uses it — now reachable as `make debug-proxy`.

It copied build/texpresso-xetex as "the engine binary" for a find_engine()
that no longer exists; under Model A that path is a second full frontend. It
also omitted xetex's ICU data, which locate_icu_data() expects beside the
executable, so the bundled app fell back to -ini and could not typeset.

Verified: the bundled binary generates a format and typesets.

ENGINE was honoured by fetch-engine but ignored by texpresso, which read
WASM_ENGINE_DIR — so `make fetch-engine ENGINE=luatex && make texpresso
ENGINE=luatex` fetched luatex and then silently built xetex. One knob now
drives both, and it is namespaced: a stray ENGINE in the environment (MySQL
uses one) would otherwise redirect the build.

The profile was also picked by strstr over the executable's path, which no
build/texpresso can match — so building with luatex linked the luatex engine
and then ran the xetex profile against it, passing -no-pdf and looking for
xelatex.fmt. Bake the engine in as TEXPRESSO_ENGINE next to WASM_ENGINE_DIR so
the two cannot disagree; engine_tex.o becomes per-engine like the host shim.

  TEXPRESSO_ENGINE=luatex make fetch-engine texpresso   # verified: LuaTeX

Both reported the right thing to the editor and then ignored the answer.

Deferred queries resumed on every step whether or not the promised file had
arrived, so the engine re-ran the same lookup and deferred again — emitting
"read promised" four times while burning through kpathsea's candidates, then
giving up before the (open) landed. Resume only once the entry has data, and
report DOC_TERMINATED while parked so the main loop idles instead of spinning.

Failed lookups never triggered a recompile. interpret_open only counts a
supplied file as a change when the entry has been seen, and a file that is
merely probed with stat/access never reaches wio_openat to be marked — so the
editor could answer and nothing happened. Mark the entry when reporting it.

Both async tests polled a regular file for a marker and slept between tries.
A regular file cannot block, hence the sleep; reacting up to an interval late
also masked a bug where the engine emitted the marker repeatedly. They now
read texpresso's output through a fifo and block on it, so the reply is sent
as soon as the marker appears and the test ends when the engine exits.

Neither test checked its feature worked: both asserted only that the marker
appeared and texpresso exited 0, and both passed while the supplied file was
never used. They now require output and no "not found".

test-replay covers what only an ad-hoc script checked before: that incremental
replay is byte-identical to a full compile, at three edit positions.

test-report runs every target with PASS/FAIL and timings. The suite went from
over two minutes — 60s of it one test waiting on a watchdog — to about 20s.

The engine job passed ENGINE=pdftex and ENGINE=luatex, which the Makefile
stopped reading when the variable was renamed, so it fetched xetex twice and
the build failed with no engine in build-wasm2c-pdftex.

Both reported only that no output was produced, which says nothing about why —
the CI failure was unactionable without a local reproduction.

test-lookup-file closed stdin the moment it sent the file, making stdin_eof
true while the recompile was still running. -test-initialize quits on
"terminated && stdin_eof", and CI showed the engine starting the re-run and
exiting partway through it, before producing output. An editor would not close
the channel there either.

Linux-only so far; macOS passed throughout.

cap only says how far a layer's shadow reaches. The protected window has to
match what the engine actually has mapped, or cow_on_grow's "new_size >
g_cow_size" test cannot fire as the heap grows again and the window stops
following it.

The host's built-in fence test pushes two COW layers, runs on, then restores
each and compares a hash of the whole linear memory against what it was when
that layer was captured. Restoring layer 0 is the case replay uses when a
non-document file changes, and the one test-replay never exercises.

Runs against the standalone engine binary (make engine-native), so it covers
the snapshot machinery with TeX and the frontend out of the picture. Skips
when there is no texmf tree usable by INITEX, since the engine then exits
before its first read and there is no snapshot to check.

TeX reads the job file twice: once looking for a %&format line, then again
after undumping the format. Checkpoint 0 is armed when the document is opened,
so it lands on that first read — before the format exists. Restoring it drops
the engine back inside its own start-up, which re-reads all 690 format chunks
and re-runs the undump, so it costs a full run and differs from one only by
resuming mid-initialisation.

That resume does not survive on Linux: the engine reads the document and exits
silently, producing nothing. Measured across edit positions, every edit that
selected checkpoint 0 (roughly the first 5% of the document, the whole
preamble, and any editor-supplied file, which sets edit_pos 0) produced no
output at all, while deeper checkpoints were byte-identical to a full compile.

Re-run when the selection is checkpoint 0. Deeper checkpoints resume an
initialised engine and are the ones that actually save work.

test-replay sampled only 10/50/90%, none of which select the base checkpoint,
so it passed on Linux throughout; add a 1% position that does.

README still said we ship a custom XeTeX "modified to interact with the
TeXpresso driver". It is stock upstream sources, unpatched, compiled to wasm2c
and linked into the frontend, with three engines to choose from.

CHANGELOG said nothing about any of it, though README tells users to read it:
no mention of the engine replacement, the removal of texpresso-xetex, or the
new build step to obtain an engine.

WASM-ENGINE.md was a branch planning document rather than documentation.
Replace it with a description of the engine: why wasm, the build pipeline, the
host and its VFS seam, the coroutine, layered copy-on-write, incremental
replay, determinism, formats, and per-engine notes.

SERVER-PROTOCOL.md documented the inter-process protocol, which no longer
exists. Point at the documents that describe the current system.

It described hot-loading via texpresso.so and a texpresso-dev binary, neither
of which exists, listed loader.c and sprotocol.c after their removal, described
engine_tex.c as driving a forked texpresso-xetex, omitted the three engine
profiles, and called synctex unused when replay rolls it back.

INSTALL.md described building texpresso-xetex as a separate engine binary and
never mentioned that an engine has to be fetched or built first, which is now
the step a fresh clone fails on. Dependency lists carried cargo and omitted the
LaTeX distribution the engine needs at runtime. The targets list offered dev
and debug, neither of which exists.

Document the two-step build, TEXPRESSO_ENGINE, first-run format generation, and
the targets that are actually there. Dependency lists now match what CI
installs, which is known to work.

The in-process engine resolves packages with kpathsea, so it needs a TeX Live
tree. Tectonic on its own passed the availability check and then found no .sty,
producing an empty document with no error at all.

Verified on a tectonic-only container (no kpsewhich): before, texpresso ran to
completion and typeset nothing; now both -tectonic and auto-detection exit 1
with an explanation.

Uses exit(1) rather than returning false, because false means "quit" to the
caller and is also what a successful run returns, so the process would exit 0
and nothing could tell the two apart.

Feeding the engine from a tectonic bundle is a separate piece of work; -tectonic
still selects tectonic for rendering resources.