Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ORE

ORE

A self-hosted systems language compiler targeting native ARM64. The compiler is written in ORE itself (stdlib/*.ore): lexer, parser, AST, type checker, three-address IR, optimizer, liveness + linear-scan register allocation, and AArch64 codegen are all ORE programs. The backend emits assembly directly and the same compiler also produces freestanding images and a small kernel that run on QEMU's virt machine with no host runtime.

No LLVM, no JIT, and no Rust in the normal build path.

Self-hosting bootstrap

The committed seed binary seed/obecir is a Mach-O build of the current compiler sources. tools/ore.sh rebuilds the seed from stdlib/*.ore whenever the sources change (a self-hosting bootstrap step) and records the binary and source hashes in seed/SHA256SUMS.

target/release/ore (the Rust reference) is not part of the build path. It survives only as an optional differential oracle for a few test harnesses when present. The src/ Rust reference is retained for comparison and bootstrap history; Rust is not required by the normal self-hosted ORE compile path.

Building and running

./tools/ore.sh prog.ore            # compile (reg backend) -> assembly on stdout
./tools/ore.sh prog.ore out.s      # compile to out.s
./tools/ore.sh --run prog.ore      # compile + as + ld + run
./tools/ore.sh --stack prog.ore    # stack backend (oracle)
./tools/ore.sh --noopt prog.ore    # optimizer disabled
./tools/ore.sh --asm prog.ore      # print assembly to stdout

Assembling and linking use the system as/ld against libSystem (hosted mode). The seed rebuild and all tests require an Apple Silicon Mac (the freestanding/kernel gates additionally require QEMU).

Language

fn fib(n: int) -> int {
    if (n < 2) { return n; }
    return fib(n - 1) + fib(n - 2);
}

fn main() -> int {
    var i = 0;
    while (i < 15) { print(fib(i)); i = i + 1; }
    return 0;
}
  • Types: int (i64), bool, str, ptr, ptr<T>, struct, enum (+ match with payload binding).
  • Literals: decimal and hex (0x10, 0xff, 0xFFFFFFFFFFFFFFFF = -1, two's complement).
  • Heap model: arrays/structs/enums are malloc'd with reference semantics; new Arr(n) for dynamic arrays; no GC.
  • Operators: + - * / %, & | ^ ~ << >>, comparisons, && || ! (eager — see below), casts int(x)/float(x).
  • Statements: var, assignment, if/else, while, break/continue, return, print(...).
  • Currently unsupported: for loops, floating-point (see below), closures and generics (supported only by the Rust reference, not the self-hosted frontend), and the other gaps tracked in tools/FEATURES.csv.

&&/|| are eager

ORE's &&/|| lower both operands before the compare branch (lower.ore emits both vregs, then IBin op 20/21). They are not short-circuit. Never use &&/|| to guard a memory access: the eager backend evaluates both sides before the guard resolves, so a guard expression such as c >= 0 && read(f.vty[c]) can still read f.vty[-1] on legal IBr(-1) sentinels. Use nested ifs for guards.

Floating point

Floating-point literals and operations are not part of the language. The lexer rejects float literals with LEX ERROR: float literals are not supported. The float(x) spelling is currently an integer identity cast — it changes the declared type, not the value's representation.

Optimizer

Liveness-driven linear-scan register allocation, compare-branch fusion, constant folding and hoisting, function inlining, LICM, induction-variable strength reduction, loop strength reduction, CFG simplification, and dead/unreachable-code elimination. Every pass is followed by the IR verifier; any violation aborts the compile. Optimized and unoptimized output are compared by the test differentials.

The freestanding/OS stack

The same reg backend that emits hosted macOS binaries emits freestanding images that boot on QEMU virt with no host runtime:

prog.ore --(seed/obecir, reg backend)--> prog.s --(tools/ore_asm.py)-->
raw image --(QEMU -bios, flash at 0x0)--> tools/ore_boot_loader.s copies
flash -> RAM at 0x40000000 --> runtime (tests/ore_rt.s: _printf, _malloc
bump, _strlen, _putchar -> PL011 UART, _exit) --> main()
  • tools/ore_asm.py — the ORE assembler (two-pass, AArch64 subset, symbol/relocation handling, byte-verified against as by tests/asm_diff.py). The ORE-written encoder (stdlib/encoder.ore) is the long-term owner of this path.
  • tools/boot_freestanding.sh — the full ORE-owned pipeline (tests/freestanding.ore: arrays, structs, enums + match, strings, allocation, recursion).

The kernel (tests/kernel.s)

An EL1 kernel with EL0 user tasks, built and booted entirely by ORE tooling:

  • hardened EL0<->EL1 boundary with full context frames (x0–x30, SP_EL0, ELR, SPSR) per task
  • first-fit physical allocator (split/coalesce) with alloc/free syscalls
  • MMU — 4KB granule, 2MB blocks, identity map, AP-bit EL0 isolation
  • EL1 physical timer (GIC PPI 30) + preemptive round-robin scheduler
  • ramdisk filesystem inside the kernel image + file syscalls (open/lseek/read/close)
  • an ORE-compiled program as an EL0 task with real file I/O
  • the compiler itself runs as an EL0 task under the kernel (see below)

Syscalls (x8 = number): 1 putchar, 2 exit, 3 alloc, 4 free, 5 open, 6 lseek, 7 read, 8 close. Each task has its own identity TTBR0 address space; kernel memory and the allocator's range are EL1-only.

Kernel context-switch invariants

The register-preservation rules that keep the preemptive switch correct:

  • The exception entry must branch (b), not bl, to the save frame: a bl would overwrite x30 with the kernel return address before the frame is pushed. The task's return address travels in x16.
  • x9 and x16 must be pushed above the frame before the msr sp_el0/elr/spsr sequence, and the task's x9 reloaded from the frame afterwards — otherwise the resumed task runs with a corrupted x9/x16.
  • A timer IRQ arriving mid-svc-handler must not orphan the frame holding the task's user registers: the IRQ entry checks SPSR_EL1.M, and IRQs taken from EL1 are deferred (plain eret; the pending IRQ fires at EL0 where it is handled).

tests/kernel_stress.s + tools/boot_stress.sh verify every register across svc + preemption rounds — a permanent gate for these invariants.

The second self-hosting loop

tools/boot_selfhost.sh runs the seed's own compiler inside the ORE environment:

  1. The compiler (obec-ir.ore, reg backend) boots as an EL0 task under the kernel, fully timer-preempted, with --backend reg argv.
  2. It reads its own source + all 10 transitive #includes from a ramdisk served by the kernel's file syscalls.
  3. It recompiles itself and emits the 1,241,469-byte assembly.
  4. The host captures it and compares byte-for-byte with the seed's own output — identical (provenance).
  5. The emitted .s is re-assembled into a compiler image whose SHA-256 equals the original (fixed point).
  6. The rebuilt image is booted and reproduces itself (reboot proof).

This is a compiler fixed point: the ORE-written compiler, running inside the ORE environment, rebuilds itself to a byte-identical image.

Tests and gates

./test.sh               # compile/run diffs vs .expected, opt-vs-noopt
                        #   differentials, IR/reg backend differentials,
                        #   encoder gate, KORE (EL0 file syscalls),
                        #   STRESS (register preservation), kernel gates
./tools/bootstrap       # 13 self-hosting gates (seed rebuild, fixed points)

test.sh is de-Rustified: the seed compiler runs everything; Rust is an optional oracle for a few differential harnesses. Current state: 160 passed (54 differential), 0 failed, 21 skipped (documented language gaps).

Roadmap

Implemented:

  • self-hosted compiler (seed bootstrap, fixed point, provenance hashes)
  • freestanding pipeline (compiler -> ORE assembler -> raw image -> QEMU)
  • kernel with EL0 isolation, preemptive scheduler, ramdisk filesystem, and an ORE program as an EL0 task
  • the second self-hosting loop (compiler inside ORE, byte-identical rebuild, fixed-point image, reboot proof)

In progress / planned:

  • ORE-owned image path: replace the Python assembler with the stdlib/encoder.ore-backed pipeline
  • kernel hardening: physical memory discovery, more syscalls, userspace stability under load
  • the third loop: the rebuilt compiler rebuilding the rebuilt kernel

Documents

  • docs/ABI.md — the precise ABI the backends generate: register classes, stack frames, struct/enum/closure representations, and the ore_argc/ore_arg/ore_read_file runtime helpers.
  • tools/FEATURES.csv — feature-by-feature status against the Rust reference.
  • bench/HISTORY.md — benchmark history and methodology.
  • docs/ATTRIBUTION.md — build/tooling dependencies.

License

Apache License 2.0. See LICENSE.

About

Self-hosted AArch64 systems compiler with a freestanding runtime, preemptive kernel, and in-ORE self-hosting loop.

Topics

Resources

Stars

7 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages