OpenBSD-style kernel hardening roadmap #190

Track incremental Vinix kernel hardening against concrete OpenBSD mechanisms. Keep each invariant in its own reviewable PR and cover amd64 + arm64 equivalents where the architecture provides them.

Landed

• security: enforce OpenBSD-style W^X mappings #186 W^X VM policy: reject writable+executable mappings and invalid protection bits in the shared VM layer
• security: prevent supervisor execution of user pages #187 supervisor execute isolation: amd64 SMEP + arm64 PXN on all EL0 mappings
• security: enforce kernel read-only mappings on amd64 #188 supervisor write protection: amd64 CR0.WP; arm64 already enforces AP read-only mappings at EL1
• security: explicitly enable amd64 NXE #189 explicit amd64 EFER.NXE enablement on BSP/APs; arm64 uses PXN/UXN descriptors directly

In review

• security: enable amd64 UMIP #197 amd64 UMIP when CPUID advertises it
• security: add OpenBSD-style immutable mappings #199 OpenBSD-style immutable VM mappings / mimmutable mechanism for amd64 + arm64 Vinix extension ABI
• security: keep private mmap bookkeeping out of user ABIs #200 keep the private brk-reservation mmap accounting bit out of all user ABIs
• security: randomize unhinted mmap placement #201 fresh per-call mmap ASLR hints in a bounded 4 GiB LP64 window on native/Linux amd64 and Linux arm64

SMAP / PAN prerequisite work

• Convert all remaining raw user-pointer I/O paths to checked usercopy
  • central read/write interposition is being prototyped on security/checked-user-io
  • do not enable SMAP/PAN until blocking/read semantics are preserved without holding supervisor user access open
• Remove/debug-harden remaining direct user dereferences (kprint, pipe result arrays, ioctl/socket/path helpers, etc.)
• Enable amd64 SMAP after the audit is clean
• Enable arm64 PAN/EPAN after the audit is clean

Next VM / loader hardening

• Integrate immutable text/rodata/RELRO with the ELF loader and dynamic linker, including textrel handling
• Broader OpenBSD-style mmap flag validation while preserving Linux no-op hint compatibility (MAP_STACK, MAP_NORESERVE, etc.)
• Add OpenBSD-style syscall pinning only after libc/loader support exists
• Audit kernel/user address-space separation and kernel ASLR

Compiler / control-flow hardening

• Kernel stack protector: Vinix currently builds with -fno-stack-protector and has no __stack_chk_guard/__stack_chk_fail; seed a non-predictable guard early enough for boot code before enabling it globally
• Audit amd64 CET/IBT and arm64 BTI/PAC/toolchain support before enabling branch-target CFI
• Audit speculative-execution mitigations and per-CPU feature enablement on amd64/arm64

Memory / exploit mitigation

• Slab metadata isolation/sequestering: current allocator already has allocation bitmaps, invalid/double-free checks, zero-on-alloc and 0xaa poison-on-free, but headers remain writable in object pages
• Harden large-allocation metadata and investigate guard-page virtual heap mappings
• User stacks already retain an unmapped guard page on both amd64 and arm64; do not duplicate this work

Process sandboxing

• Evaluate pledge/unveil-style process sandboxing as separate ABI/userland work

Validation / infrastructure

• Add security regression tests for denied RWX, allowed RW→RX, PXN/SMEP faults, CR0.WP, NX, UMIP, immutable mappings, mmap ASLR and later SMAP/PAN
• Make CI toolchain reproducible enough that hardening PR failures can be distinguished from current-V regressions

Audit notes: Vinix already has PIE/stack/initial-mmap randomization, NX PTEs, user stack guard pages, COW-aware checked usercopy, and useful slab poisoning/double-free detection. New ports should close missing invariants rather than duplicate those mechanisms.