Stars
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.