LLMs are cool but the compute itself is closed-source
thus, (very early) open-source LLM inference SoC built from scratch
more precisely, a highly parameterized 4-way out-of-order RISC-V CPU that controls a Language Processing Unit (LPU). the LPU is an embedded transformer using Digital Compute-in-Memory (DCIM) for efficient general matrix-matrix (GEMM) and matrix-vector (GEMV) operations.
3x ASICs via tinytapeout arriving Nov '26 for CPU and DCIM characterization.
- >30 tok/s, 32K window Llama 3.2 8B Q4
- >100 TOPS/W on FP8 (~12x NVIDIA H100 SXM @ 8.4 TOPS/W)
- 4-way OoO RISC-V CPU (parameterized
RV32GCorRV64GC) - optional ISA
MAFDCZbb_Zcb_Zicsr_Zifencei_Zicond_Zfbfmin - CPU boots custom uLinux, runs ML models via tinygrad
- LPU with ~1MB CIM, >4GB GDDR6 (if)
- precision: BMX8 (block-mixed-FP8, E4M3, block=32)
- 2D torous NoC inter-chip-interconnect (ICI) topology
phase CPU
- cycle-accurate model
- in-order CPU RTL
- OoO upgrade
phase LPU
- tinygrad functional models (nanoGPT, LLaMA 3.2 both on MXFP8)
- CPU/LPU memory map design
- cycle-accurate model
- CIM fabric
- primitives (relu, div, sqrt, exp, norm)
- sequencer
- attention
- end-to-end vs tinygrad golden reference
soon.
no FPGA/ASIC bring-up yet.
git clone https://github.com/ezrawolf/tinymoa.git
cd tinymoa
# build
uv sync
veryl build
# run all tests or individually
pytest -q --no-header --tb=short -n auto
pytest -q --no-header --tb=short -n auto -k test_cpu_aludocs/ (soon)
software/ (soon) custom tinygrad AoT compiler, runtime
firmware/ (soon) custom linux kernel, drivers, bootloader
hardware/rtl/ SoC home in Veryl
hardware/fpga/ FPGA targets for verification
hardware/tests/ pytest + cocotb testbenches
hardware/sandbox/ cycle-accurate python models