Local LLM inference engine written from scratch in Rust — hand-written AVX-512 assembly kernels, Metal & Vulkan compute shaders. Supports Qwen3, Mistral3, ... Q4/INT8/BF16 quantization.
-
Updated
Mar 18, 2026 - Rust
#
moe
Here are 8 public repositories matching this topic...
Rust-native MoE inference runtime with custom CUDA kernels for Blackwell GPUs. Includes DFlash speculative decoding, multi-tier Engram memory, and entropy-adaptive routing. Targets Qwen3.5-35B-A3B on a single RTX 5060 Ti 16GB.
rust ffi cuda inference moe quantization mamba state-space-models deltanet blackwell engram llm qwen speculative-decoding sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Apr 14, 2026 - Rust
SSD-streaming MoE inference engine for consumer hardware. Run 80B parameter models on a 24GB Mac.
-
Updated
Feb 22, 2026 - Rust
PERSPECTIVE v2 — A 1.05 trillion parameter sparse Mixture-of-Experts language model that runs on consumer hardware (4 GB VRAM + 32 GB RAM). Features O(1) perspective decay recurrence, 3D torus manifold routing, native ternary {-1,0,+1} weights, holographic distributed memory, and hard geometric safety constraints. Built in Rust.
nlp rust machine-learning research deep-learning neural-network transformer moe language-model ai-safety efficient-inference consumer-hardware mixture-of-experts on-device-ai ternary-weights llm sparse-models holographic-memory manifold-routing novel-architecture
-
Updated
Feb 20, 2026 - Rust
Enabling inference of large mixture-of-experts (MoE) models on Apple Silicon using dynamic offloading.
-
Updated
Mar 11, 2026 - Rust
Improve this page
Add a description, image, and links to the moe topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the moe topic, visit your repo's landing page and select "manage topics."