eLLM can infer LLM on CPUs faster than on GPUs
-
Updated
Jun 10, 2026 - Rust
#
moe
Here are 14 public repositories matching this topic...
Local LLM inference engine written from scratch in Rust — hand-written AVX-512 assembly kernels, Metal & Vulkan compute shaders. Supports Qwen3, Mistral3, ... Q4/INT8/BF16 quantization.
-
Updated
Mar 18, 2026 - Rust
Rust-native MoE inference runtime with custom CUDA kernels for Blackwell GPUs. Includes DFlash speculative decoding, multi-tier Engram memory, and entropy-adaptive routing. Targets Qwen3.5-35B-A3B on a single RTX 5060 Ti 16GB.
rust ffi cuda inference moe quantization mamba state-space-models deltanet blackwell engram llm qwen speculative-decoding sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Apr 25, 2026 - Rust
Run Qwen3.5-122B-A10B on a 16 GB MacBook Air via SSD-streamed MoE expert weights.
-
Updated
Jun 3, 2026 - Rust
SSD-streaming MoE inference engine for consumer hardware. Run 80B parameter models on a 24GB Mac.
-
Updated
Feb 22, 2026 - Rust
LLM inference in Rust - Metal & CUDA
rust metal cuda inference nvidia transformer openai moe llm llm-serving anthropic llm-inference qwen
-
Updated
Jun 5, 2026 - Rust
PERSPECTIVE v2 — A 1.05 trillion parameter sparse Mixture-of-Experts language model that runs on consumer hardware (4 GB VRAM + 32 GB RAM). Features O(1) perspective decay recurrence, 3D torus manifold routing, native ternary {-1,0,+1} weights, holographic distributed memory, and hard geometric safety constraints. Built in Rust.
nlp rust machine-learning research deep-learning neural-network transformer moe language-model ai-safety efficient-inference consumer-hardware mixture-of-experts on-device-ai ternary-weights llm sparse-models holographic-memory manifold-routing novel-architecture
-
Updated
Feb 20, 2026 - Rust
Runs a Rust inference server for hybrid State-Space and MoE language models with fast GPU throughput on consumer hardware
rust ffi cuda moe quantization mamba state-space-models github-config deltanet blackwell engram llm qwen speculative-decoding mamba2 hybrid-ssm
-
Updated
Jun 10, 2026 - Rust
Enabling inference of large mixture-of-experts (MoE) models on Apple Silicon using dynamic offloading.
-
Updated
Mar 11, 2026 - Rust
Heterogeneous Compute Cascade (HCC) — distributed 400B-parameter MoE inference across dual AMD Ryzen AI MAX+ 395 'Strix Halo' workstations via USB4.
-
Updated
May 17, 2026 - Rust
Frontier AI on the Macs you already own. Treats your SSD as memory and splits models across paired devices, so 18 GB models run on 8 GB Macs. Local, private, OpenAI-compatible — works with Claude Code, Cursor, any agent. Built on iroh + SwiftLM.
-
Updated
May 26, 2026 - Rust
Improve this page
Add a description, image, and links to the moe topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the moe topic, visit your repo's landing page and select "manage topics."