Skip to content
#

model-compression

Star

Here are 8 public repositories matching this topic...

Language: Rust
Filter by language
All 662 Python 462 Jupyter Notebook 86 Rust 8 C++ 6 HTML 5 C 4 TypeScript 4 Go 3 JavaScript 3 TeX 3

Anonym0usWork1221 / Whetstone

Star 3

Low-bit LLM inference engine in Rust + CUDA. Converts models to a 4-bit .wstone format (3.75x smaller) and runs them on consumer GPUs. Built for memory-bandwidth-bound decode on Turing, where bytes per weight — not TOPS — sets token throughput.

rust machine-learning deep-learning gpu cuda transformers cuda-kernels gpu-computing quantization performance-optimization inference-engine model-compression edge-ai tensor-cores low-bit-quantization int4 llm gptq llm-inference qwen
  • Updated Jul 29, 2026
  • Rust

VeyrForge / GreenCompress

Star 1

Post-training weight compression for low-RAM machines: Q4/Q8 quantization, green-format repair, AVX2 CPU inference, optional CUDA. ~45% less RAM at ~99.9% quality.

rust inference simd transformer neural-networks avx2 quantization int8 model-compression memory-optimization model-quantization post-training-quantization cpu-inference int4 llm gguf green-compression weight-compression low-memory-inference greencompress
  • Updated Jul 31, 2026
  • Rust

Add this topic to your repo

To associate your repository with the model-compression topic, visit your repo's landing page and select "manage topics."

Learn more