fp8

Star

Here are 3 public repositories matching this topic...

maeddesg / vulkanforge

Star

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

rust machine-learning amd vulkan inference mesa llm fp8 gguf rdna4 gfx1201 gemma4

Updated Jun 20, 2026
Rust

jackwangfeng / infero

Star

An inference engine written in Rust for GGUF, AWQ, and FP8 (W8A8) checkpoints, with hand-written CUDA and Metal kernels — no PyTorch, no libtorch, no ggml

rust metal gpu cuda llama quantization inference-engine openai-api awq fp8 llm-inference qwen gguf

Updated Sep 11, 2026
Rust

Threeneedled-visualcell778 / vllm-bench

Star

Benchmark vLLM serving endpoints with a fast Rust client. Use parallel dataset generation, low memory overhead, and static binaries without Python dependencies.

kubernetes cli aws benchmark terraform grafana inference triton safety datasets robustness claude eks promotheus llm fp8 multimodal-llm

Updated Sep 22, 2026
Rust

Add this topic to your repo

To associate your repository with the fp8 topic, visit your repo's landing page and select "manage topics."

Learn more