Senior Software Engineer @fullbeaker · Full-stack web · Self-hosting LLMs on the side
- Pacific North West
Highlights
- Pro
Starred repositories
Quantize HuggingFace models to NVFP4 and serve them with vLLM on NVIDIA Blackwell GPUs.
A high-throughput and memory-efficient inference and serving engine for LLMs