Stars
Production inference for encoder models - ColBERT, GLiNER, ColPali, embeddings etc. - as vLLM plugins for online and in-process deployment
A simple GPU reservation tool for single host shared development systems
wentao.site / Hugo Template / A template repository for Hugo based blog
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Achieve state of the art inference performance with modern accelerators on Kubernetes
An efficient, composable design pattern for range processing
A High-Performance JIT-Based C++ Expression/Script Execution Engine with SIMD Vectorization Support
A curated list of awesome SIMD frameworks, libraries and software
Performance-portable, length-agnostic SIMD with runtime dispatch
Simple Useful Libraries: C++17/20 header-only dynamic bitset
Top-level directory for documentation and general content
Notebooks using the Neural Magic libraries 📓
Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes
Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models
Sparsity-aware deep learning inference runtime for CPUs
neuralmagic / nm-vllm
Forked from vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs
ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
A high-throughput and memory-efficient inference and serving engine for LLMs
A utility for creating amalgamated single-header C++ libraries
A machine learning compiler for GPUs, CPUs, and ML accelerators
CLI utility for managing your project, a modern touch for C/C++
"See why!" Explains and suggests fixes for compile-time errors for C, C++, C#, Go, Java, LaTeX, PHP, Python, Ruby, Rust, and TypeScript
Athena++ radiation GRMHD code and adaptive mesh refinement (AMR) framework