kvcache.ai
Pinned Loading
Repositories
- kvcache-blog Public
-
- ktransformers Public
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
- Model-Optimizer Public Forked from NVIDIA/Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
- accelerate Public Forked from huggingface/accelerate
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
- transformers Public Forked from huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Top languages
Loading…
Most used topics
Loading…