llmc is an efficient LLM compression tool with various advanced compression methods, supporting multiple inference backends.

Summary of system papers/frameworks/codes/tools on training or serving large model

Transformer related optimization, including BERT, GPT

Example models using DeepSpeed

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Model Quantization Benchmark

TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.

A library for researching neural networks compression and acceleration methods.