Tools for merging pretrained large language models.

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language Models (LLMs).

Efficient Triton Kernels for LLM Training

Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models

Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models)

Repo for Paper "Dual-Space Knowledge Distillation for Large Language Models".

Large Language Model Text Generation Inference

LLM training code for MosaicML foundation models

A high-throughput and memory-efficient inference and serving engine for LLMs

Code for the paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers"

A training data creation and management system focused on information extraction

Public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C

Compute xxHash hash codes for 8 keys in parallel

Deep Voice: Real-time Neural Text-to-Speech

DeepDive

Concurrent data structures in C++

A Java ClassLoader implementation that yanks classes directly from an Amazon Web Services S3 bucket.