A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.

UP-TO-DATE LLM Adaptive thinking paper. 🔥🔥🔥

LLM KV cache compression made easy

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Language Model for All.

Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

A framework for few-shot evaluation of language models.

This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

Repository for Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

d3.js

Tell stories of your work with Notion

A high-throughput and memory-efficient inference and serving engine for LLMs