Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

A framework for few-shot evaluation of language models.

the portable Python dataframe library

verl: Volcano Engine Reinforcement Learning for LLMs

Nano vLLM

🙌 OpenHands: Code Less, Make More

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

A high-throughput and memory-efficient inference and serving engine for LLMs

SkyRL-v0: Train Real-World Long-Horizon Agents via Reinforcement Learning

AI Tensor Engine for ROCm

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

CUDA Templates for Linear Algebra Subroutines

Tile primitives for speedy kernels

NVIDIA Linux open GPU with P2P support

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)

Helpful tools and examples for working with flex-attention

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Attachment Management in Obsidian

Official inference repo for FLUX.1 models

Optax is a gradient processing and optimization library for JAX.

a state-of-the-art-level open visual language model | 多模态预训练模型

Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!

Inference code for LLaMA models