Distributed Compiler Based on Triton for Parallel Systems

An Open-source RL System from ByteDance Seed and Tsinghua AIR

Democratizing Reinforcement Learning for LLMs

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

Example models using DeepSpeed

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

[ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models

The release repo for "Vicuna: An Open Chatbot Impressing GPT-4"

Official Codebase for DiffuSeq

Official Implementation for the ICML2022 paper "Directed Acyclic Transformer for Non-Autoregressive Machine Translation"

Code for ACL 2020 paper: "Extractive Summarization as Text Matching"