RAT+: Train Dense, Infer Sparse - Recurrence Augmented Attention for Dilated Inference

verl: Volcano Engine Reinforcement Learning for LLMs

Pytorch domain library for recommendation systems

Ongoing research training transformer language models at scale, including: BERT & GPT-2

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

Expression language for Go

The official PyTorch implementation of the NeurIPS2022 paper, Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models

The official PyTorch implementation of the ICLR2022 paper, QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization