Stars
Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
A PyTorch implementation of the Transformer model in "Attention is All You Need".
The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
A collection of libraries to optimise AI model performances
An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & vLLM & Ray & Dynamic Sampling & Async Agentic RL)
A faster pytorch implementation of faster r-cnn
Accessible large language models via k-bit quantization for PyTorch.
Collection of generative models, e.g. GAN, VAE in Pytorch and Tensorflow.
[ICLR 2024] Efficient Streaming Language Models with Attention Sinks
Example models using DeepSpeed
Firefly: 大模型训练工具，支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型
A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Supercharge Your LLM with the Fastest KV Cache Layer
Example code for the book Fluent Python, 1st Edition (O'Reilly, 2015)
OpenChat: Advancing Open-source Language Models with Imperfect Data
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
An Open-Source Package for Neural Relation Extraction (NRE)
Llama3、Llama3.1 中文后训练版仓库 - 微调、魔改版本有趣权重 & 训练、推理、评测、部署教程视频 & 文档。
An Open-Source Package for Knowledge Embedding (KE)
A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型
A live stream development of RL tunning for LLM agents
Entropy Based Sampling and Parallel CoT Decoding