A very simple GRPO implement for reproducing r1-like LLM thinking.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Transformer seq2seq model, program that can build a language translator from parallel corpus

Scalable data pre processing and curation toolkit for LLMs

An implementation of Performer, a linear attention-based transformer, in Pytorch

利用Python实现中文文本关键词抽取，分别采用TF-IDF、TextRank、Word2Vec词聚类三种方法。

A simple and well styled PPO implementation. Based on my Medium series: https://medium.com/@eyyu/coding-ppo-from-scratch-with-pytorch-part-1-4-613dfc1b14c8.

Code for reproducing key results in the paper "InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets"

Customizable implementation of the self-instruct paper.

Xwin-LM: Powerful, Stable, and Reproducible LLM Alignment

LLMPerf is a library for validating and benchmarking LLMs

A lightweight multilingual LLM

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

PyTorch code for EMNLP 2019 paper "LXMERT: Learning Cross-Modality Encoder Representations from Transformers".

Fast inference from large lauguage models via speculative decoding

Crawl BookCorpus

中文实体关系抽取，pytorch，bilstm+attention

fastHan是基于fastNLP与pytorch实现的中文自然语言处理工具，像spacy一样调用方便。

A series of technical report on Slow Thinking with LLM

Plug-and-Play Image Restoration with Deep Denoiser Prior (IEEE TPAMI 2021) (PyTorch)

Expert Specialized Fine-Tuning

A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text

ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search (NeurIPS 2024)

A BERT-based Chinese Text Encoder Enhanced by N-gram Representations

Large-scale model inference.

Reference code for the paper: Deep White-Balance Editing (CVPR 2020). Our method is a deep learning multi-task framework for white-balance editing.

The codes of paper "Long Text Generation via Adversarial Training with Leaked Information" on AAAI 2018. Text generation using GAN and Hierarchical Reinforcement Learning.

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.