CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

The TechQA dataset -- http://ibm.biz/Tech_QA

Inference code for Llama models

Fast & Simple repository for pre-training and fine-tuning T5-style models

LOMO: LOw-Memory Optimization

LLM inference in C/C++

High-speed download of LLaMA, Facebook's 65B parameter GPT model

Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning

LLM as a Chatbot Service

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

A dataset featuring diverse dialogues between two ChatGPT (gpt-3.5-turbo) instances with system messages written by GPT-4. Covering various contexts and tasks (task-oriented dialogue systems, abstr…

AllenAI's post-training codebase

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

Train transformer language models with reinforcement learning.

In this repository, I place my solution for the exercises in multiple famous math textbooks, including Stochastic Differential Equation, Partial Differential Equation, Optimization and so on.

Making large AI models cheaper, faster and more accessible

Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)

SAM: Sharpness-Aware Minimization (PyTorch)

Synchronized Multi-GPU Batch Normalization

OpenMMLab Self-Supervised Learning Toolbox and Benchmark

A simple method to perform semi-supervised learning with limited data.

Implementation of an efficient variant of SVRG that relies on mini-batching implemented in Pytorch

Code release for Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers (ICML2019)

Code release for Separate to Adapt: Open Set Domain Adaptation via Progressive Separation (CVPR 2019)

Code release for "Conditional Adversarial Domain Adaptation" (NIPS 2018)