Ongoing research training transformer language models at scale, including: BERT & GPT-2

Quantized inference code for LLaMA models

FuseAI Project

A crude RLHF layer on top of nanoGPT with Gumbel-Softmax trick

Fine-tuning "ImageBind One Embedding Space to Bind Them All" with LoRA

A PyTorch implementation of EfficientNet

Experimental fork of Facebooks LLaMa model which runs it with GPU acceleration on Apple Silicon M1/M2

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

Fast and memory-efficient exact attention

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

CE-GPPO: Controlling Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning

A super-fast Python implementation of seam carving algorithm for intelligent image resizing.

🤗 GIFT-SW for PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Retrieval via attention

A high-throughput and memory-efficient inference and serving engine for LLMs

label-smooth, amsoftmax, partial-fc, focal-loss, triplet-loss, lovasz-softmax. Maybe useful

Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

DROJ, a DRO-inspired embedding-level jailbreak method.

Code for paper LAFITE: Towards Language-Free Training for Text-to-Image Generation (CVPR 2022)

8-bit CUDA functions for PyTorch

Optimizing Hyperparameters with Conformal Quantile Regression