The nnsight package enables interpreting and manipulating the internals of deep learned models.

Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of GPT-Fast, a simple, PyTorch-native generation codebase.

Recreation of RLGym done in Rust to reduce compute costs

A Data Streaming Library for Efficient Neural Network Training

Studying the variance in neural net predictions across training time

Efficiently computing & storing token n-grams from large corpora

Easily create large video dataset from video urls

Code for collecting, processing, and preparing datasets for the Common Pile

Code and Data artifact for NeurIPS 2023 paper - "Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context". `multispy` is a lsp client library in Python intended to be used to…

Reference implementation for DPO (Direct Preference Optimization)

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Pile Deduplication Code

Module for automatic summarization of text documents and HTML pages.

Repository to host architecture experiments and development using Paxml and Praxis

Training a RLHF-transformer architecture to answer farmers' queries

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RN…

Residual Shuffle-Exchange

Simple slurm tracking bot to check usage

Using queues, tqdm-multiprocess supports multiple worker processes, each with multiple tqdm progress bars, displaying them cleanly through the main process. It offers similar functionality for pyth…

The next gen UI made with simple HTML and Tailwind CSS. It's compatible with simple HTML, CSS static website or a React, Vue, Angular or Next.js Complex app. Zero JS and no-plugins installed.

Web simulation of Ubuntu 20.04, made using NEXT.js & tailwind CSS

🔥 The Complete Customizable Software Developer Portfolio Template which lets you showcase your work and provides each and every detail about you as Software Developer.

An eye-catching developer Portfolio, Built on NextJS.

Fork of kingoflolz/mesh-transformer-jax with memory usage optimizations and support for GPT-Neo, GPT-NeoX, BLOOM, OPT and fairseq dense LM. Primarily used by KoboldAI and mtj-softtuner.

Efficient Attention for Long Sequence Processing

Used for adaptive human in the loop evaluation of language and embedding models.

Provides a minimal implementation to extract FLAN datasets for further processing