cuDF - GPU DataFrame Library

SQL Parser for C++. Building C++ object structure from SQL statements.

LingoDB: A new analytical database system that blurs the lines between databases and compilers.

Google Research

Toolbox for converting analog to spiking neural networks (ANN to SNN), and running them in a spiking neuron simulator.

Flax is a neural network library for JAX that is designed for flexibility.

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data

Colossal-AI: A Unified Deep Learning System for Big Model Era

Auto parallelization for large-scale neural networks

Fast and memory-efficient exact attention

Transformer related optimization, including BERT, GPT

Distributed Communication-Optimal Shuffle and Transpose Algorithm

Trax — Deep Learning with Clear Code and Speed

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

Simple thread safe annotation based C++ profiler.

Mesh TensorFlow: Model Parallelism Made Easier

A PyTorch implementation of the Transformer model in "Attention is All You Need".

Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.

Model parallel transformers in JAX and Haiku

Parallelformers: An Efficient Model Parallelization Toolkit for Deployment

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

A flexible package manager that supports multiple versions, configurations, platforms, and compilers.

Code for "Heterogenity-Aware Cluster Scheduling Policies for Deep Learning Workloads", which appeared at OSDI 2020

Large scale embeddings on a single machine.

Distributed Communication-Optimal LU-factorization Algorithm