Open-Sora: Democratizing Efficient Video Production for All

We introduce a novel approach for parameter generation, named neural network parameter diffusion (p-diff), which employs a standard latent diffusion model to synthesize a new set of parameters

Lossless Training Speed Up by Unbiased Dynamic Data Pruning

Efficient AI Inference & Serving

☁️ Build multimodal AI applications with cloud-native stack

Making large AI models cheaper, faster and more accessible

A Python library transfers PyTorch tensors between CPU and NVMe

A curated list of awesome projects and papers for distributed training or inference

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Scalable PaLM implementation of PyTorch

Optimizing AlphaFold Training and Inference on GPU Clusters

Sky Computing: Accelerating Geo-distributed Computing in Federated Learning

Documentation for Colossal-AI

Examples of training models with hybrid parallelism using ColossalAI

Performance benchmarking with ColossalAI

Large-scale model inference.

Some basic examples of playing with RL

SimCLRv2 - Big Self-Supervised Models are Strong Semi-Supervised Learners

Reference models and tools for Cloud TPUs.

The author's officially unofficial PyTorch BigGAN implementation.

Models and examples built with TensorFlow

Compare GAN code.

Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional accumulated gradient and NVIDIA DALI dataloader.

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

Experimental ground for optimizing memory of pytorch models