Stars
VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
MSCCL++: A GPU-driven communication stack for scalable AI applications
Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation
NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the effective training time by minimizing the downtime due to fa…
A PyTorch native platform for training generative AI models
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory…
Efficient Triton Kernels for LLM Training
Survey: A collection of AWESOME papers and resources on the latest research in Mixture of Experts.
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
LLM training code for Databricks foundation models
Implementation of "SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks"
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RN…
General technology for enabling AI capabilities w/ LLMs and MLLMs
🦜🔗 Build context-aware reasoning applications
Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
Code and documentation to train Stanford's Alpaca models, and generate the data.
Human preference data for "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback"
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
Visualizer for neural network, deep learning and machine learning models
Library for reading and writing large multi-dimensional arrays.
NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems.