DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

PArallel Distributed Deep LEarning

Transformer related optimization, including BERT, GPT

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Go, Javascript and more

Open Neural Network Exchange

Tensors and Dynamic neural networks in Python with strong GPU acceleration

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

PArallel Distributed Deep LEarning

Deep LSTM RNN

Computation using data flow graphs for scalable machine learning

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Computation using data flow graphs for scalable machine learning

Computation using data flow graphs for scalable machine learning

Inference code for LLaMA models

Ongoing research training transformer models at scale

Intel staging area for llvm.org contribution. Home for Intel LLVM-based projects.

TFX is an end-to-end platform for deploying production ML pipelines

An Open Source Machine Learning Framework for Everyone

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

Transformer related optimization, including BERT, GPT

TensorFlow Neural Machine Translation Tutorial

TensorFlow code and pre-trained models for BERT

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Repository for benchmarking graph neural networks (JMLR 2023)

Chinese Pre-Trained Language Models (CPM-LM) Version-I

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Introduction to CPM

Making large AI models cheaper, faster and more accessible