Lists (3)
Stars
Manages Unified Access to Generative AI Services built on Envoy Gateway
Repository for open inference protocol specification
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…
A high-throughput and memory-efficient inference and serving engine for LLMs
Port of OpenAI's Whisper model in C/C++
Open source FPGA-based NIC and platform for in-network compute
A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
Mastering Computer Vision with PyTorch 2.0, published by Orange, AVA®
Implement Neural Networks in Cuda from Scratch
Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Neural network from scratch in CUDA/C++
Neural network from scratch in CUDA/C++
BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Ongoing research training transformer models at scale
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
This is a plugin which lets EC2 developers use libfabric as network provider while running NCCL applications.
Open source project for data preparation for GenAI applications
cuVS - a library for vector search and clustering on the GPU
RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing …