TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches. EMNLP Findings 2024

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs o…

GPU scheduler for elastic/distributed deep learning workloads in Kubernetes cluster (IC2E'23)

The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System

Transform audio-visual content into navigable knowledge.

List of TMKOC episodes

FedCV: An Industrial-grade Federated Learning Framework for Diverse Computer Vision Tasks

Semantic segmentation models with 500+ pretrained convolutional and transformer-based backbones.

A library for answering questions using data you cannot see

A complete computer science study plan to become a software engineer.