Supercharge Your LLM with the Fastest KV Cache Layer

The Modular Platform (includes MAX & Mojo)

NVIDIA Inference Xfer Library (NIXL)

A Datacenter Scale Distributed Inference Serving Framework

SGLang is a fast serving framework for large language models and vision language models.

A flexible distributed key-value database that is optimized for caching and other realtime workloads.

Production-Grade Container Scheduling and Management

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs

dstack is an open-source control plane for running development, training, and inference jobs on GPUs—across hyperscalers, neoclouds, or on-prem.

Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

A high-throughput and memory-efficient inference and serving engine for LLMs

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

OpenBMC Distribution

public tree for kernel development

Linux kernel source tree

kdevops main repository: Generalized devops infrastructure for Linux kernel development

MMTests: Benchmarking framework primarily aimed at Linux kernel testing

Provides a tool to test CXL with Kernel and Qemu setup

Pond: CXL-Based Memory Pooling Systems for Cloud Platforms (ASPLOS'23)

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more

DAMON user-space tool

The iFogSim Toolkit for Modeling and Simulation of Resource Management Techniques in Internet of Things, Edge and Fog Computing Environments