Foundry materializes CUDA graphs along with its execution context to disk to support fast cold start of serving engines.

Demo of quota based GPU rebalancing

llmd autoscaling based on token processing rate of the inference servers

Inference Platform Simulation

Repeatable LLM inference benchmarks in OpenShift.

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Offline optimization of your disaggregated Dynamo graph

Tools and technologies that are hosted on an OpenShift cluster

A limited and opensource version of the popular app WIFI WPS WPA TESTER

claude skills for llm-d by Red Hat with love

Demo - Distributed Inference on OpenShift (aka llm-d)

Starter templates for Obsidian

OB_Templates is a Obsidian reference for note templates focused on new users of the application using only core plugins.

A framework for designing, executing and analysing experiment campaigns

GenAI inference performance benchmarking tool

A powerful little TUI framework 🏗

High-performance safetensors model loader

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

Variant optimization autoscaler for distributed inference workloads

A high-throughput and memory-efficient inference and serving engine for LLMs

A benchmarking and performance analysis framework

Helper scripts for working with "Principles of Planetary Climate," including graphing and reference values

Python utilities for the book "Principles of Planetary Climate", by Raymond T. Pierrehumbert

An open-source model for extreme atmospheres on rocky exoplanets.

Generate Kubernetes architecture diagrams from Kubernetes manifest files, kustomization files, Helm charts, helmfiles, and actual cluster state

Carbon Efficient Karpenter: Optimizing Kubernetes Cluster Autoscaling for Carbon Efficiency

21 Lessons, Get Started Building with Generative AI