AI and cloud-native proxy server and framework

MCP server for troubleshooting vLLM inference workloads on Red Hat OpenShift AI — queries Prometheus, Alertmanager, Loki, Grafana, and Kubernetes from AI assistants.

Model as a Service

llm-d helm charts and deployment examples

Demo integrating Kuadrant with llm-d

A simple GPU reservation tool for single host shared development systems

Extract SRT subtitles with timestamps from a video file with the Whisper voice model

Achieve state of the art inference performance with modern accelerators on Kubernetes

Helm charts for llm-d

A high-throughput and memory-efficient inference and serving engine for LLMs

Fun with benchmarks

UI Component for Chatbot

Get your documents ready for gen AI

Running Docling as an API service

Place to hack on UI for InstructLab

On-demand self-hosted AWS EC2 runner for GitHub Actions

Interact with the Deep Search platform for new knowledge explorations and discoveries

Create fast graph language models from converted PDF documents for knowledge extraction and Q&A.

InstructLab Community wide collaboration space including contributing, security, code of conduct, etc

InstructLab Core package. Use this to chat with a model and execute the InstructLab workflow to train a model using custom taxonomy data.

Taxonomy tree that will allow you to create models tuned with your data

GitHub bot to assist with the taxonomy contribution workflow

alfred workflow jwt decoder

Mesh network using QUIC Connect-Ip Tunnels

Simple example of a Quic Client/Server

LLMs for Project Nexodus documentation question-answering

POC using OpenAI with chatbot docs