UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

qwen2 and llama3 cpp implementation

Inference framework for MoE layers based on TensorRT with Python binding

A simple weather display with a cute interactive desktop pet (❛◡❛✿)

🚀 Run any LLM on any hardware. 130% faster MoE inference with ExpertFlow + TurboQuant KV compression. Ollama-compatible API. Built on llama.cpp.

Production-grade LLM inference in Rust. Single binary, OpenAI-compatible, runs on Apple Silicon and CUDA.

Musuka・むすか・娘化　is an application, which can exchange your shortcut to any image in a new desktop, especially moe girl character image.

Downstream llama.cpp TurboQuant CUDA fork with adaptive KV layout selection for long-context inference on consumer Blackwell GPUs.

Inspired by flashDmoe paper: https://arxiv.org/abs/2506.04667 paper. Implementation for AMD's GPU ecosystem.