Distribute and run LLMs with a single file.

Locally run an Instruction-Tuned Chat-Style LLM

Web Serving and Remote Procedure Calls at 50x lower latency and 70x higher bandwidth than FastAPI, implementing JSON-RPC & REST over io_uring ☎️

Dynamic Memory Management for Serving LLMs without PagedAttention

A PostgreSQL extension for collecting statistics about sorts, helping tuning work_mem