Lists (1)
Starred repositories
SGLang is a high-performance serving framework for large language models and multimodal models.
Go/gRPC service designed to enable generic rate limit scenarios from different types of applications.
High-Performance Implementation of OpenAI's TikToken.
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Mirage Persistent Kernel: Compiling LLMs into a MegaKernel
Automatically exported from code.google.com/p/smhasher
Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)
Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, 20+ clouds, or on-prem).
HTTP load testing tool and library. It's over 9000!
Capturing SSL/TLS plaintext without a CA certificate using eBPF. Supported on Linux/Android kernels for amd64/arm64.
Intel QuickAssist Technology( QAT) OpenSSL Engine (an OpenSSL Plug-In Engine) which provides cryptographic acceleration for both hardware and optimized software using Intel QuickAssist Technology e…
Ribbon is a Inter Process Communication (remote procedure calls) library with built in software load balancers. The primary usage model involves REST calls with various serialization scheme support.
A tool that enables intelligent interaction with framework-based repositories
Ohayou(おはよう), HTTP load generator, inspired by rakyll/hey with tui animation.
Async Mode Nginx with QAT support which improves Crypto and compression performance
GPU-accelerated vector query processing system that supports large vector datasets beyond GPU memory.
The original sources of MS-DOS 1.25, 2.0, and 4.0 for reference purposes
🐶 Kubernetes CLI To Manage Your Clusters In Style!
A high-throughput and memory-efficient inference and serving engine for LLMs
A flexible distributed key-value database that is optimized for caching and other realtime workloads.
Efficient and general syntactical decoding for Large Language Models
A library for building fast, reliable and evolvable network services.
To-do list Chrome extension for PrairieLearn
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.