Stars
Open-source inference server and production cluster for all the models your agent needs.
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
A rich terminal UI for GitHub that doesn't break your flow.
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
A Nix library for building cargo projects. Never build twice thanks to incremental artifact caching.
cuda-oxide is an experimental Rust-to-CUDA compiler that lets you write (SIMT) GPU kernels in safe(ish), idiomatic Rust. It compiles standard Rust code directly to PTX — no DSLs, no foreign languag…
A Git-compatible VCS that is both simple and powerful
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…
SGLang is a high-performance serving framework for large language models and multimodal models.
Pure Rust + CUDA LLM inference engine
A high-throughput and memory-efficient inference and serving engine for LLMs
Ultra-low latency WebSocket library for HFT applications
Blazingly fast pusher drop-in replacement written in rust
Open agent protocol for NautilusTrader
The headless browser for AI agents and web scraping
Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, a…
Multi-tier firewall for AI agents — prompt injection, jailbreak, and scope violation protection
Open-source AI agent red-team engine, SDK, and CLI. Run offline or against the Humanbound Platform.
A scalable, distributed, collaborative, document-graph database, for the realtime web
Micro second messaging that stores everything to disk
IronSBE is a complete Rust implementation of the [Simple Binary Encoding (SBE)](https://www.fixtrading.org/standards/sbe/) protocol, designed for ultra-low-latency financial systems. It provides bo…
Empowering everyone to build reliable and efficient software.
Asterinas aims to be a production-grade Linux alternative—memory safe, high-performance, and more.
The easiest way to launch scripts, change settings, and run actions on Kobo e-readers.