A high-throughput and memory-efficient inference and serving engine for LLMs

The open source developer platform to build AI agents and models with confidence. Enhance your AI applications with end-to-end tracking, observability, and evaluations, all in one integrated platform.

Universal LLM Deployment Engine with ML Compilation

SGLang is a fast serving framework for large language models and vision language models.

Modin: Scale your Pandas workflows by changing a single line of code

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, 17+ clouds, or on-prem).

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

Serverless LLM Serving for Everyone.

scBSP is a specialized package designed for processing biological data, specifically in the analysis of gene expression and cell coordinates. It efficiently computes p-values for a given set of gen…