SGLang is a fast serving framework for large language models and vision language models.

Kubeflow Notebooks lets you run web-based development environments on your Kubernetes cluster by running them inside Pods.

Machine Learning Toolkit for Kubernetes

An elegant HTTP Cache implementation for HTTPX and HTTP Core.

A homebrew tap for trojan.

⚡ Building applications with LLMs through composability ⚡

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

A high-throughput and memory-efficient inference and serving engine for LLMs

Jupyter protocol client APIs

A repository for Kustomize manifests

lightweight connection pooler for PostgreSQL

Excel spreadsheet crawler and table parser for data extraction and querying

Large Language Model Text Generation Inference

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

The production toolkit for LLMs. Observability, prompt management and evaluations.