The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

SGLang is a high-performance serving framework for large language models and multimodal models.

Knowhere is an open-source vector search engine, integrating FAISS, HNSW, etc.

Graph Library for Approximate Similarity Search

vsag is a vector indexing library used for similarity search.

Vector database for scalable similarity search and AI applications.

cuVS - a library for vector search and clustering on the GPU

Redis for LLMs

A high-throughput and memory-efficient inference and serving engine for LLMs

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Accessible large language models via k-bit quantization for PyTorch.

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

LLM inference in C/C++

Framework for evaluating ANNS algorithms on billion scale datasets.

Scalable graph based indices for approximate nearest neighbor search

A library for efficient similarity search and clustering of dense vectors.

Knowhere is an open-source vector search engine, integrating FAISS, HNSW and Annoy

so-vits-svc fork with realtime support, improved interface and more features.

Benchmarks of approximate nearest neighbor libraries in Python

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (p…

A playbook for systematically maximizing the performance of deep learning models.