Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.

Achieve state of the art inference performance with modern accelerators on Kubernetes

Supercharge Your LLM with the Fastest KV Cache Layer

Verify Precision of all Kimi K2 API Vendor

Code search MCP for Claude Code. Make entire codebase the context for any coding agent.

A Model Context Protocol server for searching and analyzing arXiv papers

A lightweight express.js server implementing OpenAI’s Responses API, built on top of Chat Completions, powered by Hugging Face Inference Providers.

What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?

AIInfra（AI 基础设施）指AI系统从底层芯片等硬件，到上层软件栈支持AI大模型训练和推理。

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

A framework for few-shot evaluation of language models.

The official Python library for the OpenAI API

React app for inspecting, building and debugging with the Realtime API

A simple client and utils for interacting with OpenAI's Realtime API in Python

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens"

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

Build Python LLM apps in minutes ⚡️

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen etc. 开源大模型的统一后端接口

A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, OPT, and GALACTICA.

⛓️ Langflow is a UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows.

Examples and guides for using the OpenAI API

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

Port of Facebook's LLaMA model in C/C++

C++ implementation of ChatGLM-6B

GitHub repo for Azure Video Analyzer