A high-throughput and memory-efficient inference and serving engine for LLMs

《动手学深度学习》：面向中文读者、能运行、可讨论。中英文版被70多个国家的500多所大学用于教学。

星火大模型 python sdk库

LLM inference in C/C++

A localized open-source AI server that is better than ChatGPT.

A docker registry proxy run on cloudflare worker.

Efficient few-shot learning with Sentence Transformers

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with …

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

🦜🔗 Build context-aware reasoning applications

jieba analysis plugin for elasticsearch 7.0.0, 6.4.0, 6.0.0, 5.4.0，5.3.0, 5.2.2, 5.2.1, 5.2, 5.1.2, 5.1.1

langchain-ChatGLM, local knowledge based ChatGLM with langchain ｜ 基于本地知识的 ChatGLM 问答

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

NvChad's Official Terminal Plugin ( Unmaintained but still usable and stable), wait for v3.0

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

huggingface/text-generation-inference 适配昇腾NPU

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

Faster Whisper transcription with CTranslate2

Ascend PyTorch adapter (torch_npu). Mirror of https://gitee.com/ascend/pytorch

Blazing fast Neovim config providing solid defaults and a beautiful UI, enhancing your neovim experience.

Inference Llama 2 in one file of pure C

Fast inference engine for Transformer models

The nanoGPT-style implementation of RWKV Language Model - an RNN with GPT-level LLM performance.

Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons