Llama 3 inference in C on the PlayStation Portable (PSP).

Locaily - Making Large Language Model Inference Accessible on Consumer Hardware

llama.cpp 🦙 LLM inference in TypeScript

Kolosal AI is an OpenSource and Lightweight alternative to Ollama to run LLMs 100% offline on your device.

Explore LLM model deployment based on AXera's AI chips

Run generative AI models in sophgo BM1684X/BM1688

Kolosal AI is an OpenSource and Lightweight alternative to LM Studio to run LLMs 100% offline on your device.

No code AI agents

校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。

A high-performance inference system for large language models, designed for production environments.

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.