Free, open-source, 100% offline voice dictation for Linux. Speak and type anywhere via whisper.cpp, Whisper & VOSK engines, GPU-accelerated, works on X11 + Wayland!

Docker configuration for running VLLM on dual DGX Sparks

Experimental CUDA kernel framework unifying typed dimensions, NVRTC JIT specialization, and ML‑guided tuning.

LLM scaling book in LaTex

Inference at the speed of light.

Complete solutions to the Programming Massively Parallel Processors Edition 4

Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage Less.

Nano vLLM

Official implementation of the paper: "A Refined Analysis of Massive Activations in LLMs".

Official implementation of the "Variance control via weight rescaling in LLM pretraining" paper.

Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".

🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

🐍🌿💚 Integrate MyPy type-checking into your Python Bazel builds

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

LLM101n: Let's build a Storyteller

Curated list of kubectl plugins

The Abstraction and Reasoning Corpus

Type annotations and runtime checking for shape and dtype of JAX/NumPy/PyTorch/etc. arrays. https://docs.kidger.site/jaxtyping/

LLM training in simple, raw C/CUDA

Inference Llama 2 in one file of pure C

🙌 OpenHands: AI-Driven Development

Perplexity Inspired Answer Engine

Colossus — An example microservice architecture for Kubernetes using Bazel, Go, Java, Docker, Kubernetes, Minikube, Gazelle, gRPC, Prometheus, Grafana, and more