Stars
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
Staging repo for development of native port of TypeScript
A full-featured CLI textarea component for React Ink — Implement beautiful multi-line text input on the CLI
W8A8/W4A8 inference + optimized SDPA on Apple Silicon — unlocking unused INT8 TensorOps in M5 for 1.2–1.9× faster LLM prefill, plus FlashInfer-inspired GQA decode attention for up to 1.6× SDPA spee…
OpenTUI is a library for building terminal user interfaces (TUIs)
Sandboxed npm execution for MCP servers via Apple container
A lightweight sandboxing tool for enforcing filesystem and network restrictions on arbitrary processes at the OS level, without requiring a container.
Generate TypeScript types from OpenAPI 3 specs
Python tool for converting files and office documents to Markdown.
Speed and samples benchmark: for all types of text to speech (TTS) models on Windows/Linux/Mac.
A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows
📰 Must-read papers and blogs on Speculative Decoding ⚡️
Create a good-looking DMG for your macOS app in seconds
[ICLR 2026] ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
Everything we actually know about the Apple Neural Engine (ANE)
Control what your AI can see. LeanCTX (Lean Context) is the context intelligence layer for AI agents — one local Rust binary that decides what they read, remembers what they learn, guards what they…
Metal FP32 Vs BF16 Vs FP16 benchmark
Try to make similar features like YTLite as much as possible.
Official code repo for the O'Reilly Book - "Hands-On Large Language Models"
pyright fork with various type checking improvements, improved vscode support and pylance features built into the language server
Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"