Making Low-Bit LLMs Actually Fast in Parallel via Vector Table Lookup
Vec-LUT turns repetitive scalar table lookups into contiguous vector reads, accelerating parallel ternary LLM inference on x86 and ARM CPUs by up to 4.2×.
Blog
Enhancing GPTQv2 Format Support in vLLM: Analysis and Implementation
An analysis of GPTQv2 format limitations in vLLM and the CUDA kernel changes needed for low-bit asymmetric quantization inference.
Vision-Language-Action (VLA) Models: A Review of Recent Progress
Recent VLAs are moving from discrete to continuous control and from single-system to dual-system designs.
Reading Notes of Dario Amodei's Blog
Notes on Dario Amodei's essays about powerful AI, interpretability, and AI policy.
Cheatsheet for Setting up Android Smartphones
A practical checklist for setting up Android smartphones for development.
Cheatsheet for Setting up Termux on Android Smartphones
A practical checklist for setting up Termux as an Android development environment.
Cheatsheet for Setting up Pi Devices
A practical setup checklist for Raspberry Pi and similar single-board computers.
How Far Are We from a "GPT in Our Pocket"?
A look at the practicalities of deploying large language models on edge devices.