Starred repositories
107
stars
written in Python
Clear filter
A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems
SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)
Implementation of BTree part for paper 'The Case for Learned Index Structures'
Persist and reuse KV Cache to speedup your LLM.
Modular and structured prompt caching for low-latency LLM inference
A simple Django app to render LaTeX templates and compile them into PDF files.
Python wrappers for calling LaTeX/building LaTeX documents.
NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading
Download & install fonts from Adobe Creative Cloud
[HPCA'24] Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System
[ASPLOS'25] Towards End-to-End Optimization of LLM-based Applications with Ayo
Open Programmable Infrastructure API and Behavioral Model
SmartSSD related benchmarks and toy applications
Provide example code for machine learning class