A high-throughput and memory-efficient inference and serving engine for LLMs

This is a fork of SGLang for hip-attention integration. Please refer to hip-attention for detail.

NKI tests

SGLang is a fast serving framework for large language models and vision language models.

Robust Speech Recognition via Large-Scale Weak Supervision

Up to 4x faster decoding than vLLM using HiP Attention: https://github.com/DeepAuto-AI/hip-attention

Numba optimized version of `pypareto`. Sorting chains for pareto frontier extraction

an u-net with some algorithm to take sketch from paints

Language models are open knowledge graphs ( non official implementation )