A high-throughput and memory-efficient inference and serving engine for LLMs

This is a fork of SGLang for hip-attention integration. Please refer to hip-attention for detail.

这是一个用于显示当前网速、CPU及内存利用率的桌面悬浮窗软件，并支持任务栏显示，支持更换皮肤。