A simple weather display with a cute interactive desktop pet (❛◡❛✿)

Inference framework for MoE layers based on TensorRT with Python binding

qwen2 and llama3 cpp implementation

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.