Fast, flexible LLM inference

Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

A simple tool to manage encrypted secrets in YAML files with age encryption 🔑