Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.

The most accurate natural language detection library for Rust, suitable for short text and mixed-language text

Natural language detection library for Rust. Try demo online: https://whatlang.org/

The Jieba Chinese Word Segmentation Implemented in Rust

A fast, low-resource Natural Language Processing and Text Correction library written in Rust.

🎤 vibrato: Viterbi-based accelerated tokenizer

Checks all your documentation for spelling and grammar mistakes with hunspell and a nlprule based checker for grammar

A comprehensive Rust translation of the code from Sebastian Raschka's Build an LLM from Scratch book.

Japanese Input Method System for Linux, Neural Kana-Kanji Conversion Engine + fcitx5 IME

🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer

Yet another Japanese IME for IBus/Linux

A blazingly fast phonetic reduction/hashing algorithm.

🕷️ The pipeline for the OSCAR corpus

Official Rust Implementation of Model2Vec

Simple NLP in Rust with Python bindings

Use multiple LLM backends in a single crate, simple builder-based configuration, and built-in prompt chaining & templating.

Rust port of sentence-transformers (https://github.com/UKPLab/sentence-transformers)