#

bpe

Here are 2 public repositories matching this topic...

ankane / youtokentome-ruby

High performance unsupervised text tokenization for Ruby

unsupervised-learning word-segmentation tokenization npl bpe byte-pair-encoding

Updated Dec 27, 2023
Ruby

moznion / riktoken

Pure Ruby "partial" implementation of openai/tiktoken

ruby bpe tiktoken

Updated Dec 18, 2025
Ruby

Improve this page

Add a description, image, and links to the bpe topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the bpe topic, visit your repo's landing page and select "manage topics."