Parser Building Toolkit for JavaScript

Online playground for OpenAPI tokenizers

The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.

Contains source-code for viewers following along with my Beginners Guide To Building Interpreters series on my Youtube Channel.

VSCode extension to highlight nested code blocks

Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.

Promise based streaming tokenizer

A primitive token library used to read from, and to write a node Buffer

Web tool to count LLM tokens (GPT, Claude, Llama, ...)

Play with OpenAI API's using your own API Key. Your API Key is stored and used only from your browser.

Amazon S3 tokenizer

Tokenize CSS according to the CSS Syntax

📐 Fast token estimation at 94% accuracy of a full tokenizer in a 2kB bundle

🌟 A Visual Studio Code extension that displays the count of OpenAI GPT-3 tokens for the current document in the status bar. 🎯

ZON → 35-70% cheaper LLM prompts than JSON/TOON. Zero overhead.

⚙️ A simple tokenizer for deno

n8n node for working with BPE Tokens with GPT in mind.

Lexer / tokenizer

🐉 JavaScript expression tokenizer and evaluator

A lossless JSON/JSON5 tokenizer and parser for Node.js that maintains indentation, spacing, and comments.