Token-Oriented Object Notation is a compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage. It's intended for LLM input as a lossless, drop-in representation of JSON data.

TOON's sweet spot is uniform arrays of objects – multiple fields per row, same structure across items. It borrows YAML's indentation-based structure for nested objects and CSV's tabular format for uniform data rows, then optimizes both for token efficiency in LLM contexts. For deeply nested or non-uniform data, JSON may be more efficient.

TOON achieves CSV-like compactness while adding explicit structure that helps LLMs parse and validate data reliably.

• Why TOON?
• Key Features
• Benchmarks
• 📋 Full Specification
• Installation & Quick Start
• CLI
• Format Overview
• API
• Using TOON in LLM Prompts
• Notes and Limitations
• Syntax Cheatsheet
• Other Implementations

AI is becoming cheaper and more accessible, but larger context windows allow for larger data inputs as well. LLM tokens still cost money – and standard JSON is verbose and token-expensive:

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

TOON conveys the same information with fewer tokens:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

• 💸 Token-efficient: typically 30–60% fewer tokens than JSON¹
• 🤿 LLM-friendly guardrails: explicit lengths and fields enable validation
• 🍱 Minimal syntax: removes redundant punctuation (braces, brackets, most quotes)
• 📐 Indentation-based structure: like YAML, uses whitespace instead of braces
• 🧺 Tabular arrays: declare keys once, stream data as rows

Benchmarks are organized into two tracks to ensure fair comparisons:

• Mixed-Structure Track: Datasets with nested or semi-uniform structures (TOON vs JSON, YAML, XML). CSV excluded as it cannot properly represent these structures.
• Flat-Only Track: Datasets with flat tabular structures where CSV is applicable (CSV vs TOON vs JSON, YAML, XML).

Token counts are measured using the GPT-5 o200k_base tokenizer via gpt-tokenizer. Savings are calculated against formatted JSON (2-space indentation) as the primary baseline, with additional comparisons to compact JSON (minified), YAML, and XML. Actual savings vary by model and tokenizer.

The benchmarks test datasets across different structural patterns (uniform, semi-uniform, nested, deeply nested) to show where TOON excels and where other formats may be better.

Datasets with nested or semi-uniform structures. CSV excluded as it cannot properly represent these structures.

🛒 E-commerce orders with nested structures  ┊  Tabular: 33%
   │
   TOON                █████████████░░░░░░░    72,743 tokens
   ├─ vs JSON          (−33.1%)               108,731 tokens
   ├─ vs JSON compact  (+5.5%)                 68,936 tokens
   ├─ vs YAML          (−14.1%)                84,724 tokens
   └─ vs XML           (−40.5%)               122,313 tokens

🧾 Semi-uniform event logs  ┊  Tabular: 50%
   │
   TOON                █████████████████░░░   153,223 tokens
   ├─ vs JSON          (−15.0%)               180,196 tokens
   ├─ vs JSON compact  (+19.9%)               127,740 tokens
   ├─ vs YAML          (−0.8%)                154,514 tokens
   └─ vs XML           (−25.2%)               204,800 tokens

🧩 Deeply nested configuration  ┊  Tabular: 0%
   │
   TOON                ██████████████░░░░░░       631 tokens
   ├─ vs JSON          (−31.3%)                   919 tokens
   ├─ vs JSON compact  (+11.9%)                   564 tokens
   ├─ vs YAML          (−6.2%)                    673 tokens
   └─ vs XML           (−37.4%)                 1,008 tokens

──────────────────────────────────── Total ────────────────────────────────────
   TOON                ████████████████░░░░   226,597 tokens
   ├─ vs JSON          (−21.8%)               289,846 tokens
   ├─ vs JSON compact  (+14.9%)               197,240 tokens
   ├─ vs YAML          (−5.5%)                239,911 tokens
   └─ vs XML           (−30.9%)               328,121 tokens

Datasets with flat tabular structures where CSV is applicable.

👥 Uniform employee records  ┊  Tabular: 100%
   │
   CSV                 ███████████████████░    46,956 tokens
   TOON                ████████████████████    49,827 tokens   (+6.1% vs CSV)
   ├─ vs JSON          (−60.7%)               126,854 tokens
   ├─ vs JSON compact  (−36.8%)                78,850 tokens
   ├─ vs YAML          (−50.0%)                99,701 tokens
   └─ vs XML           (−66.0%)               146,440 tokens

📈 Time-series analytics data  ┊  Tabular: 100%
   │
   CSV                 ██████████████████░░     8,396 tokens
   TOON                ████████████████████     9,128 tokens   (+8.7% vs CSV)
   ├─ vs JSON          (−59.0%)                22,258 tokens
   ├─ vs JSON compact  (−35.8%)                14,224 tokens
   ├─ vs YAML          (−48.9%)                17,871 tokens
   └─ vs XML           (−65.7%)                26,629 tokens

⭐ Top 100 GitHub repositories  ┊  Tabular: 100%
   │
   CSV                 ███████████████████░     8,513 tokens
   TOON                ████████████████████     8,745 tokens   (+2.7% vs CSV)
   ├─ vs JSON          (−42.3%)                15,145 tokens
   ├─ vs JSON compact  (−23.7%)                11,455 tokens
   ├─ vs YAML          (−33.4%)                13,129 tokens
   └─ vs XML           (−48.8%)                17,095 tokens

──────────────────────────────────── Total ────────────────────────────────────
   CSV                 ███████████████████░    63,865 tokens
   TOON                ████████████████████    67,700 tokens   (+6.0% vs CSV)
   ├─ vs JSON          (−58.8%)               164,257 tokens
   ├─ vs JSON compact  (−35.2%)               104,529 tokens
   ├─ vs YAML          (−48.2%)               130,701 tokens
   └─ vs XML           (−64.4%)               190,164 tokens

{
  "metrics": [
    {
      "date": "2025-01-01",
      "views": 7708,
      "clicks": 595,
      "conversions": 69,
      "revenue": 15369.93,
      "bounceRate": 0.35
    },
    {
      "date": "2025-01-02",
      "views": 5894,
      "clicks": 381,
      "conversions": 21,
      "revenue": 2112.12,
      "bounceRate": 0.3
    },
    {
      "date": "2025-01-03",
      "views": 6835,
      "clicks": 422,
      "conversions": 35,
      "revenue": 4525.73,
      "bounceRate": 0.5
    },
    {
      "date": "2025-01-04",
      "views": 5325,
      "clicks": 305,
      "conversions": 22,
      "revenue": 2445.3,
      "bounceRate": 0.44
    },
    {
      "date": "2025-01-05",
      "views": 2974,
      "clicks": 61,
      "conversions": 6,
      "revenue": 956.57,
      "bounceRate": 0.47
    }
  ]
}

metrics[5]{date,views,clicks,conversions,revenue,bounceRate}:
  2025-01-01,7708,595,69,15369.93,0.35
  2025-01-02,5894,381,21,2112.12,0.3
  2025-01-03,6835,422,35,4525.73,0.5
  2025-01-04,5325,305,22,2445.3,0.44
  2025-01-05,2974,61,6,956.57,0.47

{
  "repositories": [
    {
      "id": 28457823,
      "name": "freeCodeCamp",
      "repo": "freeCodeCamp/freeCodeCamp",
      "description": "freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming,…",
      "createdAt": "2014-12-24T17:49:19Z",
      "updatedAt": "2025-10-28T11:58:08Z",
      "pushedAt": "2025-10-28T10:17:16Z",
      "stars": 430886,
      "watchers": 8583,
      "forks": 42146,
      "defaultBranch": "main"
    },
    {
      "id": 132750724,
      "name": "build-your-own-x",
      "repo": "codecrafters-io/build-your-own-x",
      "description": "Master programming by recreating your favorite technologies from scratch.",
      "createdAt": "2018-05-09T12:03:18Z",
      "updatedAt": "2025-10-28T12:37:11Z",
      "pushedAt": "2025-10-10T18:45:01Z",
      "stars": 430877,
      "watchers": 6332,
      "forks": 40453,
      "defaultBranch": "master"
    },
    {
      "id": 21737465,
      "name": "awesome",
      "repo": "sindresorhus/awesome",
      "description": "😎 Awesome lists about all kinds of interesting topics",
      "createdAt": "2014-07-11T13:42:37Z",
      "updatedAt": "2025-10-28T12:40:21Z",
      "pushedAt": "2025-10-27T17:57:31Z",
      "stars": 410052,
      "watchers": 8017,
      "forks": 32029,
      "defaultBranch": "main"
    }
  ]
}

repositories[3]{id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch}:
  28457823,freeCodeCamp,freeCodeCamp/freeCodeCamp,"freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming,…","2014-12-24T17:49:19Z","2025-10-28T11:58:08Z","2025-10-28T10:17:16Z",430886,8583,42146,main
  132750724,build-your-own-x,codecrafters-io/build-your-own-x,Master programming by recreating your favorite technologies from scratch.,"2018-05-09T12:03:18Z","2025-10-28T12:37:11Z","2025-10-10T18:45:01Z",430877,6332,40453,master
  21737465,awesome,sindresorhus/awesome,😎 Awesome lists about all kinds of interesting topics,"2014-07-11T13:42:37Z","2025-10-28T12:40:21Z","2025-10-27T17:57:31Z",410052,8017,32029,main

Benchmarks test LLM comprehension across different input formats using 201 data retrieval questions on 4 models.

Each format's overall performance, balancing accuracy against token cost:

TOON           ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓   15.6  │  68.7% acc  │  4,389 tokens
CSV            ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓   15.3  │  62.3% acc  │  4,080 tokens
JSON compact   ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░   13.5  │  67.2% acc  │  4,982 tokens
YAML           ▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░   11.2  │  66.7% acc  │  5,976 tokens
JSON           ▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░    9.0  │  65.7% acc  │  7,260 tokens
XML            ▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░    8.1  │  66.8% acc  │  8,251 tokens

TOON achieves 68.7% accuracy (vs JSON's 65.7%) while using 39.5% fewer tokens.

Accuracy across 4 LLMs on 201 data retrieval questions:

gpt-5-nano
→ TOON           ██████████████████░░    88.6% (178/201)
  JSON compact   ██████████████████░░    88.1% (177/201)
  CSV            ██████████████████░░    88.0% (88/100)
  YAML           █████████████████░░░    84.6% (170/201)
  XML            ████████████████░░░░    81.6% (164/201)
  JSON           ████████████████░░░░    80.1% (161/201)

claude-haiku-4-5-20251001
  YAML           ██████████░░░░░░░░░░    52.2% (105/201)
→ TOON           ██████████░░░░░░░░░░    50.7% (102/201)
  JSON           ██████████░░░░░░░░░░    50.2% (101/201)
  JSON compact   ██████████░░░░░░░░░░    49.8% (100/201)
  XML            ██████████░░░░░░░░░░    49.3% (99/201)
  CSV            ████████░░░░░░░░░░░░    39.0% (39/100)

gemini-2.5-flash
  XML            █████████████████░░░    86.1% (173/201)
→ TOON           █████████████████░░░    84.1% (169/201)
  CSV            ████████████████░░░░    82.0% (82/100)
  JSON compact   ████████████████░░░░    81.1% (163/201)
  YAML           ████████████████░░░░    81.1% (163/201)
  JSON           ████████████████░░░░    81.1% (163/201)

grok-4-fast-non-reasoning
→ TOON           ██████████░░░░░░░░░░    51.2% (103/201)
  JSON           ██████████░░░░░░░░░░    51.2% (103/201)
  XML            ██████████░░░░░░░░░░    50.2% (101/201)
  JSON compact   ██████████░░░░░░░░░░    49.8% (100/201)
  YAML           ██████████░░░░░░░░░░    48.8% (98/201)
  CSV            ████████░░░░░░░░░░░░    40.0% (40/100)

Key tradeoff: TOON achieves 68.7% accuracy (vs JSON's 65.7%) while using 39.5% fewer tokens on these datasets.

# npm
npm install @toon-format/toon

# pnpm
pnpm add @toon-format/toon

# yarn
yarn add @toon-format/toon

Example usage:

import { encode } from '@toon-format/toon'

const data = {
  users: [
    { id: 1, name: 'Alice', role: 'admin' },
    { id: 2, name: 'Bob', role: 'user' }
  ]
}

console.log(encode(data))
// users[2]{id,name,role}:
//   1,Alice,admin
//   2,Bob,user

Command-line tool for converting between JSON and TOON formats.

npx @toon-format/cli [options] [input]

Standard input: Omit the input argument or use - to read from stdin. This enables piping data directly from other commands.

Auto-detection: The CLI automatically detects the operation based on file extension (.json → encode, .toon → decode). When reading from stdin, use --encode or --decode flags to specify the operation (defaults to encode).

# Encode JSON to TOON (auto-detected)
npx @toon-format/cli input.json -o output.toon

# Decode TOON to JSON (auto-detected)
npx @toon-format/cli data.toon -o output.json

# Output to stdout
npx @toon-format/cli input.json

# Pipe from stdin (no argument needed)
cat data.json | npx @toon-format/cli
echo '{"name": "Ada"}' | npx @toon-format/cli

# Explicit stdin with hyphen (equivalent to above)
cat data.json | npx @toon-format/cli -

# Decode from stdin
cat data.toon | npx @toon-format/cli --decode

# Show token savings when encoding
npx @toon-format/cli data.json --stats -o output.toon

# Tab-separated output (often more token-efficient)
npx @toon-format/cli data.json --delimiter "\t" -o output.toon

# Pipe-separated with length markers
npx @toon-format/cli data.json --delimiter "|" --length-marker -o output.toon

# Lenient decoding (skip validation)
npx @toon-format/cli data.toon --no-strict -o output.json

# Stdin workflows
echo '{"name": "Ada", "age": 30}' | npx @toon-format/cli --stats
cat large-dataset.json | npx @toon-format/cli --delimiter "\t" > output.toon

Simple objects with primitive values:

encode({
  id: 123,
  name: 'Ada',
  active: true
})

id: 123
name: Ada
active: true

Nested objects:

encode({
  user: {
    id: 123,
    name: 'Ada'
  }
})

user:
  id: 123
  name: Ada

encode({
  tags: ['admin', 'ops', 'dev']
})

tags[3]: admin,ops,dev

When all objects share the same primitive fields, TOON uses an efficient tabular format:

encode({
  items: [
    { sku: 'A1', qty: 2, price: 9.99 },
    { sku: 'B2', qty: 1, price: 14.5 }
  ]
})

items[2]{sku,qty,price}:
  A1,2,9.99
  B2,1,14.5

Tabular formatting applies recursively: nested arrays of objects (whether as object properties or inside list items) also use tabular format if they meet the same requirements.

encode({
  items: [
    {
      users: [
        { id: 1, name: 'Ada' },
        { id: 2, name: 'Bob' }
      ],
      status: 'active'
    }
  ]
})

items[1]:
  - users[2]{id,name}:
    1,Ada
    2,Bob
    status: active

Arrays that don't meet the tabular requirements use list format:

items[3]:
  - 1
  - a: 1
  - text

When objects appear in list format, the first field is placed on the hyphen line:

items[2]:
  - id: 1
    name: First
  - id: 2
    name: Second
    extra: true

When you have arrays containing primitive inner arrays:

encode({
  pairs: [
    [1, 2],
    [3, 4]
  ]
})

pairs[2]:
  - [2]: 1,2
  - [2]: 3,4

Empty containers have special representations:

encode({ items: [] }) // items[0]:
encode([]) // [0]:
encode({}) // (empty output)
encode({ config: {} }) // config:

TOON quotes strings only when necessary to maximize token efficiency:

• Inner spaces are allowed; leading or trailing spaces force quotes.
• Unicode and emoji are safe unquoted.
• Quotes and control characters are escaped with backslash.

Keys are unquoted if they match the identifier pattern: start with a letter or underscore, followed by letters, digits, underscores, or dots (e.g., id, userName, user_name, user.name, _private). All other keys must be quoted (e.g., "user name", "order-id", "123", "order:id", "").

String values are quoted when any of the following is true:

Examples of unquoted strings: Unicode and emoji are safe (hello 👋 world), as are strings with inner spaces (hello world).

Some non-JSON types are automatically normalized for LLM-safe output:

Converts any JSON-serializable value to TOON format.

Parameters:

• value – Any JSON-serializable value (object, array, primitive, or nested structure). Non-JSON-serializable values (functions, symbols, undefined, non-finite numbers) are converted to null. Dates are converted to ISO strings, and BigInts are emitted as decimal integers (no quotes).
• options – Optional encoding options:
  • indent?: number – Number of spaces per indentation level (default: 2)
  • delimiter?: ',' | '\t' | '|' – Delimiter for array values and tabular rows (default: ',')
  • lengthMarker?: '#' | false – Optional marker to prefix array lengths (default: false)

Returns:

A TOON-formatted string with no trailing newline or spaces.

Example:

import { encode } from '@toon-format/toon'

const items = [
  { sku: 'A1', qty: 2, price: 9.99 },
  { sku: 'B2', qty: 1, price: 14.5 }
]

encode({ items })

Output:

items[2]{sku,qty,price}:
  A1,2,9.99
  B2,1,14.5

The delimiter option allows you to choose between comma (default), tab, or pipe delimiters for array values and tabular rows. Alternative delimiters can provide additional token savings in specific contexts.

Using tab delimiters instead of commas can reduce token count further, especially for tabular data:

const data = {
  items: [
    { sku: 'A1', name: 'Widget', qty: 2, price: 9.99 },
    { sku: 'B2', name: 'Gadget', qty: 1, price: 14.5 }
  ]
}

encode(data, { delimiter: '\t' })

Output:

items[2	]{sku	name	qty	price}:
  A1	Widget	2	9.99
  B2	Gadget	1	14.5

Benefits:

• Tabs are single characters and often tokenize more efficiently than commas.
• Tabs rarely appear in natural text, reducing the need for quote-escaping.
• The delimiter is explicitly encoded in the array header, making it self-descriptive.

Considerations:

• Some terminals and editors may collapse or expand tabs visually.
• String values containing tabs will still require quoting.

Pipe delimiters offer a middle ground between commas and tabs:

encode(data, { delimiter: '|' })

Output:

items[2|]{sku|name|qty|price}:
  A1|Widget|2|9.99
  B2|Gadget|1|14.5

The lengthMarker option adds an optional hash (#) prefix to array lengths to emphasize that the bracketed value represents a count, not an index:

const data = {
  tags: ['reading', 'gaming', 'coding'],
  items: [
    { sku: 'A1', qty: 2, price: 9.99 },
    { sku: 'B2', qty: 1, price: 14.5 },
  ],
}

console.log(
  encode(data, { lengthMarker: '#' })
)
// tags[#3]: reading,gaming,coding
// items[#2]{sku,qty,price}:
//   A1,2,9.99
//   B2,1,14.5

// Custom delimiter with length marker
console.log(
  encode(data, { lengthMarker: '#', delimiter: '|' })
)
// tags[#3|]: reading|gaming|coding
// items[#2|]{sku|qty|price}:
//   A1|2|9.99
//   B2|1|14.5

Converts a TOON-formatted string back to JavaScript values.

Parameters:

• input – A TOON-formatted string to parse
• options – Optional decoding options:
  • indent?: number – Expected number of spaces per indentation level (default: 2)
  • strict?: boolean – Enable strict validation (default: true)

Returns:

A JavaScript value (object, array, or primitive) representing the parsed TOON data.

Example:

import { decode } from '@toon-format/toon'

const toon = `
items[2]{sku,qty,price}:
  A1,2,9.99
  B2,1,14.5
`

const data = decode(toon)
// {
//   items: [
//     { sku: 'A1', qty: 2, price: 9.99 },
//     { sku: 'B2', qty: 1, price: 14.5 }
//   ]
// }

Strict Mode:

By default, the decoder validates input strictly:

• Invalid escape sequences: Throws on "\x", unterminated strings.
• Syntax errors: Throws on missing colons, malformed headers.
• Array length mismatches: Throws when declared length doesn't match actual count.
• Delimiter mismatches: Throws when row delimiters don't match header.

• Format familiarity and structure matter as much as token count. TOON's tabular format requires arrays of objects with identical keys and primitive values only. When this doesn't hold (due to mixed types, non-uniform objects, or nested structures), TOON switches to list format where JSON can be more efficient at scale.
  • TOON excels at: Uniform arrays of objects (same fields, primitive values), especially large datasets with consistent structure.
  • JSON is better for: Non-uniform data, deeply nested structures, and objects with varying field sets.
  • CSV is more compact for: Flat, uniform tables without nesting. TOON adds structure ([N] length markers, delimiter scoping, deterministic quoting) that improves LLM reliability with minimal token overhead.
• Token counts vary by tokenizer and model. Benchmarks use a GPT-style tokenizer (cl100k/o200k); actual savings will differ with other models (e.g., SentencePiece).
• TOON is designed for LLM input where human readability and token efficiency matter. It's not a drop-in replacement for JSON in APIs or storage.

TOON works best when you show the format instead of describing it. The structure is self-documenting – models parse it naturally once they see the pattern.

Wrap your encoded data in a fenced code block (label it ```toon for clarity). The indentation and headers are usually enough – models treat it like familiar YAML or CSV. The explicit length markers ([N]) and field headers ({field1,field2}) help the model track structure, especially for large tables.

For output, be more explicit. When you want the model to generate TOON:

• Show the expected header (users[N]{id,name,role}:). The model fills rows instead of repeating keys, reducing generation errors.
• State the rules: 2-space indent, no trailing spaces, [N] matches row count.

Here's a prompt that works for both reading and generating:

Data is in TOON format (2-space indent, arrays show length and fields).

```toon
users[3]{id,name,role,lastLogin}:
  1,Alice,admin,2025-01-15T10:30:00Z
  2,Bob,user,2025-01-14T15:22:00Z
  3,Charlie,user,2025-01-13T09:45:00Z
```

Task: Return only users with role "user" as TOON. Use the same header. Set [N] to match the row count. Output only the code block.

// Object
{ id: 1, name: 'Ada' }          → id: 1
                                  name: Ada

// Nested object
{ user: { id: 1 } }             → user:
                                    id: 1

// Primitive array (inline)
{ tags: ['foo', 'bar'] }        → tags[2]: foo,bar

// Tabular array (uniform objects)
{ items: [                      → items[2]{id,qty}:
  { id: 1, qty: 5 },                1,5
  { id: 2, qty: 3 }                 2,3
]}

// Mixed / non-uniform (list)
{ items: [1, { a: 1 }, 'x'] }   → items[3]:
                                    - 1
                                    - a: 1
                                    - x

// Array of arrays
{ pairs: [[1, 2], [3, 4]] }     → pairs[2]:
                                    - [2]: 1,2
                                    - [2]: 3,4

// Root array
['x', 'y']                      → [2]: x,y

// Empty containers
{}                              → (empty output)
{ items: [] }                   → items[0]:

// Special quoting
{ note: 'hello, world' }        → note: "hello, world"
{ items: ['true', true] }       → items[2]: "true",true

• Python: toon_format (in development)
• Rust: toon_format (in development)

• .NET: ToonSharp
• C++: ctoon
• Clojure: toon
• Crystal: toon-crystal
• Dart: toon
• Elixir: toon_ex
• Gleam: toon_codec
• Go: gotoon
• Java: JToon
• Lua/Neovim: toon.nvim
• OCaml: ocaml-toon
• PHP: toon-php
• Python: python-toon
• Ruby: toon-ruby
• Swift: TOONEncoder

MIT License © 2025-PRESENT Johann Schopplich