toonconv is a Rust CLI tool for converting JSON data into TOON (Token-Oriented Object Notation) format. It's designed for LLM applications, data processing pipelines, and scenarios where token efficiency matters.
- 📦 Multiple Input Methods: Direct strings, stdin, files, and recursive directories
- 📊 Token Optimization: Reduces token count by 35-54% compared to JSON
cargo install toonconv# Clone the repository
git clone https://github.com/lst97/toonconv.git
cd toonconv
# Build release version
cargo build --release
# Install globally (optional)
cargo install --path .- Rust 1.75+ (Install Rust)
# Direct JSON string (NEW!)
toonconv '{"name": "Alice", "age": 30}'
# From file
toonconv input.json -o output.toon
# From stdin
echo '{"status": "ok"}' | toonconv --stdin
# Batch directory conversion
toonconv input_dir/ -o output_dir/# Input JSON
{"users": [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]}
# Output TOON
users[2]{id,name}: 1,Alice 2,Bobtoonconv supports 4 different ways to convert JSON to TOON:
Perfect for quick conversions and testing.
# Convert a JSON object
toonconv '{"name": "Alice", "age": 30}'
# Convert a JSON array
toonconv '[1, 2, 3, 4, 5]'
# Nested structures
toonconv '{"user": {"name": "Bob", "email": "bob@example.com"}}'Great for Unix pipelines and data workflows.
# From echo
echo '{"status": "ok"}' | toonconv --stdin
# From API response
curl -s https://api.example.com/data | toonconv --stdin
# Chain with jq
jq '.results[]' data.json | toonconv --stdinConvert individual files with custom output paths.
# Basic conversion
toonconv input.json
# Custom output
toonconv input.json -o custom_output.toon
# Pretty formatting
toonconv input.json --format prettyProcess entire directories while preserving structure.
# Recursive conversion (default)
toonconv input_dir/ -o output_dir/
# Non-recursive
toonconv input_dir/ -o output_dir/ --no-recursive
# Continue on errors
toonconv input_dir/ -o output_dir/ --continue-on-error# Pretty-print output (default)
toonconv data.json --format pretty
# Compact output
toonconv data.json --format compact
# Minified output
toonconv data.json --format minified# Set memory limit (default: 512MB)
toonconv large.json --memory-limit 1073741824 # 1GB
# For very large files
toonconv huge.json --memory-limit 2147483648 # 2GB# Quiet mode (errors only)
toonconv data.json --quiet
# Verbose mode (detailed progress)
toonconv data.json --verbose
# Debug mode (maximum detail)
toonconv data.json --debugtoonconv/
├── src/
│ ├── cli/ # CLI argument parsing and commands
│ ├── conversion/ # Core conversion engine
│ ├── formatter/ # TOON format output generation
│ ├── parser/ # JSON parsing with validation
│ ├── validation/ # Input validation and error handling
│ ├── error/ # Custom error types
│ ├── lib.rs # Library entry point
│ └── main.rs # Binary entry point
├── tests/ # Comprehensive test suite
├── benches/ # Performance benchmarks
├── examples/ # Usage examples
└── specs/ # TOON format specifications
- Language: Rust 1.75+
- JSON Processing:
serde,serde_json - CLI Framework:
clapwith derive macros - Performance: Optional
simd-jsonsupport - Terminal UI:
indicatif,console - File Operations:
walkdirfor recursive traversal
Run the comprehensive test suite:
# All tests
cargo test
# TOON specification compliance
cargo test --test toon_spec_compliance_test
# Performance benchmarks
cargo bench
# Code quality
cargo clippy
cargo fmt# Development build
cargo build
# Release build (optimized)
cargo build --release
# Check code quality
cargo clippy
cargo fmt --check- Fork the repository
- Create a feature branch:
git checkout -b feature-name - Make your changes and add tests
- Run the test suite:
cargo test - Submit a pull request
See AGENTS.md for development guidelines.
- USAGE_GUIDE.md: Comprehensive usage examples and patterns
- BUILD.md: Build instructions and troubleshooting
- BENCHMARK.md: Performance analysis and comparisons
- specs/: TOON format specifications
# Fetch, filter, convert
curl -s https://api.example.com/users | \
jq '.data.users' | \
toonconv --stdin -o users.toon# Convert all JSON files, skip errors
find . -name "*.json" -type f | while read file; do
toonconv "$file" -o "${file%.json}.toon" --continue-on-error
done# Clean build
cargo clean
cargo update
cargo build --release- Invalid JSON: Validate input with a JSON validator
- Memory limits: Increase with
--memory-limitflag - File not found: Use absolute paths if relative paths fail
This project is licensed under the MIT License - see the LICENSE file for details.