text-compression

PAQ8PX – Experimental Lossless Data Compressor & Entropy Estimator

AI-powered text compression library for RAG systems and API calls. Reduce token usage by up to 50-60% while preserving semantic meaning with advanced compression strategies.

Offline Dictionary-based Compression (Re-Pair, Recursive Pairing)

[EMNLP'26] Code and data for VTCBench, a VLM benchmark for long-context understanding capabilities under vision-text compression paradigm.

LLMA = LLM + Arithmetic coder, which use LLM to do insane text data compression. LLMA=大模型+算术编码，它能使用LLM对文本数据进行暴力的压缩，达到极高的压缩率。

Extreme inline text compression for HTML / JS. A custom pipeline that generates stand-alone HTML or JS files which embed competitively compressed self-extracting text, with file sizes of 25% - 40% the original.

Reverse T9 for LLMs. Free, open-source prompt compressor for your AI prompts and agents.

🍶 llm-distillery ⇢ use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.

This project is about the implementation of the LZ77 and Deflate COMPRESSION algorithm.

A simple visualization for huffman tree, which is used in encoding text.

Cross platform diff engine for .net core used to compare string values and generate a comparison output

Text compression tool ⚡

Using Huffman code to encode and compress the secret text data along with using histogram shifting to encode the data into the image.

Easy data compression algorithm.

Huffman code

LZ77 algorithm, providing efficient lossless data compression and decompression capabilities. It offers a reliable and flexible solution for reducing file sizes in various programming languages.

World's best JSON/NDJSON compressor. Format-aware lossless compression engine in Rust - schema inference, columnar reorg, type-specific encoding. 2-3x better than zstd on structured data.

An attempt to compress the enwik8 file

A text compression tool & library