Skip to content

imazen/zenzstd

Repository files navigation

zenzstd CI crates.io lib.rs docs.rs MSRV license

A pure-Rust Zstandard (RFC 8878) compressor and decompressor. #![forbid(unsafe_code)] by default and no_std + alloc, so it runs anywhere from servers to embedded targets to WebAssembly.

The decoder builds on the well-maintained ruzstd decoder (39M+ downloads). zenzstd adds a full compressor (levels 1-22), std::io streaming encode/decode, dictionary support, and optional SIMD acceleration via archmage.

Quick start

[dependencies]
zenzstd = "0.1.0"

The stream module mirrors the zstd crate's one-shot helpers (requires the default std feature):

use zenzstd::stream;

// Compress a byte slice at level 3 (Zstandard's default level).
let compressed = stream::encode_all(&b"hello hello hello world"[..], 3).unwrap();

// Decompress. `decode_all` applies a 1 GiB output-size cap to guard against
// decompression bombs; use `decode_all_unbounded` for fully trusted input.
let original = stream::decode_all(&compressed[..]).unwrap();
assert_eq!(original.as_slice(), &b"hello hello hello world"[..]);

// Reader-to-writer variants, also `zstd`-crate compatible:
// stream::copy_encode(source, &mut dest, 3)?;
// stream::copy_decode(compressed_reader, &mut dest)?;

Low-level / no_std

Without std, use the buffer-oriented API directly:

use zenzstd::encoding::{compress_to_vec, CompressionLevel};
use zenzstd::decoding::FrameDecoder;

let data = b"the quick brown fox the quick brown fox";
let compressed = compress_to_vec(&data[..], CompressionLevel::Default);

// `decode_all_to_vec` writes into the vector's spare capacity and never
// reallocates, so reserve enough room up front.
let mut decoder = FrameDecoder::new();
let mut out = Vec::with_capacity(data.len() + 4096);
decoder.decode_all_to_vec(&compressed, &mut out).unwrap();
assert_eq!(out.as_slice(), &data[..]);

Streaming encode (std)

use std::io::Write;
use zenzstd::encoding::{StreamingEncoder, CompressionLevel};

let mut output = Vec::new();
let mut encoder = StreamingEncoder::new(&mut output, CompressionLevel::Level(3));
encoder.write_all(b"streamed payload").unwrap();
encoder.finish().unwrap();

For incremental decoding, decoding::StreamingDecoder implements std::io::Read.

Compression levels

CompressionLevel maps named presets onto numeric zstd levels, or you can pass an explicit Level(1..=22):

Preset zstd level Status
Uncompressed 0 raw block, wrapped in a Zstandard frame
Fastest 1 stable
Default 3 stable (used when no level is given)
Better 7 stable
Best 11 stable
Level(1..=15) 1-15 stable — round-trip fuzzed; output verified decodable by the reference C zstd library
Level(16..=22) 16-22 experimental — see note below

Levels 16-22 are experimental. The optimal-parse match finder (BtOpt/BtUltra) has a known, data-dependent corruption bug at these levels, so they are excluded from round-trip fuzzing and not yet recommended for production. Use levels 1-15 today; level 3 is the default. The decoder handles valid Zstandard streams at all levels (verified against C zstd output for levels 1-22).

Decompression safety

stream::decode_all and stream::copy_decode apply a 1 GiB output-size cap by default — a few KB of crafted RLE blocks can otherwise expand to terabytes. For other policies:

  • decode_all_with_max(reader, Some(bytes)) / copy_decode_with_max(...) — custom cap.
  • decode_all_unbounded(reader) / copy_decode_unbounded(...) — no cap (trusted input only).
  • On StreamingDecoder, set_max_output_size(Some(bytes)).

Features

Feature Default Description
std yes std::io traits, StreamingEncoder/StreamingDecoder, and the stream module
hash yes XXH64 content checksums in frames
simd yes AVX2/BMI2 acceleration via archmage / magetypes (#[autoversion] on hot loops)
dict_builder no Dictionary training from sample data (pulls in fastrand; implies std)
unsafe-decompress no Unchecked indexing in decode hot paths
unsafe-compress no Unchecked indexing in encode hot paths (reserved)
fuzz_exports no Exposes FSE/Huffman internals for fuzz targets

For no_std, disable default features and re-enable what you need:

[dependencies]
zenzstd = { version = "0.1.0", default-features = false, features = ["hash"] }

Safety

By default the crate is #![forbid(unsafe_code)]. Enabling unsafe-decompress or unsafe-compress switches it to #![deny(unsafe_code)], with unsafe permitted only inside small, documented unsafe_ops modules in the hot paths. The safe-by-default build is the one fuzzed and tested in CI.

Benchmarks

zenzstd ships two comparison harnesses against the reference C library (zstd crate, bundled libzstd) and upstream ruzstd. Results are machine-specific — reproduce on your own hardware:

cargo bench --bench compress_compare    # zenbench: ratio + encode + decode vs C zstd
cargo run --release --example compare   # quick table across levels and datasets

At levels 1-15 the encoder is competitive with the reference library on compression ratio while trading encode speed for memory safety; the decoder (built on ruzstd) is competitive with C zstd on typical inputs. Methodology, environment, pinned competitor versions, and exact repro commands live in benchmarks/README.md. No fixed throughput numbers are published here because they vary by CPU; run the harness for figures on your machine.

Fuzzing

Six cargo-fuzz targets cover decode, round-trip, streaming, dictionary, FSE, and Huffman paths (round-trip fuzzing is restricted to levels 0-15 while 16-22 are experimental):

cargo +nightly fuzz run fuzz_decode
cargo +nightly fuzz run fuzz_roundtrip
cargo +nightly fuzz run fuzz_streaming_roundtrip

Minimum supported Rust version

zenzstd builds on Rust 1.89 and newer. Bumping the MSRV is treated as a minor-version change while the crate is pre-1.0.

License

MIT. A fork of ruzstd by Moritz Borcherding, extended with full compression. See LICENSE.

Image tech I maintain

Codecs ¹ zenjpeg · zenpng · zenwebp · zengif · zenavif · zenjxl · zenbitmaps · heic · zentiff · zenpdf · zensvg · zenjp2 · zenraw · ultrahdr
Codec internals zenjxl-decoder · jxl-encoder · zenrav1e · rav1d-safe · zenavif-parse · zenavif-serialize
Compression zenflate · zenzop · zenzstd
Processing zenresize · zenquant · zenblend · zenfilters · zensally · zentone
Pixels & color zenpixels · zenpixels-convert · linear-srgb · garb
Pipeline & framework zenpipe · zencodec · zencodecs · zenlayout · zennode · zenwasm · zentract
Metrics zensim · fast-ssim2 · butteraugli · zenmetrics · resamplescope-rs
Pickers & ML zenanalyze · zenpredict · zenpicker
Products Imageflow image engine (.NET · Node · Go) · Imageflow Server · ImageResizer (C#)

¹ pure-Rust, #![forbid(unsafe_code)] codecs, as of 2026

General Rust awesomeness

zenbench · archmage · magetypes · enough · whereat · cargo-copter

Open source · @imazen · @lilith · lib.rs/~lilith

About

Pure Rust Zstandard compression — forbid(unsafe_code), no_std+alloc, full levels 1-22, SIMD via archmage

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages