Pco (Pcodec) losslessly compresses and decompresses numerical sequences with high compression ratio and moderately fast speed.

Quick Start

use pco::ChunkConfig;
use pco::errors::PcoResult;
use pco::standalone::{simple_compress, simple_decompress};

fn main() -> PcoResult<()> {
  // your data
  let mut my_nums = Vec::new();
  for i in 0..100000 {
    my_nums.push(i as i64);
  }

  // compress
  let compressed: Vec<u8> = simple_compress(&my_nums, &ChunkConfig::default())?;
  println!("compressed down to {} bytes", compressed.len());

  // decompress
  let recovered: Vec<i64> = simple_decompress(&compressed)?;
  assert_eq!(recovered, my_nums);
  Ok(())
}

Compilation Notes

For best performance on x86_64, compile with bmi1, bmi2, and avx2. This improves compression speed slightly and decompression speed substantially! Almost all hardware nowadays supports these instruction sets. To make sure you're using these, you can:

• Add the following to your ~/.cargo/config.toml:

[target.'cfg(target_arch = "x86_64")']
rustflags = ["-C", "target-feature=+bmi1,+bmi2,+avx2"]

• OR compile with RUSTFLAGS="-C target-feature=+bmi1,+bmi2,+avx2" cargo build --release ...

Note that setting target-cpu=native does not always have the same effect, since LLVM compiles for the lowest common denominator of instructions for a broad CPU family.