BitPolar

An independent open-source implementation of training-free vector quantization (TurboQuant · PolarQuant · QJL)

Compress embeddings to 3-8 bits with provably unbiased inner products and no calibration data. Implements TurboQuant (ICLR 2026), PolarQuant (AISTATS 2026), and QJL (AAAI 2025) from Google Research.

• Data-oblivious — no training, no codebooks, no calibration data
• Deterministic — fully defined by 4 integers: (dimension, bits, projections, seed)
• Provably unbiased — inner product estimates satisfy E[estimate] = exact at 3+ bits
• Near-optimal — distortion within ~2.7x of the Shannon rate-distortion limit
• Instant indexing — vectors compress on arrival, 600x faster than Product Quantization

• 58 integrations — every major AI framework, vector database, and ML library
• PyTorch torchao — embedding quantizer, BitPolarLinear, KV cache
• FAISS drop-in — API-compatible IndexBitPolarIP/L2 replacement
• LlamaIndex, Haystack, DSPy — VectorStore and Retriever integrations
• Agentic AI — LangGraph, CrewAI, OpenAI Agents, Google ADK, SmolAgents, PydanticAI
• Agent memory — Mem0, Zep, Letta backends
• 11 vector databases — Milvus, Weaviate, Pinecone, Redis, ES, DuckDB, SQLite, and more
• LLM inference — llama.cpp, SGLang, TensorRT, Ollama, MLX KV cache compression
• ML frameworks — JAX/Flax, TensorFlow/Keras, scikit-learn pipeline
• 30 Python examples covering all integrations
• Comprehensive benchmarks — throughput, recall, KV cache fidelity, FAISS comparison

• Walsh-Hadamard Transform — O(d log d) rotation with O(d) memory (577x less than Haar QR)
• Python bindings — PyO3 + maturin, zero-copy numpy integration
• WASM bindings — browser-side vector search via wasm-bindgen
• no_std support — embedded/edge deployment with alloc feature

[dependencies]
bitpolar = "0.3"

use bitpolar::TurboQuantizer;
use bitpolar::traits::VectorQuantizer;

// Create quantizer from 4 integers — no training needed
let q = TurboQuantizer::new(128, 4, 32, 42).unwrap();

// Encode a vector
let vector = vec![0.1_f32; 128];
let code = q.encode(&vector).unwrap();

// Estimate inner product without decompression
let query = vec![0.05_f32; 128];
let score = q.inner_product_estimate(&code, &query).unwrap();

// Decode back to approximate vector
let reconstructed = q.decode(&code);

pip install bitpolar

import numpy as np
import bitpolar

# Create quantizer — no training needed
q = bitpolar.TurboQuantizer(dim=768, bits=4, projections=192, seed=42)

# Encode/decode
embedding = np.random.randn(768).astype(np.float32)
code = q.encode(embedding)
decoded = q.decode(code)

# Build a search index
index = bitpolar.VectorIndex(dim=768, bits=4)
for i, vec in enumerate(embeddings):
    index.add(i, vec)

ids, scores = index.search(query, top_k=10)

import init, { WasmQuantizer, WasmVectorIndex } from 'bitpolar-wasm';

await init();

const q = new WasmQuantizer(128, 4, 32, 42n);
const code = q.encode(new Float32Array(128).fill(0.1));
const decoded = q.decode(code);

const index = new WasmVectorIndex(128, 4, 32, 42n);
index.add(0, vector);
const results = index.search(query, 5);

The WHT provides an O(d log d) alternative to Haar QR rotation:

use bitpolar::wht::WhtRotation;
use bitpolar::traits::RotationStrategy;

let wht = WhtRotation::new(768, 42).unwrap();
let rotated = wht.rotate(&embedding);
let recovered = wht.rotate_inverse(&rotated);

BitPolar is the single canonical library for vector quantization across the entire AI/ML ecosystem.

Input f32 vector
    │
    ▼
┌─────────────────┐
│ Random Rotation  │  WHT (O(d log d)) or Haar QR (O(d²))
│                  │  Spreads energy uniformly across coordinates
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   PolarQuant     │  Groups d dims into d/2 pairs → polar coords
│   (Stage 1)      │  Radii: lossless f32 │ Angles: b-bit quantized
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  QJL Residual    │  Sketches reconstruction error
│   (Stage 2)      │  1 sign bit per projection → unbiased correction
└────────┬────────┘
         │
         ▼
TurboCode { polar: PolarCode, residual: QjlSketch }

Inner product estimation: ⟨v, q⟩ ≈ IP_polar(code, q) + IP_qjl(sketch, q)

BitPolar works on embedded/edge targets with no_std:

[dependencies]
bitpolar = { version = "0.2", default-features = false, features = ["alloc"] }

Uses libm for math functions and alloc for Vec/String. The Walsh-Hadamard rotation is available without std (unlike Haar QR which requires nalgebra).

BitPolar exposes composable traits for ecosystem integration:

• VectorQuantizer — core encode/decode/IP/L2 interface
• BatchQuantizer — parallel batch operations (behind parallel feature)
• RotationStrategy — pluggable rotation (QR, Walsh-Hadamard, identity)
• SerializableCode — compact binary serialization

30 Python examples + 9 Rust examples + JavaScript, Go, Java examples.

# Rust
cargo run --example search_vector_database
cargo run --example llm_kv_cache

# Python (30 examples covering all 58 integrations)
python examples/python/01_quickstart.py           # Core API
python examples/python/12_pytorch_quantizer.py     # PyTorch integration
python examples/python/13_llamaindex_vectorstore.py # LlamaIndex
python examples/python/14_faiss_dropin.py          # FAISS replacement
python examples/python/18_openai_agents_tool.py    # OpenAI Agents
python examples/python/23_vector_databases.py      # DuckDB, SQLite, etc.
python examples/python/30_complete_rag.py          # End-to-end RAG pipeline

See examples/README.md for the full list.

Independently verified on standard datasets following MLCommons and ANN Benchmarks methodology. All results reproducible with seed=42. Full details in BENCHMARKS.md.

Single-vector compression speed (Rust native, 4-bit, Criterion.rs, 100 samples):

Dimension    Vectors/sec
   64    ████████████████████████████████████████  169,000
  128    █████████████                              40,000
  256    ████                                        9,400
  512    ██                                          2,500
 1024    █                                             585

Approximate inner product estimation on compressed codes (no decompression):

Dimension    Ops/sec
   64    ████████████████████████████████████████  204,000
  128    █████████████████                          45,700
  256    █████                                      10,900
  512    ██                                          2,580
 1024    █                                             614

Cosine similarity between exact and approximate attention output (32 heads, 512 seq_len, 128 head_dim):

Bits    Cosine Similarity
  6     ████████████████████████████████████████  0.9931
  4     █████████████████████████████████████     0.9206
  3     █████████████████████████████████         0.8143
        ├───────┼───────┼───────┼───────┼───────┤
        0.0    0.2    0.4    0.6    0.8    1.0

BitPolar guarantees provably unbiased inner product estimates. Verified on 10K random pairs:

Mean bias ≈ 0 at all bit-widths, confirming the theoretical guarantee from TurboQuant (ICLR 2026): E[estimate] = exact.

The compact format stores lossless f32 radii + quantized angles + QJL sketch. For KV cache applications, the paper's "6x compression" refers to bits-per-coordinate (angle quantization only).

# Rust benchmarks (recommended — native speed, Criterion.rs)
cargo bench --bench quantization_bench    # Encode/decode/IP latency
cargo bench --bench search_bench          # Search patterns + f32 baseline
cargo bench --bench dataset_bench         # Recall@10 + throughput scaling

# Python benchmarks (requires: cd bitpolar-python && maturin develop --release)
python benchmarks/bench_compression.py    # Compression ratio table
python benchmarks/bench_kv_cache.py       # Attention fidelity + IP bias
python benchmarks/bench_vs_faiss.py       # Head-to-head vs FAISS PQ/SQ

# Full results document
python benchmarks/generate_results.py     # → BENCHMARKS.md

Note on recall: Recall@10 on uniform random data is ~0.3-0.4 due to the curse of dimensionality (random vectors are nearly orthogonal). Real-world embeddings with semantic structure achieve significantly higher recall — the TurboQuant paper reports Recall@10 > 0.95 on GloVe at 4-bit.

• TurboQuant (ICLR 2026): arXiv 2504.19874
• PolarQuant (AISTATS 2026): arXiv 2502.02617
• QJL (AAAI 2025): arXiv 2406.03482

Contributions are welcome! See CONTRIBUTING.md for development setup, coding standards, and how to add a new quantization strategy.

Licensed under either of:

• MIT License (LICENSE-MIT)
• Apache License, Version 2.0 (LICENSE-APACHE)

at your option.