BSI - columnar storage engine for Prometheus

BSI is a new storage engine for Prometheus, it replaces existing tsdb used by upstream prometheus with a compressed roaring bitmaps based storage that has the following core features.

• Low memory footprint
• Low disc storage footprint
• Fast and efficient
• Permanent storage
• Fully promql compliant
• Designed for high cardinality

With BSI there is no need for third party remote storage, a single prometheus instance can handle up to 1 million active timeseries without being OOM killed.

Below figure shows amount of ram after ingesting 108000000 rows with 10000 active timeseries.

Please run your own benchmarks with real data before reaching any conclusions

With BSI there is no big difference between a million active timeseries and one active timeseries in term of memory consumption.

Our goal is to eventually upstream the changes, however it is a massive task because the underlying Roaring Bitmap Format was extracted from FeatureBase. With enough interests and sufficient performance gain, I will consider taking on the challenge.

We port prometheus v3.7.1 main entrypoint to use BSI. This is strictly for benchmarking and evaluating only.

Some things removed includes

• tsdb: only bsi is supported
• agent mode: we do not generate prometheus wal files
• remote storage: bsi is optimized as a permanent storage, there is no point to send data to other nodes

go install github.com/gernest/bsi@latest

The bsi binary is the drop in replacement for prometheus v3.7.1.