A distributed, multi-tenant key-value & object store written in Rust

minikv provides strong consistency (Raft + 2PC), durability (WAL), and production-grade observability, security, and multi-tenancy — all in a modern Rust codebase.

Built in public as a learning-by-doing project — now evolved into a complete, reference implementation of distributed systems in Rust.

minikv v0.8.0 brings enterprise-grade features for distributed deployments:

• Cross-datacenter replication: Async replication with multiple conflict resolution strategies (LWW, Vector Clocks)
• Change Data Capture (CDC): Real-time event streaming to Webhook, Kafka, or file sinks
• Admin Web UI: Embedded dashboard for cluster monitoring and management
• Backup & Restore: Full and incremental backups with encryption support
• Plugin system: Extensible architecture for custom storage, auth, and hooks

Previous highlights (v0.7.0): secondary indexes, multi-key transactions, batch import/export, durable S3 backend.

• What is minikv?
• Tech Stack
• Quick Start
• Architecture
• Performance
• Features
• Roadmap
• Story
• Documentation
• Development
• Contributing
• Contact

minikv is a distributed key-value store written in Rust, designed for simplicity, speed, and reliability.

Who is this for ?
minikv is for engineers learning distributed systems, teams experimenting with Rust-based infrastructure, and anyone curious about consensus, durability, and system trade-offs.

• Clustered : Raft consensus and 2PC for transactional writes
• Virtual Sharding : 256 vshards for elastic scaling & balancing
• WAL : Write-ahead log for durability
• gRPC for node communication, HTTP REST & S3 API for clients
• Bloom filters, snapshots, watch/subscribe for performance & reactivity

• Rust – core logic
• Shell – orchestration/automation
• JavaScript – benchmarks, tools
• Makefile – build flows

git clone https://github.com/whispem/minikv.git
cd minikv
cargo build --release

# Start a node
cargo run -- --config config.example.toml

# API examples
curl localhost:8080/health/ready   # readiness
curl localhost:8080/metrics        # Prometheus metrics
curl localhost:8080/admin/status   # admin dashboard

# Create API key (admin)
curl -X POST http://localhost:8080/admin/keys -d '{"role":"ReadWrite","tenant_id":"acme"}'

# S3 (demo)
curl -X PUT localhost:8080/s3/mybucket/mykey -d 'hello minikv!'
curl localhost:8080/s3/mybucket/mykey

For cluster setup and advanced options, see the documentation.

• Raft: consensus and leader election
• 2PC: atomic distributed/batch writes
• Virtual Shards: scale and rebalance across 256 partitions
• Pluggable Storage: in-memory, RocksDB, Sled
• Admin API: HTTP endpoints for status, metrics and config
• Config: via environment, file or CLI flags

• Write throughput : over 50,000 operations/sec (single node, in-memory)
• Sub-millisecond read latency
• Cluster tested (3–5 nodes, commodity VMs)
• Built-in Prometheus metrics

• Raft consensus (multi-node, strong consistency)
• Two-phase commit (2PC) for atomic multi-key transactions
• 256 virtual shards for cluster scaling and rebalancing
• Write-ahead log (WAL) for durability
• Auto-rebalancing, graceful leader failover, hot-join and node removal

• Time-To-Live keys (TTL)
• LZ4 compression (configurable)
• Bloom filters and index snapshots
• Pluggable and persistent storage: in-memory, RocksDB, Sled
• Batch & range operations, prefix queries

• HTTP REST (CRUD, batch, range, admin)
• S3-compatible API (with TTL extensions)
• gRPC (internal)
• WebSocket and SSE endpoints for real-time watch/subscribe events

• API keys (Argon2) and JWT authentication
• Role-based access control (RBAC) and audit logging
• Multi-tenant isolation
• AES-256-GCM encryption at rest
• Per-tenant quotas (storage, requests, rate limits)
• TLS (HTTP & gRPC)

• Admin dashboard
• Prometheus metrics (counters, histograms)
• Request and endpoint statistics
• Structured logging and tracing spans
• Kubernetes health probes

• Memory-safe Rust
• Test suite, automated CI
• Documentation and sample config
• Single static binary

• Cross-datacenter replication
• Change Data Capture (CDC)
• Admin Web UI
• Backup & Restore
• Plugin system

• Secondary indexes
• Multi-key transactions
• Durable S3-backed object store
• Batch import/export

• Kubernetes Operator
• GraphQL API
• Time-series optimizations
• Geo-partitioning

minikv started as a 24-hour challenge by a Rust learner (82 days into the language!). It now serves as both a playground and a reference for distributed systems, demonstrating curiosity, learning-by-doing, and robust engineering.

• Example config : config.example.toml
• Cluster, API, usage : see docs/
• Certificate generation : certs/README.md

cargo test           # Run all tests
cargo clippy --fix   # Lint and fix
cargo fmt            # Format code

Continuous Integration runs on push & PR via .github/workflows/ci.yml.

Issues and PRs welcome! See CONTRIBUTING.md.

• GitHub: whispem/minikv
• Email: via GitHub profile