Mini Database 🚀

A high-performance graph database built in Rust with both embedded and network modes, featuring ultra-fast node/edge operations, advanced caching, comprehensive graph algorithms, and SQL-like join operations.

✨ Features

🏎️ High Performance

138,794 ops/sec node retrieval with LRU caching
19,503 ops/sec node creation with compression
10,492 ops/sec network throughput
100% cache hit rate for frequently accessed data

🌐 Dual Architecture

Embedded Mode: Direct in-process database for maximum performance
Network Mode: TCP client-server with connection pooling (port 5432)
Zero-copy operations with memory-mapped files

📊 Graph Database Features

Nodes & Edges with typed properties and metadata
Graph Algorithms: BFS, DFS, shortest path, connected components
Advanced Queries: Label-based, property-based, range queries
Fluent Query Builder with method chaining

🔗 SQL-Like Join Operations ✨ NEW

INNER JOIN: Edge-based and property-based joining
LEFT JOIN: Include all left records with NULL handling
RIGHT JOIN: Include all right records
FULL OUTER JOIN: Complete union of both sides
CROSS JOIN: Cartesian product with filtering
Aggregate Joins: SUM, AVG, COUNT, MAX, MIN with GROUP BY
WHERE Clauses: Rust closure-based filtering
ORDER BY & LIMIT: Sorting and pagination
Fluent Builder API: Chain operations with type safety

🔧 Production Ready

Connection pooling with idle timeout and cleanup
LZ4 compression for storage efficiency
Async/await architecture with Tokio
Custom binary protocol for network communication
Comprehensive error handling and logging

🔗 Join Operations Deep Dive

Comprehensive Join Support

Mini Database provides enterprise-grade join functionality that rivals traditional SQL databases while leveraging the power of graph relationships:

// All join types supported
let result = client.join("user", "order")
    .join_type(JoinType::Inner)           // INNER, LEFT, RIGHT, FullOuter, CROSS
    .on_edge("places_order".to_string())  // Edge-based joining
    .select(vec![
        ("user".to_string(), "name".to_string()),
        ("order".to_string(), "total".to_string()),
    ])
    .where_condition(|row| {              // Custom filtering
        if let Some(Value::Float(total)) = row.get("order.total") {
            *total > 100.0
        } else { false }
    })
    .order_by("order.total".to_string(), false)  // DESC ordering
    .limit(10)                            // Pagination
    .execute().await?;

result.print();  // Beautiful table output

Join Types & Performance

Join Type	Use Case	Performance	Graph Advantage
INNER JOIN	Matching records only	⚡ Very Fast	Direct edge traversal
LEFT JOIN	All left + matching right	🚀 Fast	NULL handling optimized
CROSS JOIN	All combinations	⚠️ Use with LIMIT	Efficient iteration
AGGREGATE	GROUP BY operations	🚀 Fast	Native graph grouping

Graph vs SQL Join Advantages

// Traditional SQL approach (slow)
SELECT u.name, o.total
FROM users u
INNER JOIN orders o ON u.id = o.user_id
WHERE o.total > 100;

// Mini Database approach (fast)
client.join("user", "order")
    .on_edge("places_order")              // Direct graph relationship
    .where_condition(|row| total > 100.0) // Rust closure filtering
    .execute().await?;

Why Graph Joins are Superior:

🔗 Direct relationship traversal (no expensive hash joins)
⚡ Sub-microsecond edge walking vs millisecond SQL joins
🎯 Type-safe filtering with Rust closures
📈 Linear scaling with relationship count

📦 Installation

Add to your Cargo.toml:

[dependencies]
mini-database = { git = "https://github.com/AarambhDevHub/mini-database.git" }
tokio = { version = "1.0", features = ["full"] }

🚀 Quick Start

Embedded Mode (Single Process)

use mini_database::{Database, DatabaseClient, DatabaseConfig, Node, Edge, Value};
use mini_database::{JoinType, AggregateFunction}; // Import join types

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create database
    let config = DatabaseConfig::new("./data")
        .with_cache_size(128) // 128MB cache
        .with_compression(true);

    let database = Database::new(config).await?;
    let client = DatabaseClient::new(database);

    // Create nodes
    let alice = Node::new("person")
        .with_property("name", Value::String("Alice".to_string()))
        .with_property("age", Value::Integer(30));

    let bob = Node::new("person")
        .with_property("name", Value::String("Bob".to_string()))
        .with_property("age", Value::Integer(25));

    let alice_id = client.create_node(alice).await?;
    let bob_id = client.create_node(bob).await?;

    // Create relationship
    let friendship = Edge::new(&alice_id, &bob_id, "friends")
        .with_property("since", Value::String("2020".to_string()));

    client.create_edge(friendship).await?;

    // Query with fluent API
    let people = client.execute_query(
        client.query_nodes()
            .with_label("person")
            .where_gt("age", Value::Integer(25))
            .limit(10)
    ).await?;

    // ✨ NEW: SQL-like JOIN operations
    let join_result = client.join("person", "person")
        .join_type(JoinType::Inner)
        .on_edge("friends".to_string())
        .select(vec![
            ("person".to_string(), "name".to_string()),
        ])
        .execute().await?;

    join_result.print(); // Beautiful table output

    // Graph traversal
    let connections = client.bfs(&alice_id, 3).await?;
    println!("Found {} connected nodes", connections.len());

    Ok(())
}

Network Mode (Client-Server)

Start the Database Server:

cargo run --example database_server

Connect from Client:

use mini_database::{NetworkDatabaseClient, Node, Value};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Connect to server
    let mut client = NetworkDatabaseClient::connect("127.0.0.1", 5432).await?;

    // Create node over network
    let node = Node::new("user")
        .with_property("email", Value::String("user@example.com".to_string()));

    let node_id = client.create_node(node).await?;

    // Retrieve node
    let retrieved = client.get_node(&node_id).await?;
    println!("Retrieved: {:?}", retrieved);

    // Graph traversal over network
    let neighbors = client.bfs(&node_id, 2).await?;

    Ok(())
}

🏗️ Architecture

┌─────────────────────────────────┐    ┌─────────────────────────────────┐
│          CLIENT MODE            │    │          SERVER MODE            │
├─────────────────────────────────┤    ├─────────────────────────────────┤
│                                 │    │  ┌─────────────────────────────┐ │
│  ┌─────────────────────────────┐ │    │  │     Network Clients         │ │
│  │     Application Code        │ │    │  │   (TCP Connections)         │ │
│  └─────────────────────────────┘ │    │  └─────────────────────────────┘ │
│  ┌─────────────────────────────┐ │    │  ┌─────────────────────────────┐ │
│  │    DatabaseClient API       │ │    │  │    Connection Pool          │ │
│  │     + Join Operations       │ │    │  │   + Request Handler         │ │
│  └─────────────────────────────┘ │    │  └─────────────────────────────┘ │
│                                 │    │                                 │
└─────────────────────────────────┘    └─────────────────────────────────┘
                │                                        │
                ▼                                        ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                           DATABASE ENGINE                                    │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐               │
│  │  Query Builder  │  │ Graph Operations │  │ Query Executor  │               │
│  │   - Fluent API  │  │   - BFS/DFS     │  │  - Optimization │               │
│  │   - Conditions  │  │ - Shortest Path │  │   - Execution   │               │
│  │   - JOIN Ops ✨ │  │ - Join Traversal│  │   - Join Engine │               │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘               │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐               │
│  │   Node Store    │  │   Edge Store    │  │   Index Store   │               │
│  │  - Serialization│  │ - Adjacency List│  │   - B-tree      │               │
│  │   - Properties  │  │  - Relationships│  │   - Fast Lookup │               │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘               │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐               │
│  │  LRU Cache      │  │  File Reader    │  │  Compression    │               │
│  │ - Hot Data      │  │ - Memory Map    │  │    - LZ4        │               │
│  │ - 100% Hit Rate │  │ - Async I/O     │  │ - Space Saving  │               │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘               │
└─────────────────────────────────────────────────────────────────────────────┘

🎯 Performance Benchmarks

Local Mode Performance

Operation          │    Count │ Duration │    Ops/Sec │  Latency
─────────────────────────────────────────────────────────────────
Node Creation      │   10,000 │   512ms  │    19,503  │   51μs
Node Retrieval     │   10,000 │    72ms  │   138,794  │    7μs
Node Updates       │    5,000 │   341ms  │    14,635  │   68μs
Edge Creation      │    1,000 │    58ms  │    16,965  │   59μs
Graph Traversal    │       10 │     2ms  │     4,958  │  202μs
Batch Operations   │      100 │     3ms  │    26,033  │   38μs

✨ Join Operations Performance NEW

Join Operation     │    Count │ Duration │    Ops/Sec │  Latency
─────────────────────────────────────────────────────────────────
INNER JOIN         │    1,000 │    45ms  │    22,222  │   45μs
LEFT JOIN          │    1,000 │    52ms  │    19,230  │   52μs
CROSS JOIN         │      100 │    15ms  │     6,666  │  150μs
Aggregate SUM      │      500 │    28ms  │    17,857  │   56μs
Complex Filter     │      750 │    35ms  │    21,428  │   47μs

Network Mode Performance

Operation          │    Count │ Duration │    Ops/Sec │  Latency
─────────────────────────────────────────────────────────────────
Node Creation      │    5,000 │   631ms  │     7,917  │  126μs
Node Retrieval     │    5,000 │   328ms  │    15,234  │   66μs
Graph Traversal    │        5 │     1ms  │    10,743  │   93μs
Network Latency    │      100 │     3ms  │    25,354  │   39μs

Overall Throughput: 10,492 ops/sec

Comparison with Production Databases

Database	Throughput	Use Case	Mini Database
Redis	200K ops/sec	In-memory cache	✅ 138K retrieval
MongoDB	50K ops/sec	Document store	✅ 19K creation
PostgreSQL	10K ops/sec	Relational DB	✅ 10K network
Neo4j	5K ops/sec	Graph database	✅ 5K traversal
Joins	2-5K ops/sec	SQL JOINs	✅ 22K joins

📚 API Documentation

Node Operations

// Create node with properties
let node = Node::new("label")
    .with_property("key", Value::String("value".to_string()))
    .with_property("count", Value::Integer(42));

let node_id = client.create_node(node).await?;

// Retrieve node
let node = client.get_node(&node_id).await?;

// Update node
if let Some(mut node) = client.get_node(&node_id).await? {
    node.set_property("updated", Value::Boolean(true));
    client.update_node(&node).await?;
}

// Delete node
client.delete_node(&node_id).await?;

// Find nodes by criteria
let nodes = client.find_nodes_by_label("person").await?;
let nodes = client.find_nodes_by_property("age", &Value::Integer(25)).await?;

Edge Operations

// Create edge
let edge = Edge::new(&source_id, &target_id, "relationship_type")
    .with_property("weight", Value::Float(0.8))
    .with_property("created_at", Value::String("2023-01-01".to_string()));

let edge_id = client.create_edge(edge).await?;

// Get node connections
let all_edges = client.get_node_edges(&node_id).await?;
let outgoing = client.get_outgoing_edges(&node_id).await?;
let incoming = client.get_incoming_edges(&node_id).await?;

✨ Join Operations NEW

use mini_database::{JoinType, AggregateFunction};

// INNER JOIN with edge relationships
let inner_result = client.join("user", "order")
    .join_type(JoinType::Inner)
    .on_edge("places_order".to_string())
    .select(vec![
        ("user".to_string(), "name".to_string()),
        ("order".to_string(), "total".to_string()),
    ])
    .execute().await?;

// LEFT JOIN with property matching
let left_result = client.join("user", "profile")
    .join_type(JoinType::Left)
    .on_property("id".to_string(), "user_id".to_string())
    .select(vec![
        ("user".to_string(), "name".to_string()),
        ("profile".to_string(), "bio".to_string()),
    ])
    .execute().await?;

// Complex JOIN with filtering and ordering
let complex_result = client.join("user", "order")
    .join_type(JoinType::Inner)
    .on_edge("places_order".to_string())
    .select(vec![
        ("user".to_string(), "name".to_string()),
        ("user".to_string(), "city".to_string()),
        ("order".to_string(), "total".to_string()),
    ])
    .where_condition(|row| {
        // Filter high-value orders
        if let Some(Value::Float(total)) = row.get("order.total") {
            *total > 100.0
        } else { false }
    })
    .where_condition(|row| {
        // Filter specific cities
        if let Some(Value::String(city)) = row.get("user.city") {
            city == "New York" || city == "San Francisco"
        } else { false }
    })
    .order_by("order.total".to_string(), false) // DESC
    .limit(20)
    .execute().await?;

complex_result.print(); // Beautiful table output

// Aggregate operations
let user_totals = client.aggregate_join(
    "user",
    "order",
    "places_order",
    "name",        // Group by user name
    "total",       // Sum order totals
    AggregateFunction::Sum,
).await?;

for (user, total_spent) in user_totals {
    println!("{}: ${:.2}", user, total_spent);
}

Graph Traversal

// Breadth-first search
let connected = client.bfs(&start_node_id, 3).await?;

// Depth-first search
let connected = client.dfs(&start_node_id, 3).await?;

// Shortest path
if let Some(path) = client.shortest_path(&start_id, &end_id).await? {
    println!("Path length: {}", path.len());
}

// Get neighbors within distance
let neighbors = client.get_neighbors(&node_id, 2).await?;

Query Builder

// Complex node queries
let results = client.execute_query(
    client.query_nodes()
        .with_label("person")
        .where_eq("department", Value::String("engineering".to_string()))
        .where_gt("salary", Value::Integer(100000))
        .where_contains("skills", "rust")
        .order_by_desc("salary")
        .limit(20)
        .offset(0)
).await?;

// Edge queries
let relationships = client.execute_query(
    client.query_edges()
        .with_label("friendship")
        .where_gt("strength", Value::Float(0.7))
        .order_by_asc("created_at")
).await?;

🛠️ Examples

Run Examples

# Local embedded usage
cargo run --example basic_usage
cargo run --example graph_example

# ✨ NEW: Join operation examples
cargo run --example comprehensive_joins
cargo run --example join_queries

# Network client-server mode
cargo run --example database_server    # Terminal 1
cargo run --example database_client    # Terminal 2

# Multi-database examples
cargo run --example two_databases
cargo run --example database_federation

# Performance benchmarks
cargo run --example benchmark_local --release
cargo run --example benchmark_network --release
cargo run --example benchmark_comparison --release

Project Structure

mini-database/
├── src/
│   ├── lib.rs                 # Main library entry
│   ├── core/                  # Database engine
│   │   ├── database.rs        # Main database instance
│   │   └── config.rs          # Configuration management
│   ├── storage/               # Storage layer
│   │   ├── file_reader.rs     # Memory-mapped I/O
│   │   ├── node_store.rs      # Node persistence
│   │   ├── edge_store.rs      # Edge & adjacency lists
│   │   ├── index.rs           # B-tree indexing
│   │   └── cache.rs           # LRU caching
│   ├── query/                 # Query processing
│   │   ├── builder.rs         # Fluent query API
│   │   ├── executor.rs        # Query execution
│   │   └── graph_ops.rs       # Graph algorithms
│   ├── client/                # Client interfaces
│   │   ├── database_client.rs # Local client
│   │   ├── network_client.rs  # Network client
│   │   ├── join.rs           # ✨ Join operations
│   │   └── result.rs          # Query results
│   ├── server/                # Network server
│   │   ├── server.rs          # TCP server
│   │   ├── handler.rs         # Request handling
│   │   ├── protocol.rs        # Binary protocol
│   │   └── connection_pool.rs # Connection management
│   ├── types/                 # Data structures
│   │   ├── node.rs            # Node definition
│   │   ├── edge.rs            # Edge definition
│   │   └── value.rs           # Property values
│   └── utils/                 # Utilities
│       ├── error.rs           # Error handling
│       └── serde.rs           # Serialization
├── examples/                  # Usage examples
│   ├── basic_usage.rs         # Basic operations
│   ├── graph_example.rs       # Graph algorithms
│   ├── database_server.rs     # Server startup
│   ├── database_client.rs     # Network client
│   ├── join_queries.rs        # ✨ Join examples
│   ├── two_databases.rs       # Multi-database
│   └── benchmark_*.rs         # Performance tests
└── Cargo.toml                 # Dependencies

⚙️ Configuration

Database Configuration

let config = DatabaseConfig::new("./data")
    .with_cache_size(256)        // 256MB cache
    .with_compression(true)      // Enable LZ4 compression
    .with_sync_writes(false)     // Async writes for performance
    .with_index_page_size(4096); // 4KB index pages

Server Configuration

let server_config = ServerConfig {
    host: "127.0.0.1".to_string(),
    port: 5432,
    max_connections: 100,        // Connection pool limit
    buffer_size: 8192,           // 8KB network buffer
    cleanup_interval: Duration::from_secs(60), // Connection cleanup
};

🔬 Use Cases

When to Use Local Mode

✅ Single-process applications (desktop, mobile, embedded)
✅ Ultra-low latency requirements (<10μs)
✅ High-frequency operations (>50K ops/sec)
✅ Offline-first applications

When to Use Network Mode

✅ Multi-client applications (web services, APIs)
✅ Microservices architecture
✅ Language-agnostic access (Python, JavaScript clients)
✅ Distributed deployments

✨ When to Use Join Operations NEW

✅ Complex relationship queries (user orders, social networks)
✅ Analytics and reporting (aggregations, grouping)
✅ Multi-table operations (traditional RDBMS migration)
✅ Real-time dashboards (fast aggregation queries)

📊 Optimization Tips

Performance Tuning

// Optimize for read-heavy workloads
let config = DatabaseConfig::new("./data")
    .with_cache_size(512)        // Larger cache
    .with_compression(false);    // Skip compression for speed

// Optimize for write-heavy workloads
let config = DatabaseConfig::new("./data")
    .with_cache_size(128)        // Smaller cache
    .with_compression(true)      // Save disk space
    .with_sync_writes(false);    // Async writes

Query Optimization

// Use specific queries for better performance
let nodes = client.find_nodes_by_label("person").await?;  // Fast label index
let nodes = client.find_nodes_by_property("age", &Value::Integer(25)).await?; // Property lookup

// Limit results to reduce memory usage
let results = client.execute_query(
    client.query_nodes()
        .with_label("user")
        .limit(1000)            // Limit results
        .offset(page * 1000)    // Pagination
).await?;

✨ Join Optimization NEW

// Use edge-based joins for better performance (faster than property joins)
let fast_join = client.join("user", "order")
    .on_edge("places_order")    // Direct edge traversal
    .execute().await?;

// Apply filters early to reduce intermediate results
let optimized = client.join("user", "order")
    .on_edge("places_order")
    .where_condition(|row| /* filter early */)  // Before sorting
    .order_by("order.total", false)
    .limit(100)                 // Limit final results
    .execute().await?;

🧪 Testing

# Run all tests
cargo test

# Run with optimizations
cargo test --release

# Test specific module
cargo test storage::tests

# ✨ Test join operations
cargo test client::join::tests

# Test with logging
RUST_LOG=debug cargo test

🚀 Roadmap

We are building Mini Database step-by-step to become a full-featured, production-quality graph database. Here is the planned progression:

Phase 1: Core Stability (Completed / In Progress)

Write-Ahead Logging (WAL) for crash recovery
ACID Transactions with commit/rollback
Custom Query Language parser and execution
Basic query optimization

Phase 2: Production Features (Upcoming)

Authentication and Role-Based Authorization
Security layer integrated into network server
Metrics collection using Prometheus support
Monitoring HTTP endpoint for real-time stats
(Optional) Admin dashboard with query console and metrics

Phase 3: Scalability & Distribution (Mid Term)

Cluster management with Raft consensus for high availability
Data sharding and replication across nodes
Inter-node communication and cluster network protocol
Distributed query execution and result aggregation
Graph algorithms library (PageRank, Community detection, etc.)
Temporal graph support with snapshotting and versioning

Phase 4: Ecosystem & Adoption (Long Term)

Multi-language drivers: Rust, Python, JavaScript, and more
ORM/ODM support with trait and macros
GraphQL API server integration for flexible querying
Connectors for Apache Kafka and Apache Spark for streaming and analytics
Comprehensive documentation, examples, tutorials
CI/CD pipelines, community engagement, and open source growth

Your contributions and feedback welcome! Help us build the future of graph databases in Rust.

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Development Setup

git clone https://github.com/AarambhDevHub/mini-database.git
cd mini-database
cargo build
cargo test
cargo run --example basic_usage

# ✨ Test join functionality
cargo run --example comprehensive_joins

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

☕ Support & Community

If you find Ignitia helpful, consider supporting the project:

🙏 Acknowledgments

Tokio - Asynchronous runtime
Serde - Serialization framework
LZ4 - High-speed compression
DashMap - Concurrent hash maps
Rust Community - Amazing ecosystem

📞 Support

Issues: GitHub Issues
Discussions: GitHub Discussions

Made with ❤️ and 🦀 Rust ❤️ by Aarambh Dev Hub

Mini Database - Where graph performance meets SQL familiarity

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
examples		examples
src		src
.gitignore		.gitignore
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Uh oh!

License

AarambhDevHub/mini-database

Folders and files

Latest commit

History

Repository files navigation

Mini Database 🚀

✨ Features

🏎️ High Performance

🌐 Dual Architecture

📊 Graph Database Features

🔗 SQL-Like Join Operations ✨ NEW

🔧 Production Ready

🔗 Join Operations Deep Dive

Comprehensive Join Support

Join Types & Performance

Graph vs SQL Join Advantages

📦 Installation

🚀 Quick Start

Embedded Mode (Single Process)

Network Mode (Client-Server)

🏗️ Architecture

🎯 Performance Benchmarks

Local Mode Performance

✨ Join Operations Performance NEW

Network Mode Performance

Comparison with Production Databases

📚 API Documentation

Node Operations

Edge Operations

✨ Join Operations NEW

Graph Traversal

Query Builder

🛠️ Examples

Run Examples

Project Structure

⚙️ Configuration

Database Configuration

Server Configuration

🔬 Use Cases

When to Use Local Mode

When to Use Network Mode

✨ When to Use Join Operations NEW

📊 Optimization Tips

Performance Tuning

Query Optimization

✨ Join Optimization NEW

🧪 Testing

🚀 Roadmap

Phase 1: Core Stability (Completed / In Progress)

Phase 2: Production Features (Upcoming)

Phase 3: Scalability & Distribution (Mid Term)

Phase 4: Ecosystem & Adoption (Long Term)

🤝 Contributing

Development Setup

📄 License

☕ Support & Community

🙏 Acknowledgments

📞 Support

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Sponsor this project

Uh oh!

Packages 0

Languages

Packages