LokiKV is intended to be a simple to use in memory Key-Value Store that can also persist data on disk. This project is specifically for learning purposes only.

To try out loki-kv you can follow these steps:

Create a control file(toml):

host = "localhost"
port = 8765
last_wal_timeline = 0
last_checkpoint_id = 0
checkpoint_directory_path = "/home/akshat/lokikv/checkpoints_follower"
wal_directory_path = "/home/akshat/lokikv/wal_follower"
current_leader_value = 12 # optional
self_identifier = 13 # optional
send_addr = "0.0.0.0:8070"
consume_addr = "0.0.0.0:8071"
checkpoint_timer_interval = 1
paxos_timer_interval = 2
gossip_timeout = 300

git clone https://github.com/destrex271/LokiKV

# path to control file 
export CONTROL_FILE_PATH="/home/akshat/control_follow.toml"

cargo run --bin server-db # in a separate terminal
# runs on localhost:8765 by default

# in a separate terminal to start CLI
cargo run  --bin client -- localhost 8765

• Blob: <BLOB_BEGINS>data of blob<BLOB_ENDS>
• Integer
• Boolean
• Float
• String
• HyperLogLog: To estimate cardinality

• Set key values
• Get value for key
• Print all values in collection as a string
• Create multiple types of collections (\c_bcol, \c_hcol, \c_bcust)
• Select Collections
• List all available collections

• Overall Process

• LokiKV structure

• LokiQL Executor

LokiQL is a custom query language for interacting with the LokiKV database. This document describes the supported commands and their syntax.

• Create multiple Collections(similar to tables)
• Collections are of the following types:
  • Hashmap
  • BTreeMap
  • Custom BTree
• List collections
• Select one collection at a time

• Spaces, newlines, carriage returns, and tabs are ignored where applicable.

• Integer (INT): Signed or unsigned integer numbers.
• Float (FLOAT): Signed or unsigned floating point numbers.
• Boolean (BOOL): true or false.
• String (STRING): Enclosed in single quotes ('example').
• Blob (BLOB): Enclosed in <BLOB_BEGINS> and <BLOB_ENDS>.
• HyperLogLog(HLL): Init by ADDHLL command

• ID: Any string without whitespace.

LokiQL supports three types of commands:

SET mykey 'hello'
SET count 42
SET enabled true
SET temperature 98.6
SET file <BLOB_BEGINS>aGVsbG8=<BLOB_ENDS>

GET mykey
INCR count
DECR count
/selectcol users

DISPLAY
/getcur_colname
/listcolnames

A LokiQL command file follows this structure:

COMMAND
COMMAND
COMMAND
COMMAND; COMMAND; COMMAND;

SET mykey 'hello'
GET mykey
DISPLAY

Multiple command in a single line must be separated by ;. Single commands don't need to follow a ;

LokiKV currently uses a very adhoc version of the gossip protocol where we broadcast all the information across the network. All the nodes have varying values for paxos timer i.e. service discovery timer to facilitate for a smoother discovery flow and avoiding deadlocks(i.e. all the nodes begin consumption and broadcast at the same time).

• Add support for distributed setup via Paxos Algorithm