A Rust library for heterogeneous, typed key–value maps where each key holds exactly one value of a known type at a time. Values are stored in a compact 16-byte tagged representation; strings, byte slices, typed slices, large integers, and fixed-size hashes use an internal arena so reads return borrowed references (&str, &[u8], &[T], &[u8; N]) without per-lookup allocation.
Three map types share the same value encoding and getter API, but differ in how keys are represented:
| Map | Key type | Best when |
|---|---|---|
VarMap |
Precomputed Key (64-bit hash) |
Keys are known at compile time (var! macro) |
StrVarMap |
&str (FNV-1a hash at runtime) |
Keys come from user input or config at runtime |
EnumVarMap |
Enum variant (#[derive(EnumVarMap)]) |
Fixed, closed set of keys known at compile time |
VarMap is aimed at write once, read many workloads: you populate keys (configuration, request context, parsed attributes) and then read them repeatedly with cheap typed getters and borrowed &str / &[u8] / &[T] results.
The internal arena is append-only. Calling set on an existing key updates the map entry, but any previous arena allocation for that key is not reclaimed. Each overwrite of an arena-backed value (strings and byte slices longer than 14 bytes, typed slices, i128 / u128, Ipv6Addr, Duration, hashes, custom types, etc.) adds new arena space while the old payload remains until the whole map is reset. Scalars and short strings/bytes (≤14 bytes) live entirely in the fixed 16-byte cell, so overwriting those does not grow the arena. In-place update of an arena-backed value (including hashes) mutates the existing allocation and does not grow the arena.
| Pattern | Suitability |
|---|---|
| Set each key once, then many reads | Ideal |
| Occasional updates to a few keys | Fine |
| Frequent updates to the same keys with large / arena-backed values | Poor — arena memory grows; prefer clear() and repopulate, or another map |
| Hot loop re-assigning every key every tick | Poor — use clear() between logical snapshots, or avoid varmap |
Use clear() when you need a fresh binding of all keys (for example between requests or benchmark iterations). That resets the arena and removes current entries while preserving already allocated capacity for future writes. To recycle that capacity across many short-lived maps, use a pool.
All three maps are Sync. Mutation always takes &mut self, so shared & access is read-only and data-race-free. After you finish writing, call as_readonly() and copy the Readonly view to other threads. The view borrows the map (use std::thread::scope, not thread::spawn).
use std::thread;
use varmap::StrVarMap;
let mut map = StrVarMap::new();
map.set("port", 8080u16);
let ro = map.as_readonly();
thread::scope(|s| {
s.spawn(|| assert_eq!(ro.get_u16("port"), Some(8080)));
s.spawn(|| assert_eq!(ro.get_u16("port"), Some(8080)));
});When threads must write as well, consume the map with into_shared(). Clones share one Arc<RwLock<_>> and are 'static (so thread::spawn is fine). Use read() / write(); drop the guard to release the lock.
use std::thread;
use varmap::StrVarMap;
let mut map = StrVarMap::new();
map.set("count", 0u32);
let shared = map.into_shared();
thread::scope(|s| {
let a = shared.clone();
let b = shared.clone();
s.spawn(move || {
a.write().update::<u32>("count", |n| *n += 1);
});
s.spawn(move || {
b.write().update::<u32>("count", |n| *n += 1);
});
});
assert_eq!(shared.read().get_u32("count"), Some(2));Add to your Cargo.toml:
[dependencies]
varmap = "0.1"The proc-macros (var!, #[derive(EnumVarMap)], #[derive(VarMapValue)]) are re-exported from the varmap crate.
Built-in types (via set / get / typed getters):
- Integers:
i8…i64,u8…u64,i128,u128 - Floats:
f32,f64 bool,char&str,&[u8](stored in the arena; returned as borrows)- Typed slices:
&[bool],&[i8],&[i16],&[i32],&[i64],&[u16],&[u32],&[u64],&[f32],&[f64](always arena-backed; returned as borrows) IpAddr,Ipv4Addr,Ipv6Addrstd::time::Duration(stored in the arena, including subsecond nanos)- Fixed-size hashes:
[u8; 16],[u8; 20],[u8; 32],[u8; 48],[u8; 64](always arena-backed; returned as&[u8; N])
Strings and byte slices (&str, &[u8]) up to 14 bytes are stored inline in the map; larger payloads are arena-allocated. Typed slices are always arena-backed (no inline path), so they stay aligned for element types wider than a byte.
Hash128, Hash160, Hash256, Hash384, and Hash512 are type aliases for those arrays (Hash128 is [u8; 16], and so on). You can use either name:
use varmap::{Hash256, StrVarMap};
let mut map = StrVarMap::new();
let digest = [0xABu8; 32];
map.set("file.hash", digest);
assert_eq!(map.get::<[u8; 32]>("file.hash"), Some(&digest));
assert_eq!(map.get::<Hash256>("file.hash"), Some(&digest));
assert_eq!(map.get_hash256("file.hash"), Some(&digest));
assert!(map.update::<[u8; 32]>("file.hash", |h| h[0] = 0xFF));
assert_eq!(map.get_hash256("file.hash").unwrap()[0], 0xFF);A hash is distinct from a byte slice of the same length: get_bytes / get::<&[u8]> will not match a stored [u8; 32], and vice versa.
Typed slices are distinct from each other and from &[u8]:
use varmap::StrVarMap;
let mut map = StrVarMap::new();
map.set("samples", &[1u16, 2, 3][..]);
assert_eq!(map.get::<&[u16]>("samples"), Some(&[1, 2, 3][..]));
assert_eq!(map.get_u16_slice("samples"), Some(&[1, 2, 3][..]));
assert_eq!(map.get_bytes("samples"), None);
assert_eq!(map.get_u32_slice("samples"), None);Custom Copy types with alignment 1–16 can use #[derive(VarMapValue)] (see Custom types).
Use when variable names are ordinary strings at runtime (configuration keys, script variables, etc.). Keys are hashed with FNV-1a on each set / get.
use varmap::StrVarMap;
let mut map = StrVarMap::new();
map.set("port", 8080u16);
map.set("enabled", true);
map.set("host", "api.example.com");
assert_eq!(map.get_u16("port"), Some(8080));
assert_eq!(map.get_bool("enabled"), Some(true));
assert_eq!(map.get_str("host"), Some("api.example.com"));
assert_eq!(map.contains("port"), true);
// Generic get when the type is inferred or specified explicitly
let port: u16 = map.get("port").unwrap();All three maps provide the same convenience methods:
get_bool, get_u8, get_u16, get_u32, get_u64, get_i8, get_i16, get_i32, get_i64, get_f32, get_f64, get_str, get_bytes, get_bool_slice, get_i8_slice, get_i16_slice, get_i32_slice, get_i64_slice, get_u16_slice, get_u32_slice, get_u64_slice, get_f32_slice, get_f64_slice, get_char, get_ip, get_ipv4, get_ipv6, get_duration, get_hash128, get_hash160, get_hash256, get_hash384, get_hash512
Or use the generic API:
map.get::<u32>("user.age");
map.get::<&str>("user.name");
map.get::<&[u16]>("samples");
map.get::<std::time::Duration>("timeout");
map.get::<[u8; 32]>("file.hash");All three maps (VarMap, StrVarMap, and EnumVarMap) support in-place mutation through update.
This is useful for counters, hashes ([u8; N]), and custom Copy values that implement VarMapValue::update (the derive macro generates this automatically).
use varmap::{StrVarMap, VarMapValue};
#[derive(VarMapValue, Copy, Clone, Eq, PartialEq, Debug)]
struct Stats {
hits: u32,
misses: u32,
}
let mut map = StrVarMap::new();
map.set("count", 10u32);
map.set("stats", Stats { hits: 1, misses: 0 });
assert!(map.update::<u32>("count", |n| *n += 5));
assert!(map.update::<Stats>("stats", |s| s.hits += 1));
assert_eq!(map.get_u32("count"), Some(15));
assert_eq!(map.get::<Stats>("stats").map(|s| s.hits), Some(2));update returns false if the key is missing, the stored type does not match, or the type cannot be updated in place.
update_or_default is the same when the key already exists. If the key is missing, it inserts T::default() and returns true without calling the closure. T must implement Default.
use varmap::StrVarMap;
let mut map = StrVarMap::new();
assert!(map.update_or_default::<u32>("count", |_| {}));
assert_eq!(map.get_u32("count"), Some(0));
assert!(map.update_or_default::<u32>("count", |n| *n += 5));
assert_eq!(map.get_u32("count"), Some(5));map.clear(); // drop all keys and reset the arenaUse when key names are fixed at compile time. The var! macro expands to a Key holding the FNV-1a hash of the string literal, so lookups skip string hashing and comparison.
VarMap::new() is a const constructor, so an empty map can be created in const / static contexts:
use varmap::VarMap;
const EMPTY: VarMap = VarMap::new();use varmap::{VarMap, Key, var};
let mut map = VarMap::new();
// Compile-time keys (recommended)
map.set(var!("user.age"), 32u32);
map.set(var!("user.name"), "Alice");
assert_eq!(map.get_u32(var!("user.age")), Some(32));
assert_eq!(map.get_str(var!("user.name")), Some("Alice"));
assert!(map.update::<u32>(var!("user.age"), |age| *age += 1));
assert_eq!(map.get_u32(var!("user.age")), Some(33));
// Or supply a precomputed hash yourself
map.set(Key::new(0x0123_4567_89AB_CDEF), 1u8);StrVarMap is a thin wrapper around VarMap that hashes &str keys at runtime; VarMap + var! is the faster option when names are static.
VarMap (and therefore StrVarMap) supports at most 65 536 distinct keys (index stored in the low 16 bits of the internal hash entry).
Use when the key set is a closed enum known at compile time. Each variant maps to a fixed slot (one per enum discriminant), giving O(1) access with no hashing or binary search.
Derive EnumVarMap on your enum (requires #[repr(u16)], Copy, and at most 65 536 variants):
use varmap::EnumVarMap;
#[derive(EnumVarMap, Copy, Clone, Debug)]
#[repr(u16)]
enum ConfigKey {
Port,
Enabled,
Host,
}
let mut map = EnumVarMap::<ConfigKey>::new();
map.set(ConfigKey::Port, 8080u16);
map.set(ConfigKey::Enabled, true);
map.set(ConfigKey::Host, "api.example.com");
assert_eq!(map.get_u16(ConfigKey::Port), Some(8080));
assert_eq!(map.get_bool(ConfigKey::Enabled), Some(true));
assert_eq!(map.get_str(ConfigKey::Host), Some("api.example.com"));
assert_eq!(map.contains(ConfigKey::Port), true);EnumVarMap owns its own arena (unlike StrVarMap, which delegates to an inner VarMap). Memory for every enum variant slot is reserved up front, including variants that have not been set yet.
as_readonly() on VarMap, StrVarMap, or EnumVarMap returns a Readonly wrapper: Copy + Send + Sync, with Deref to the map so all getters work. There is no mutable access through the view. See Concurrent reads.
into_shared() consumes the map and returns a Shared<T> handle (Arc<RwLock<T>>). Cloning the handle shares the same instance; it does not clone the map. Call read() / write() (or try_read / try_write) from any thread. handle_count() is the number of live handles; try_unwrap() recovers the map when this is the last one.
Prefer Readonly when threads only need to read. Use Shared when they must also set / update. See Concurrent reads.
Each map type has a matching pool that checks out reusable maps. release clears the map but keeps heap and arena capacity, which is the intended reuse path when you would otherwise clear() and throw the map away (one map per request, per snapshot, per benchmark iteration).
| Pool | Map |
|---|---|
VarMapPool |
VarMap |
StrVarMapPool |
StrVarMap |
EnumVarMapPool |
EnumVarMap |
use varmap::{var, Key, VarMapPool};
let mut pool = VarMapPool::new();
let h = pool.allocate();
pool.get_mut(h).unwrap().set(var!("port"), 8080u16);
assert_eq!(pool.get(h).unwrap().get_u16(var!("port")), Some(8080));
pool.release(h);
assert!(pool.get(h).is_none());StrVarMapPool and EnumVarMapPool use the same checkout API:
use varmap::StrVarMapPool;
let mut pool = StrVarMapPool::new();
let h = pool.allocate();
pool.get_mut(h).unwrap().set("host", "localhost");
assert_eq!(pool.get(h).unwrap().get_str("host"), Some("localhost"));
pool.release(h);use varmap::{EnumVarMap, EnumVarMapKey, EnumVarMapPool};
#[derive(EnumVarMap, Copy, Clone, Debug)]
#[repr(u16)]
enum ConfigKey {
Port,
Host,
}
let mut pool = EnumVarMapPool::<ConfigKey>::new();
let h = pool.allocate();
pool.get_mut(h).unwrap().set(ConfigKey::Port, 8080u16);
assert_eq!(pool.get(h).unwrap().get_u16(ConfigKey::Port), Some(8080));
pool.release(h);allocate returns a PoolHandle. Handles are Copy; copies refer to the same slot. A handle is valid only for the pool that minted it, and only until that slot is released or the pool is cleared. Stale handles yield None / a no-op.
The handle type is shared by all three pools and is not tagged with the map type. Do not pass a handle from one pool into another: slot index and generation can coincide, so the other pool may treat it as live.
clear invalidates every outstanding handle. ClearStrategy chooses what to retain:
| Strategy | Effect |
|---|---|
FreeEntireMemory |
Drop every map and return memory to the allocator |
KeepSmallestN(n) |
Keep the n smallest maps by allocated size |
KeepNClosestToAverage(n) |
Keep the n maps closest to the mean allocated size |
KeepLargestN(n) |
Keep the n largest maps (best when you want to retain expensive arenas) |
n is a count of maps, not a byte budget, and is clamped to the current pool size. Kept maps are emptied; their capacity is kept for later allocates. Maps that are not kept are dropped.
use varmap::{ClearStrategy, VarMapPool};
let mut pool = VarMapPool::new();
let h = pool.allocate();
pool.release(h);
pool.clear(ClearStrategy::KeepLargestN(4));Types that are Copy, have alignment between 1 and 16 bytes, and contain only plain data can derive VarMapValue:
use varmap::{StrVarMap, VarMapValue};
#[derive(VarMapValue, Copy, Clone, Eq, PartialEq, Debug)]
struct Point {
x: i32,
y: i32,
}
let mut map = StrVarMap::new();
let p = Point { x: 1, y: 2 };
map.set("origin", p);
let back: &Point = map.get("origin").unwrap();
assert_eq!(*back, p);#[derive(VarMapValue)] also generates in-place update support, so custom values can be mutated with map.update::<T>(...) on all three map types.
Generics, unions, and non-Copy types are not supported by the derive macro.
compile-time keys?
│
┌────────────┴────────────┐
yes no
│ │
fixed enum? StrVarMap
│
┌────────┴────────┐
yes no
│ │
EnumVarMap VarMap + var!
EnumVarMap— fastest reads/writes when the key set is an enum; pays fixed storage for every variant slot.VarMap+var!— same value layer asStrVarMap, but no per-operation string hashing.StrVarMap— simplest API when keys are dynamic strings.
Benchmarks live in the banches crate. They reflect the intended read-heavy pattern: each run performs 10 000 outer iterations over 256 string keys. Update tests call clear() at the start of every iteration, then write all keys once (so arena growth from repeated overwrites is not measured). Read tests populate the map once, then only read in the timed loop — matching write-once, read-many usage.
Results below are averaged over repeated runs; columns are time, allocation during init, and allocation during the timed loop (via a tracking allocator).
| Implementation | Scenario | Time | Init alloc | Run alloc |
|---|---|---|---|---|
| EnumVarMap | Update – large strings | 23 ms | 4 112 B | 16 384 B |
| VarMap | Update – large strings | 74 ms | 0 B | 28 672 B |
| StrVarMap | Update – large strings | 90 ms | 0 B | 29 696 B |
| HashMap | Update – large strings | 175 ms | 0 B | 30 641 B |
| BTreeMap | Update – large strings | 258 ms | 0 B | 26 985 B |
| EnumVarMap | Update – small strings | 31 ms | 4 112 B | 0 B |
| VarMap | Update – small strings | 75 ms | 0 B | 12 288 B |
| StrVarMap | Update – small strings | 92 ms | 0 B | 12 288 B |
| HashMap | Update – small strings | 174 ms | 0 B | 22 026 B |
| BTreeMap | Update – small strings | 255 ms | 0 B | 18 370 B |
| EnumVarMap | Read – large strings | 16 ms | 20 496 B | 0 B |
| VarMap | Read – large strings | 59 ms | 28 672 B | 0 B |
| HashMap | Read – large strings | 58 ms | 30 641 B | 0 B |
| StrVarMap | Read – large strings | 85 ms | 28 672 B | 0 B |
| BTreeMap | Read – large strings | 93 ms | 26 985 B | 0 B |
| EnumVarMap | Read – small strings | 10 ms | 4 112 B | 0 B |
| VarMap | Read – small strings | 35 ms | 12 288 B | 0 B |
| HashMap | Read – small strings | 52 ms | 22 026 B | 0 B |
| StrVarMap | Read – small strings | 54 ms | 12 288 B | 0 B |
| BTreeMap | Read – small strings | 90 ms | 18 370 B | 0 B |
Large values are long descriptive strings (~50+ bytes); small values are short tokens (a few bytes), many of which use inline storage (≤14 bytes) instead of the arena.
- Read benchmarks are the sweet spot (populate once, read many):
EnumVarMapleads by a wide margin (e.g. 10 ms vs 35–54 ms for small strings on read). EnumVarMapis fastest overall: direct indexing avoids hash tables and, forVarMap/StrVarMap, sorted hash-vector lookup.VarMapbeatsStrVarMapby roughly 15–20% becausevar!removes runtime FNV-1a over key names.- All three varmap types beat
HashMapandBTreeMapon the measured workloads. Update numbers assumeclear()per iteration; sustained in-place overwrites withoutclear()would allocate more because of the append-only arena. EnumVarMaptrades memory for speed: ~4 KiB upfront per 256-key enum (empty slots), but zero run-time allocation on small-string read/update tests once initialized.- Inline small strings (≤14 bytes) avoid arena use:
EnumVarMapreports 0 bytes allocated during small-string update loops.
Reproduce locally from the workspace root:
cargo run -p banches -- RUN 10000 10Use cargo run -p banches -- LIST to see individual benchmark names and filter with an optional fourth argument (e.g. enumvarmap).
MIT — see repository and crate metadata for details.