Mutable local test-data pools, leases, and response capture for k6 — without Redis.
If you only need read-only fixtures, use k6 SharedArray. If you need mutable local allocation, leases, or response capture, use xk6-kv.
Use xk6-kv when your k6 test needs writable shared state inside one k6 process:
- give each VU a unique user or credential without races;
- claim or pop rows from a local pool (CSV/JSONL import,
setMany, or runtime writes); - build task queues and producer/consumer flows;
- share mutable state between scenarios;
- export captured data after a run (
exportJSONL,exportCSV, disk backend, snapshots).
SharedArray is read-only after initialization and is not meant for communication between VUs. xk6-kv adds a process-local, concurrency-safe store with:
- lease-based allocation:
claimRandom,claimKey,claimKeys,claimRandomMany; - one-shot drains:
popRandom,popRandomMany; - streaming seed import:
importCSV,importJSONL; - portable export:
exportJSONL,exportCSV,backup,restore; - CAS-style operations (
compareAndSwap,incrementBy, …) and operation metrics.
| Need | k6 built-in | xk6-kv |
|---|---|---|
| Read-only shared fixtures | SharedArray |
yes (import once, read many) |
| Mutable shared state across VUs | no | yes (memory or disk backend) |
| Unique credential / user allocation | manual indexing / external store | claimRandom, claimKey, claimKeys, claimRandomMany |
| One-shot task queue (consume once) | manual | popRandom, popRandomMany |
| Large CSV/JSONL seed files | load in setup yourself |
importCSV, importJSONL |
| Response capture to file | handleSummary / custom scripts |
exportJSONL, exportCSV, backup / restore |
| Local mutable coordination without Redis | no | yes |
- You need coordination across multiple k6 processes or machines (use Redis, a queue, or your orchestrator).
- You need Grafana Cloud to resolve the extension automatically without a custom binary (Extensions Registry listing helps discovery; you still build
xk6or use release binaries). - You need a production database or distributed locks — claim APIs are local coordination primitives inside one k6 process, not a lock service.
- You cannot ship a custom k6 binary (
xk6 build --with …or pre-built releases).
Use Redis (or similar) when VUs across multiple k6 processes or machines must share the same pool or queue.
Use xk6-kv when you only need mutable state inside one k6 process and want fewer moving parts: no extra service, no network hop, claims/pops coordinated in-process.
From the repo root, with a k6 binary that includes this extension:
# Lease-based unique allocation (complete on success, release on failure)
k6 run examples/claim-random-default-ttl.js
# One-time user pool drain (no duplicate allocation)
k6 run examples/pop-random-unique-users.js
# CSV seed import
k6 run examples/import-csv.js
# Export a prefix to JSONL after writes
k6 run examples/export-jsonl.js
# Export a prefix to flat CSV after writes
k6 run examples/export-csv.jsProduction-style scenarios (metrics, thresholds, disk paths): e2e/ — e.g. e2e/import-csv-portable-seed.js, e2e/credential-pool-drain-observability.js. Full script index: examples/README.md.
This project is GNU AGPL v3.0 (fork of oleiade/xk6-kv, extended with claims, batch allocation, CSV/JSONL, metrics, and more). AGPL is acceptable for many test-tooling workflows, but some companies block AGPL dependencies in their supply chain. If you need a permissive license for a specific use case, open an issue with context — relicensing requires rights on the full codebase.
- Unique allocation:
claimRandom,claimKey,claimKeys,claimRandomMany,renewClaim,releaseClaim,completeClaim,renewClaims,releaseClaims,completeClaims. - One-shot queues:
popRandom,popRandomMany(each successful pop removes the key). - Large seed files: streaming
importCSVandimportJSONL. - Response capture & handoff:
exportJSONL,exportCSV,validateCSV,validateJSONL,backup/restore, disk backend. - Batch KV:
setMany,getMany,deleteMany. - Prefix workflows:
scan,scanKeys,list,listKeys,deleteByPrefix,count,randomKey,randomKeys. - CAS-style ops:
incrementBy,getOrSet,swap,compareAndSwap,compareAndDelete, and related helpers. - Observability: optional operation metrics,
stats(),allocationStats(),reportStats(). - Backends: memory (sharded, fast) or disk/bbolt (persistent); optional
trackKeysfor faster random/claim paths on large datasets. - TypeScript: declarations in
typescript/for IntelliSense.
xk6-kv stays a local k6 test-data lifecycle engine. The following are intentionally out of scope:
- no Redis backend;
- no distributed locks;
- no general key TTL (outside claim leases);
- no general transactions;
- no ETL transforms/schema inference/compression pipelines;
- no
feedNextin this release.
Unique allocation with claimRandom (full script: examples/claim-random-default-ttl.js):
import { openKv } from "k6/x/kv";
const kv = openKv({ backend: "memory", trackKeys: true });
export async function setup() {
await kv.setMany({
"user:1": { name: "Alice" },
"user:2": { name: "Bob" },
});
}
export default async function () {
const claim = await kv.claimRandom({ prefix: "user:" });
if (claim === null) return;
try {
console.log(claim.entry.value.name);
await kv.completeClaim(claim, { deleteKey: false });
} catch (err) {
await kv.releaseClaim(claim);
throw err;
}
}Build or download a k6 binary with this extension — see Installation below.
Download k6 binaries with xk6-kv from the Releases page.
Release artifacts are named xk6-kv_<version>_<os>_<arch>.tar.gz for Linux/macOS and
xk6-kv_<version>_<os>_<arch>.zip for Windows. Set VERSION to the release you want
to install (replace with the latest release), for example <XK6_KV_VERSION>.
Linux:
VERSION=<XK6_KV_VERSION>
curl -L "https://github.com/oshokin/xk6-kv/releases/download/${VERSION}/xk6-kv_${VERSION}_linux_amd64.tar.gz" -o k6.tar.gz
tar -xzf k6.tar.gz && chmod +x k6
./k6 versionmacOS:
VERSION=<XK6_KV_VERSION>
curl -L "https://github.com/oshokin/xk6-kv/releases/download/${VERSION}/xk6-kv_${VERSION}_darwin_arm64.tar.gz" -o k6.tar.gz
tar -xzf k6.tar.gz && chmod +x k6
./k6 versionWindows (PowerShell):
$VERSION = "<XK6_KV_VERSION>"
Invoke-WebRequest -Uri "https://github.com/oshokin/xk6-kv/releases/download/$VERSION/xk6-kv_${VERSION}_windows_amd64.zip" -OutFile k6.zip
Expand-Archive -Path k6.zip -DestinationPath .
.\k6.exe version- Install xk6:
go install go.k6.io/xk6/cmd/xk6@v1.4.1- Build k6 with xk6-kv:
# Latest version
xk6 build --with github.com/oshokin/xk6-kv@latest
# Pin a specific release
xk6 build --with github.com/oshokin/xk6-kv@<XK6_KV_VERSION>
# k6 v1.7.x legacy line (frozen at v1.4.31)
xk6 build --with github.com/oshokin/xk6-kv@v1.4.31- Verify:
./k6 versionRequirements: Go 1.25.11 or higher.
Current development targets k6 v2.0.x (go.k6.io/k6/v2). The JavaScript import is unchanged:
import { openKv } from "k6/x/kv";Use xk6 v1.4.1 or newer when building this extension from source; xk6 resolves the required k6 major version from the extension dependency graph.
| xk6-kv version | k6 core version | Notes |
|---|---|---|
| v1.5.0+ | v2.0.x | Current supported line |
| v1.4.31 | v1.7.x | Frozen legacy line; pin this tag when building against k6 v1 |
Projects that still require k6 v1.7.x should pin the extension explicitly:
xk6 build --with github.com/oshokin/xk6-kv@v1.4.31Allocation flows: 60-second example and Start here.
import { openKv } from "k6/x/kv";
const kv = openKv(); // Default: disk backend
export async function setup() {
await kv.clear();
}
export default async function () {
await kv.set("foo", "bar");
await kv.set("user:1", { name: "Alice" });
const key = await kv.randomKey({ prefix: "user:" });
if (key) {
console.log(`Random user: ${key}`);
}
const entries = await kv.list({ prefix: "user:" });
console.log(`Found ${entries.length} users`);
}
export async function teardown() {
kv.close();
}Async note: every
kv.*helper returns a Promise. In k6 scripts you should mark yoursetup,default, or helper functions asasync(or use.then) andawaiteach call—otherwise errors are swallowed and the operation may never finish.Concurrency note: avoid unbounded
Promise.all()with very large KV batches (for example, tens of thousands ofkv.set()calls at once). PrefersetMany()for object-map writes, sequentialawait, or bounded concurrency.
Bounded concurrency helper:
async function mapLimit(items, limit, fn) {
const width = Math.max(1, Math.min(limit, items.length || 1));
const workers = Array.from({ length: width }, async (_, worker) => {
for (let i = worker; i < items.length; i += width) {
await fn(items[i], i);
}
});
await Promise.all(workers);
}Every rejected kv.* promise carries a typed error object with name and message fields. It is a structured plain object, not a JavaScript Error instance, so do not rely on err instanceof Error. Check err.name (or err.Name when k6 serialises it as Go struct) instead of matching raw strings.
Batch APIs such as setMany() may also include a stable err.errors array. Each item in err.errors has this shape:
{
key?: string
name: string
message: string
}General error handling example:
try {
await kv.backup({ fileName: "./snapshots/run.kv" });
} catch (err) {
if (err?.name === "BackupInProgressError") {
console.log("Another VU is already writing a snapshot—safe to ignore.");
} else if (err?.name === "SnapshotPermissionError") {
fail(`Backup path is not writable: ${err.message}`);
} else {
throw err;
}
}Batch error details example:
try {
await kv.setMany({
"ok": { name: "Alice" },
"bad": () => {}, // JSON serializer rejects functions
});
} catch (err) {
if (err?.name === "InvalidOptionsError") {
for (const item of err.errors ?? []) {
console.log(item.key, item.name, item.message);
}
} else {
throw err;
}
}High-level categories:
- Options & inputs - typed guards such as
BackupOptionsRequiredError,ValueNumberRequiredError,UnsupportedValueTypeError. - Concurrency & lifecycle - e.g.
BackupInProgressError,RestoreInProgressError,OperationCanceledError,StoreReadOnlyError,StoreClosedError. - Disk & snapshot IO - precise signals for path issues, permission problems, bbolt failures, or restore budget overruns.
OperationCanceledError is returned when the owning k6 VU context is canceled or reaches its deadline while an async operation is running.
Typical causes:
- test abort;
- scenario stop;
- iteration timeout;
- Ctrl+C;
- k6 engine shutdown.
This is a control-flow error, not data corruption. Long-running operations such as importJSONL, exportJSONL, exportCSV, importCSV, validateCSV, validateJSONL, backup, and restore may stop early. For imports, already committed batches may remain committed.
📚 A complete catalogue with root causes and remediation tips lives in examples/README.md.
Full TypeScript support with IntelliSense and type safety! Copy the typescript/ folder to your project for a ready-to-use starter kit.
See typescript/README.md for complete setup instructions.
typescript/ is a local starter example project (template), not a published npm package.
| Area | Methods |
|---|---|
| Allocation | claimRandom, claimKey, claimKeys, claimRandomMany, releaseClaim, releaseClaims, renewClaim, renewClaims, completeClaim, completeClaims, popRandom, popRandomMany |
| Import / export | importCSV, importJSONL, exportJSONL, exportCSV, validateCSV, validateJSONL, backup, restore, rebuildKeyList |
| Batch | setMany, getMany, deleteMany |
| Query | get, set, delete, exists, list, listKeys, scan, scanKeys, count, size, randomKey, randomKeys, clear, close |
| CAS-style | incrementBy, getOrSet, swap, compareAndSwap, setIfAbsent, deleteIfExists, compareAndDelete |
| Observability | stats, allocationStats, reportStats (+ optional operation metrics via openKv({ metrics: … })) |
Full signatures, options, and error shapes: API Reference below. Scenario recipes: Usage Examples and examples/README.md.
Opens a key-value store. Must be called in the init context (outside of default/setup/teardown functions). The returned KV handle is shared across VUs, so treat its lifecycle as test-wide.
⚠️ Configuration lock-in: the first successfulopenKv()call fixes the store configuration for the whole test run. All subsequentopenKv()calls must provide equivalent options; otherwiseopenKv()throwsKVOptionsConflictError. Callingclose()does not reset this process-wide configuration.
interface OpenKvOptions {
backend?: "memory" | "disk" // default: "disk"
path?: string // default: "./.k6.kv" (disk only)
serialization?: "json" | "string" // default: "json"
trackKeys?: boolean // default: false
memory?: {
shardCount?: number // default: 0 (auto-detect, memory only)
// when omitted: defaults are applied
}
disk?: {
timeout?: number | string // wait for file lock; number=ms, string=Go duration (e.g. "1s"); default 1s
noSync?: boolean // disable fsync on commit; default false
noGrowSync?: boolean // skip fsync on growth; default false
noFreelistSync?: boolean // rebuild freelist on open; default false
preLoadFreelist?: boolean // load freelist into memory; default false
freelistType?: "" | "array" | "map" // freelist representation; default "array"
readOnly?: boolean // open DB read-only; mutating APIs reject with StoreReadOnlyError (requires pre-existing DB/bucket); default false
initialMmapSize?: number | string // initial mmap size; number=bytes, string supports SI ("MB") and IEC ("MiB"); 0 keeps default/no preallocation (default)
mlock?: boolean // mlock pages (UNIX); default false
// when omitted: bbolt defaults are applied
}
metrics?: {
operations?: boolean // default: false
}
}Options:
backend:"memory"(ephemeral, fastest) or"disk"(persistent bbolt)serialization:"json"(structured) or"string"(string/raw bytes)trackKeys: Enable in-memory key indexing for fasterrandomKey()/randomKeys()selection (see Performance & Complexity)path: (Disk only) Override bbolt file locationmemory.shardCount: (Memory only) Number of shards for concurrent performance. If<= 0or omitted, defaults toruntime.NumCPU()(automatic, recommended). If> 65536, automatically capped at 65536. Ignored by disk backend. Whenmemoryis omitted, defaults are applied.disk: (Disk only) Optional bbolt tuning. Whendiskis omitted, bbolt defaults apply (1s lock timeout, syncs enabled, array freelist, etc.).disk.readOnly: Requires the bbolt file (andk6bucket) to already exist; opening in read-only mode cannot create the bucket and will fail if the file is missing or empty.metrics.operations: Enables automatic per-method metrics (xk6_kv_operations_total,xk6_kv_operation_duration,xk6_kv_operation_failed,xk6_kv_errors_total,xk6_kv_empty_result).
Note: With serialization: "string", string values are stored as-is. Non-string values are converted with Go fmt %v formatting (for example, an object can become map[a:1]). This mode is not JSON and is not intended for structured value round-trips; use serialization: "json" for objects/arrays.
const kv = openKv({ serialization: 'string' });
await kv.set('x', { a: 1 });
console.log(await kv.get('x')); // Go-style string, not { a: 1 }
⚠️ Snapshot path sharing: If you omitbackup().fileNameorrestore().fileName, the memory backend deliberately falls back to the same.k6.kvfile the disk backend uses. This lets you run ultra-fast tests withbackend: "memory"and then immediately replay the generated dataset viabackend: "disk"without touching paths. If you don't want that coupling (for example, you run disk workloads concurrently), always pass an explicitfileName.
Memory Backend Sharding:
The memory backend shards data across multiple internal partitions to improve concurrent performance by reducing lock contention:
- Automatic (recommended): Set
memory.shardCount: 0(or omit) to auto-detect based on CPU count (e.g., 32 shards on a 32-core system) - Manual: Set
memory.shardCountto a specific value (1-65536) for fine-tuned control - Performance: On high-core systems sharding delivers:
- 3.5x faster
set()operations - 2x faster
get()operations
- 3.5x faster
- How it works: Keys are distributed across shards using a hash function, allowing concurrent operations on different shards to proceed in parallel
- Maximum: Shard count is capped at 65536 (2^16) to provide excellent hash distribution while keeping memory overhead minimal (~5MB for empty shard structures)
- Memory-only: Sharding applies only to the
"memory"backend; disk backend uses bbolt's transaction-based concurrency
All methods return Promises except close().
get(key: string): Promise<any>- Retrieves a value by key. Throws if key doesn't exist.getMany(keys: string[]): Promise<Array<{ key: string, exists: boolean, value: any | null }>>- Reads many keys in one logical batch and preserves input order. Missing keys return{ exists: false, value: null }; stored JSONnullreturns{ exists: true, value: null }; duplicate keys are allowed. Empty-string keys are accepted for reads and resolve like missing keys in normal public API usage.getMany()backend note: disk reads run inside one bbolt read transaction; memory reads use per-shard locks and do not provide cross-key snapshot isolation under concurrent writes. Mutating APIs (set,setMany,delete,deleteMany) reject empty-string keys.set(key: string, value: any): Promise<any>- Sets a key-value pair. Empty-string keys are rejected.setMany(entries: Record<string, any>): Promise<{ written: number }>- Writes an object map in one logical batch. Keys must be non-empty strings. Validates the input shape and serializes all values before writing; rejects witherr.errorsand writes nothing if any entry fails.setMany()provides all-or-nothing validation/serialization semantics, but is not intended to provide cross-key snapshot isolation for concurrent readers on the memory backend.deleteMany(keys: string[]): Promise<{ deleted: number, missing: number }>- Deletes an explicit list of non-empty keys. Missing keys are not errors and are counted inmissing; duplicate keys are processed in input order. Rejects invalid input before deleting anything.deleteByPrefix(options: { prefix: string, limit: number }): Promise<{ deleted: number, done: boolean }>- Deletes up tolimitkeys matching a non-empty prefix. This operation is destructive and bounded by explicit limit.delete(key: string): Promise<boolean>- Removes a key-value pair (always resolves totrue).exists(key: string): Promise<boolean>- Checks if a key exists.clear(): Promise<boolean>- Removes all entries (always resolves totrue).clear()is destructive and best used in setup/teardown or maintenance windows. Under concurrent writers, new keys can appear immediately after the wipe completes. WithtrackKeys: true, in-memory indexes are reset together with store state.size(): Promise<number>- Returns current store size (number of keys).
getMany() example:
const items = await kv.getMany(["user:1", "user:missing", "user:null"]);
// items[0] -> { key: "user:1", exists: true, value: ... }
// items[1] -> { key: "user:missing", exists: false, value: null }
// items[2] -> { key: "user:null", exists: true, value: null }deleteMany() example:
const result = await kv.deleteMany(["user:1", "user:2", "user:missing"]);
// result -> { deleted: 2, missing: 1 }deleteByPrefix() example:
const result = await kv.deleteByPrefix({
prefix: "tmp:",
limit: 1000,
});
// result -> { deleted: <number>, done: <boolean> }deleteByPrefix() details:
prefixis required and must be a non-empty string.limitis required, must be a positive integer, and is capped at100000.done === truemeans no matching keys remain after this call.- If
done === false, repeat the same call until completion. - Use
listKeys({ prefix, limit })first for a read-only preview.
Backend note:
- disk backend deletes inside one bbolt write transaction;
- memory backend deletes through shard locks and does not provide cross-key visibility transaction under concurrent writes;
- claim metadata is removed for physically deleted keys.
incrementBy(key: string, delta: number): Promise<number>- Atomically increments numeric value. Treats missing keys as0.JS numbers are IEEE‑754 doubles, so anything above
Number.MAX_SAFE_INTEGER(~9e15) loses precision before the value reaches Go. Keep counters below that threshold or encode larger values as strings/custom objects in your script before callingincrementBy.getOrSet(key: string, value: any): Promise<{ value: any, loaded: boolean }>- Gets existing value or sets if absent.loaded: truemeans pre-existing.swap(key: string, value: any): Promise<{ previous: any|null, loaded: boolean }>- Replaces value atomically. Returns previous value if existed.compareAndSwap(key: string, oldValue: any, newValue: any): Promise<boolean>- SetsnewValueonly if current value equalsoldValue. Passnull/undefinedasoldValueto mean "only if the key is absent" (set-if-not-exists).compareAndSwapDetailed(key: string, oldValue: any, newValue: any, options?: { includeCurrentOnMismatch?: boolean }): Promise<{ swapped: true, reason: "swapped" } | { swapped: false, reason: "mismatch", existed: boolean, current?: any }>- Detailed CAS diagnostics. On mismatch, includesexistedand optionallycurrent(only whenincludeCurrentOnMismatch: trueand key existed).setIfAbsent(key: string, value: any): Promise<boolean>- Convenience API for first-writer-wins key initialization. Equivalent tocompareAndSwap(key, null, value).deleteIfExists(key: string): Promise<boolean>- Deletes key if it exists. Returnstrueif deleted.compareAndDelete(key: string, oldValue: any): Promise<boolean>- Deletes key only if current value equalsoldValue.compareAndDeleteDetailed(key: string, oldValue: any, options?: { includeCurrentOnMismatch?: boolean }): Promise<{ deleted: true, reason: "deleted" } | { deleted: false, reason: "mismatch", existed: boolean, current?: any }>- Detailed compare-and-delete diagnostics.oldValue: null/undefinedis treated as regular expected-value comparison through the configured serializer (not an absent-key sentinel).
compareAndSwap()andcompareAndDelete()behavior and return types are unchanged.compareAndSwapDetailed()/compareAndDeleteDetailed()are additive opt-in APIs for richer mismatch diagnostics.setIfAbsent()is a convenience API over absent-key CAS semantics; existing APIs are not redefined.
-
scan(options?: ScanOptions): Promise<ScanResult>- Streams entries in lexicographic order using cursor-based pagination.interface ScanOptions { prefix?: string; // Filter by key prefix limit?: number; // Max results per page; positive values are capped at 100000 cursor?: string; // Opaque cursor produced by the previous page ("" starts a new scan) } interface ScanResult { entries: Array<{ key: string; value: any }>; cursor: string; // Opaque cursor for the next page done: boolean; // True when the scan reached the end of the prefix window }
Use
scan()with a bounded positivelimitwhen the keyspace is too large to materialize withlist()or when you need restart-safe pagination. Treatcursoras an opaque continuation token. Do not parse, modify, or construct it manually. Use a cursor only with the same logical scan options that produced it, especially the sameprefix. Pagination is cursor-based, but it is not a long-lived snapshot. If keys are inserted or deleted between page calls, later pages may reflect those changes. On disk backend, eachscan()call is a separate bbolt read transaction. On memory backend, concurrent writes can also affect what you observe during scan/list iteration. -
scanKeys(options?: ScanKeysOptions): Promise<ScanKeysResult>- Streams key names in lexicographic order using cursor-based pagination.interface ScanKeysOptions { prefix?: string; // Filter by key prefix limit?: number; // Max keys per page; positive values are capped at 100000 cursor?: string; // Opaque cursor produced by the previous page ("" starts a new scan) } interface ScanKeysResult { keys: string[]; cursor: string; // Opaque cursor for the next page done: boolean; // True when the scan reached the end of the prefix window }
scanKeys()is the key-only equivalent ofscan(). It does not clone, deserialize, or return values. PreferscanKeys()with a bounded positivelimitfor large stores or load-test paths. Avoidlimit <= 0on large keyspaces unless you intentionally want to materialize every matching key in one VU. Treatcursoras an opaque continuation token. Do not parse, modify, or construct it manually. Use a cursor only with the same logical scan options that produced it, especially the sameprefix. Pagination is cursor-based, but it is not a long-lived snapshot. If keys are inserted or deleted between page calls, later pages may reflect those changes. For exclusive allocation workflows, useclaimRandom()orpopRandom()instead of scan/list pagination.let cursor = ""; do { const page = await kv.scanKeys({ prefix: "user:", cursor, limit: 1000, }); const users = await kv.getMany(page.keys); cursor = page.cursor; } while (!page.done);
-
list(options?: ListOptions): Promise<Array<{ key: string; value: any }>>- Returns entries sorted lexicographically by key.interface ListOptions { prefix?: string // Filter by key prefix limit?: number // Max results; positive values are capped at 250000 }
-
listKeys(options?: ListKeysOptions): Promise<string[]>- Lists key names without returning values.interface ListKeysOptions { prefix?: string // Optional key prefix filter limit?: number // Max keys; positive values are capped at 250000 }
listKeys()is read-only, returns keys in ascending lexicographic order, and does not clone, deserialize, or return values. Use it for small datasets, debugging, setup validation, and bounded previews. For large stores, preferscanKeys({ limit })so each VU only materializes one page at a time. Useful flow before destructive calls:const keys = await kv.listKeys({ prefix: "tmp:", limit: 1000 }); await kv.deleteMany(keys);
Backend note:
- disk backend reads key names from bbolt cursors so results reflect the durable store;
- memory backend uses key-only shard iterators and merges them lexicographically;
listKeys()is not cursor-paginated; usescanKeys()for key-only cursor pagination;- use
scan()when you need key/value entries.
-
count(options?: CountOptions): Promise<number>- Returns number of keys matching prefix.
count()(or omitted options) is equivalent tosize().interface CountOptions { prefix?: string // Filter by key prefix }
-
randomKey(options?: RandomKeyOptions): Promise<string>- Returns a random key, or""if none match.interface RandomKeyOptions { prefix?: string // Optional prefix filter }
randomKey()only returns a key string and does not create/observe leases. No-match is not an error, but the promise may reject for invalid options, closed stores, backend I/O errors, or other technical failures. -
randomKeys(options: RandomKeysOptions): Promise<string[]>- Returns random key names matching an optional prefix.interface RandomKeysOptions { prefix?: string // Optional prefix filter count: number // Required integer in range [1, 1000000] unique?: boolean // Defaults to true }
const keys = await kv.randomKeys({ prefix: "user:", count: 100, unique: true, }); const users = await kv.getMany(keys);
randomKeys()returns keys only. It does not clone, deserialize, or return values.countis capped at1000000to protect the k6 process from unbounded allocations. Whenuniqueistrueand fewer matching keys exist than requested, all available matching keys are returned in random order. UseclaimRandom(),claimKey(),claimRandomMany(),popRandom(), orpopRandomMany()when you need exclusive allocation. -
popRandom(options?: { prefix?: string }): Promise<{ key: string, value: any } | null>- Claims one random free matching entry and removes it. Resolves tonullwhen no match exists. -
claimRandom(options?: { prefix?: string, owner?: string, ttl?: number }): Promise<{ id: string, key: string, token: number, owner?: string, expiresAt: number, entry: { key: string, value: any } } | null>- Leases one random matching free entry. Live claims are excluded from laterclaimRandom()andpopRandom()calls until released/completed or expired. Ifttlis omitted, the default lease is 30000ms (30 seconds).ttlmust be a positive integer and is capped at 86400000ms (24 hours).owneris optional diagnostic metadata capped at 256 bytes and is not emitted as a metrics label. -
claimKey(key: string, options?: { owner?: string, ttl?: number }): Promise<{ id: string, key: string, token: number, owner?: string, expiresAt: number, entry: { key: string, value: any } } | null>- Leases one specific key. Resolves tonullwhen the key is missing or already live-claimed. -
claimKeys(keys: string[], options?: { owner?: string, ttl?: number, allOrNothing?: boolean }): Promise<{ claimed: Array<{ id: string, key: string, token: number, owner?: string, expiresAt: number, entry: { key: string, value: any } }>, busy: string[], missing: string[] }>- Leases explicit keys in one call and returnsclaimed+busy+missingsets. Duplicate/empty keys reject as invalid options. WithallOrNothing: true, processing stops on the firstbusy/missingkey and only attempts best-effort rollback for claims acquired by this call when a later key is missing or busy.allOrNothingis a best-effort cleanup helper and does not makeclaimKeys()transactional. UseallOrNothing: falsewhen you need a full diagnostic partition of all requested keys;allOrNothing: trueis optimized for fast reservation and early rollback.claimKeys()accepts explicit keys only; it does not support prefix filtering in options.claimKeys()classifiesmissing/busyon a best-effort basis under concurrent mutation. It first attemptsClaimKey()and only when that returnsnulldoes a fallbackExists()check formissingvsbusyclassification. It is designed for deterministic fixture reservation, not transaction-level classification; a key can still be reclassified by concurrent mutation between claim attempt and fallback existence check. Only technical storage errors reject rollback; expiry/missing races during rollback are tolerated. -
claimRandomMany(options: { prefix?: string, count: number, owner?: string, ttl?: number }): Promise<Array<{ id: string, key: string, token: number, owner?: string, expiresAt: number, entry: { key: string, value: any } }>>- Leases up tocountunique random free matching entries from one store call. Resolves to[]when no free matches exist. -
popRandomMany(options: { prefix?: string, count: number }): Promise<Array<{ key: string, value: any }>>- Claims up tocountrandom free matching entries, decodes them, and completes each claim withdeleteKey=true. Resolves to[]when no free matches exist. Completed deletes are not rolled back if a later completion fails. Remaining live claims are released best-effort. -
releaseClaim(claim: { id: string, key: string, token: number }): Promise<boolean>- Releases a live claim back to the pool. Returnsfalsefor stale/expired/missing claims. -
releaseClaims(claims: Array<{ id: string, key: string, token: number }>): Promise<{ attempted: number, released: number, failed: Array<{ index: number, id: string, key: string, name: string, message: string }> }>- Batch release helper with partial success. Payload shape errors reject; stale/missing claims are reported infailedwithname: "ClaimNotUpdated". Technical storage errors reject the whole promise, include applied-progress details in the error message (after releasing X of Y), and may happen after earlier items in the same batch were already applied. -
completeClaim(claim, options?: { deleteKey?: boolean }): Promise<boolean>- Completes a live claim. By default it also deletes the underlying key (deleteKey: true). -
completeClaims(claims: Array<{ id: string, key: string, token: number }>, options?: { deleteKey?: boolean }): Promise<{ attempted: number, completed: number, failed: Array<{ index: number, id: string, key: string, name: string, message: string }> }>- Batch completion helper with partial success and the samedeleteKeydefault ascompleteClaim(). Stale/missing claims are reported infailedwithname: "ClaimNotUpdated". Technical storage errors reject the whole promise, include applied-progress details in the error message (after completing X of Y), and may happen after earlier items in the same batch were already applied. -
renewClaim(claim: { id: string, key: string, token: number }, options: { ttl: number }): Promise<boolean>- Extends a live claim lease without changing the claim token. Returnsfalsefor stale/expired/missing claims or when the claim no longer owns the key. -
renewClaims(claims: Array<{ id: string, key: string, token: number }>, options: { ttl: number }): Promise<{ attempted: number, renewed: number, failed: Array<{ index: number, id: string, key: string, name: string, message: string }> }>- Batch lease-renew helper with partial success and per-item failure diagnostics. Stale/missing claims are reported infailedwithname: "ClaimNotUpdated". Technical storage errors reject the whole promise, include applied-progress details in the error message (after renewing X of Y), and may happen after earlier items in the same batch were already applied.Batch lifecycle helper semantics:
releaseClaims(),completeClaims(), andrenewClaims()are lifecycle convenience helpers.- They execute claim operations sequentially in Go and return partial success summaries.
- Per-item stale/missing failures use stable
name: "ClaimNotUpdated"in thefailed[]array;failed[].indexmaps each failure to the original input position. - Technical storage errors reject the whole promise and can happen after earlier items in the same batch were already applied.
- Rejection messages include applied-progress context (
after releasing/completing/renewing X of Y) and earlier successful items are not rolled back. - They are not cross-claim transactions and are intended for cleanup ergonomics, not backend-native high-throughput batch mutation.
Claim lifecycle guidance:
- Use
completeClaim()/completeClaims()on success (deleteKey: trueconsumes items permanently). - Use
releaseClaim()/releaseClaims()on failure to return items to the pool.
const claim = await kv.claimRandom({ prefix: 'user:' }); if (claim === null) return; try { // do work with claim.entry.value await kv.completeClaim(claim); // success path } catch (err) { await kv.releaseClaim(claim); // failure path throw err; }
⚠️ Claim APIs (claimRandom,claimKey,claimKeys,claimRandomMany,releaseClaim,releaseClaims,renewClaim,renewClaims,completeClaim,completeClaims,popRandom,popRandomMany) are local coordination primitives for VUs sharing the samexk6-kvprocess/store. They are not distributed lock services.claimRandom()andpopRandom()are random allocation helpers optimized for low/moderate live-claim occupancy. Under very high live-claim occupancy, fallback selection can be biased toward scan order, but exclusivity is still preserved.claim.tokenis exposed as a JavaScript number. It should not approachNumber.MAX_SAFE_INTEGERin practical k6 runs; if that ever becomes realistic, a future major API should expose it as a string.
-
rebuildKeyList(): Promise<boolean>- Rebuilds in-memory key indexes (useful for disk backend withtrackKeys: true). -
stats(): Promise<KVStats>- Returns a structured diagnostic snapshot of the current store state.interface KVStats { backend: "memory" | "disk" serialization: "json" | "string" trackKeys: boolean count: number claims: { live: number; expired: number } index?: { enabled: boolean keysList?: number keysMap?: number ost?: number consistent: boolean } | null disk?: { path: string sizeBytes: number readOnly?: boolean } | null }
const snapshot = await kv.stats(); console.log(snapshot.count, snapshot.claims.live);
-
allocationStats(options?: { prefix?: string }): Promise<{ prefix: string, total: number, claimable: number, claimedLive: number, claimedExpired: number, backend: "memory" | "disk", trackKeys: boolean }>- Returns prefix-scoped allocation health for claim workflows.prefixis optional; empty/omitted means all keys.const pool = await kv.allocationStats({ prefix: "users:" }); console.log(pool.total, pool.claimable, pool.claimedLive, pool.claimedExpired);
Use
stats()/reportStats()for global store health. UseallocationStats({ prefix })when you manage independent pools by prefix.allocationStats()is intentionally an on-demand pool-availability diagnostic (not a metrics label dimension and not a low-level bbolt consistency checker). Prefix tags would create high-cardinality time-series in Prometheus-style metrics. Backend behavior:- memory backend: scans matching in-memory keyspace/shards;
- disk backend +
trackKeys: true: writable handles read process-local operational key index used by claim APIs; - disk backend +
trackKeys: true+ read-only open: falls back to durable bbolt key/claim scan; - disk backend +
trackKeys: false: scans durable bbolt keys for the prefix. Treattotalas "keys visible to the allocation index" for disktrackKeys: true, not as a forensic durable bbolt key count after out-of-band mutations. After out-of-band durable mutations, checkstats().index.consistentand callrebuildKeyList()before trusting prefix totals. Recovery pattern:
const stats = await kv.stats(); if (stats.backend === "disk" && stats.trackKeys && stats.index && !stats.index.consistent) { await kv.rebuildKeyList(); } const pool = await kv.allocationStats({ prefix: "users:" });
For large prefixes it may still be scan-heavy; use it in setup/teardown, health checks, or low-frequency diagnostics, not on every hot-path iteration. Expired claims are counted as claimable; allocation operations can lazily reap or overwrite expired claim metadata. The diagnostic call itself does not promise physical cleanup of stale lease records.
-
reportStats(): Promise<void>- Emits state gauges to k6 custom metrics using the current snapshot.Emitted metrics:
xk6_kv_keysxk6_kv_claims_livexk6_kv_claims_expiredxk6_kv_index_keys(withindex=keys_list|keys_map|ost)xk6_kv_index_consistentxk6_kv_disk_size_bytes(disk backend only)
Prefix-specific diagnostics are intentionally excluded from metrics labels to avoid high-cardinality time-series. Use
allocationStats({ prefix })for those checks.await kv.reportStats();
-
metrics.operations(openKv option) - Enables automatic operation metrics for every async KV method except syncclose().Emitted metrics:
xk6_kv_operations_total(Counter, tags:op,backend,status,track_keys,serialization)xk6_kv_operation_duration(Trend in milliseconds, tags:op,backend,status,track_keys,serialization)xk6_kv_operation_failed(Rate, tags:op,backend,track_keys,serialization)xk6_kv_errors_total(Counter, tags:op,backend,error_type,track_keys,serialization)xk6_kv_empty_result(Rate forrandom_key/random_keys/pop_random/claim_random/claim_key/claim_random_many/pop_random_many, tags:op,backend,track_keys,serialization)xk6_kv_async_in_flight(Gauge for async store operations currently running, tags:backend,track_keys,serialization)
Batch lifecycle helpers (
releaseClaims/completeClaims/renewClaims) can resolve with item-levelfailed[]entries without promise rejection. Those partial outcomes are returned in API results and are not emitted as operation-level error metrics unless the promise itself rejects on a technical storage error.xk6_kv_async_in_flightis the current saturation signal for background store operations in the async bridge. It is decremented when the store goroutine queues its event-loop completion callback, so it is not a count of unresolved JavaScript promises. During shutdown/cancellation, the VU context may already be canceled and the final decrement sample can be dropped by the k6 metrics sink, even though xk6-kv still decrements its internal in-flight counter. There is nowaitingmetric because xk6-kv does not currently have an async limiter or queue; if one is added later, a low-cardinality waiting gauge can be added alongside it.const kv = openKv({ backend: "memory", metrics: { operations: true }, });
Quickstart:
import { openKv } from "k6/x/kv"; const kv = openKv({ backend: "memory", trackKeys: true, metrics: { operations: true }, }); export const options = { thresholds: { "xk6_kv_operation_failed": ["rate==0"], "xk6_kv_empty_result{op:claim_random}": ["rate<0.05"], "xk6_kv_async_in_flight": ["value<1000"], "xk6_kv_index_consistent{track_keys:true}": ["value==1"], }, }; export default async function () { await kv.claimRandom({ prefix: "user:" }); await kv.reportStats(); }
Troubleshooting: if k6 reports
no metric name "xk6_kv_..." found, make sure:openKv({ metrics: { operations: true } })runs in init context, and- you rebuilt your binary with this extension (
task build-k6).
-
backup(options?: BackupOptions): Promise<BackupSummary>
Writes the current dataset to a bbolt file. Always setfileName(leaving it blank points at the backend’s live bbolt file) and useallowConcurrentWrites: truefor a best-effort dump that releases writers sooner (summary includesbestEffort+warningso you can alarm on it).Memory backend caution: when
allowConcurrentWritesis left at the defaultfalse, the memory backend holds its global mutation gate for the entire duration of the backup (from key snapshot through streaming). On large datasets that can pause every writer/VU for minutes. EnableallowConcurrentWrites: trueif you need the cluster to keep serving traffic during the export (accepting the best-effort snapshot) or schedule strict backups during quiet windows.Shared-file workflow: Leaving
fileNameblank while running the memory backend is intentional—it writes into.k6.kv, the same file the disk backend mounts by default. That makes a common DX pattern possible: run the hot path withbackend: "memory", callbackup()without arguments inteardown(), and later rerun the same test withbackend: "disk"to replay the captured dataset. If you want snapshots to live somewhere else (or you run disk workloads in parallel), provide an explicitfileNameso you don’t clobber the shared DB.File replacement semantics: backup/export writes to a temp file in the destination directory, fsyncs, then replaces the destination and syncs the parent directory. On Unix-like filesystems this is same-directory atomic rename. On Windows, replacement uses best-effort remove-then-rename fallback when direct rename cannot replace an existing file, so treat it as crash-safer overwrite rather than a portable transactional primitive.
-
restore(options?: RestoreOptions): Promise<RestoreSummary>
Replaces the dataset with a snapshot produced bybackup(). OptionalmaxEntries/maxBytesguards protect against oversized or corrupted inputs. -
exportJSONL(options: { fileName: string, prefix?: string, limit?: number }): Promise<{ exported: number, fileName: string, bytesWritten: number }>
Exports key/value entries to a JSON Lines file for portable seed data and diff-friendly snapshots. Each line is:{"key":"...","value":...}. Values are exported after normal KV deserialization, not as raw backend bytes. This is a scan-based export, not a long-lived point-in-time snapshot. Concurrent writes/deletes may affect later pages; usebackup()for snapshot-style capture. -
exportCSV(options: { fileName: string, prefix?: string, limit?: number, delimiter?: string, columns: string[], includeKey?: boolean }): Promise<{ exported: number, fileName: string, bytesWritten: number }>
Exports tabular object data to CSV for portable handoff/reporting. Header row is always written. Values are exported after normal KV deserialization, not as raw backend bytes. This is a scan-based export, not a long-lived point-in-time snapshot. Concurrent writes/deletes may affect later pages; usebackup()for snapshot-style capture. -
importJSONL(options: { fileName: string, limit?: number, batchSize?: number }): Promise<{ imported: number, fileName: string, bytesRead: number }>
Imports key/value entries from a JSON Lines file. Each line must be:{"key":"...","value":...}. -
importCSV(options: { fileName: string, keyColumn: string, delimiter?: string, hasHeader?: boolean, limit?: number, batchSize?: number }): Promise<{ imported: number, fileName: string, bytesRead: number }>
Imports key/value rows from a CSV file. Each row becomes one object value;keyColumnselects the key field. -
validateCSV(options: { fileName: string, keyColumn?: string, delimiter?: string, hasHeader?: boolean, limit?: number }): Promise<{ valid: boolean, rows: number, bytesRead: number, checkedAll: boolean, firstError?: { row: number, name: string, message: string } }>
Read-only CSV preflight check with two modes:- syntax mode (
keyColumnomitted): validates CSV readability, delimiter/header shape; - import-shape mode (
keyColumnprovided): validates syntax plus key extraction rules used byimportCSV(). Validation APIs are KV-instance methods for API consistency and metrics collection. They require an open KV handle (reject withStoreClosedErrorafterclose()), but do not read or write KV store contents. Content errors resolve withvalid: false; invalid options and file I/O errors reject.bytesReadis diagnostic progress information only. It is not a stable resume offset and should not be used as a cursor.checkedAll=truemeans validation reached EOF.checkedAll=falsemeans validation stopped becauselimitwas reached. Contract: treatvalid: trueas full-file validity only whencheckedAll === true.delimiterfollows the same contract asimportCSV()(1character, not\r,\n,", or Unicode replacement rune\uFFFD). If omitted or<= 0,limitvalidates all rows. Whenlimit > 0, only the firstNdata rows are checked, sovalid: trueapplies to the inspected prefix.
- syntax mode (
-
validateJSONL(options: { fileName: string, limit?: number }): Promise<{ valid: boolean, records: number, bytesRead: number, checkedAll: boolean, firstError?: { line: number, name: string, message: string } }>
Read-only JSONL preflight check. Content errors resolve withvalid: false; invalid options and file I/O errors reject. Validation APIs are KV-instance methods for API consistency and metrics collection. They require an open KV handle (reject withStoreClosedErrorafterclose()), but do not read or write KV store contents.bytesReadis diagnostic progress information only. It is not a stable resume offset and should not be used as a cursor.checkedAll=truemeans validation reached EOF.checkedAll=falsemeans validation stopped becauselimitwas reached. Contract: treatvalid: trueas full-file validity only whencheckedAll === true. Whenlimit > 0, only the firstNrecords are checked, sovalid: trueapplies to the inspected prefix.
await kv.backup({
fileName: "./backups/kv-latest.kv",
allowConcurrentWrites: true,
});
await kv.restore({ fileName: "./backups/kv-latest.kv" });Disk backend note: pointing
fileNameat the currently mounted bbolt path is treated as a no-op (backup just returns metadata; restore leaves the DB untouched), so when you’re already runningbackend: "disk"you still need a differentfileName. The “shared.k6.kvtrick” only applies when you begin on the memory backend and want to seed the disk backend later.
exportJSONL() example:
const result = await kv.exportJSONL({
fileName: "./exports/users.jsonl",
prefix: "user:",
});
console.log(result.exported);
console.log(result.bytesWritten);exportJSONL() options:
fileNameis required and must be a non-empty string.prefixis optional; empty or omitted means all keys.limitis optional; if omitted or<= 0, all matching entries are exported. Positive values are capped at1000000.
exportJSONL() writes to a temporary file, flushes and fsyncs it, replaces the target, then syncs the parent directory. On Unix-like filesystems this gives atomic replacement in the common same-directory case. On Windows, replacement falls back to remove-then-rename when needed; this is still best-effort crash safety and avoids intentionally writing partial data directly into the target file.
exportCSV() example:
const result = await kv.exportCSV({
fileName: "./exports/responses.csv",
prefix: "responses:",
includeKey: true,
columns: ["status", "requestId", "userId", "bodyHash"],
});
console.log(result.exported);
console.log(result.bytesWritten);exportCSV() options:
fileNameis required and must be a non-empty string.columnsis required, must be non-empty, and column names must be unique non-empty strings.- Column names are exact; leading/trailing whitespace is significant.
includeKeyis optional and defaults totrue.- When
includeKey=true,columnsmust not contain"key"becauseexportCSV()writes the store key as the first CSV column. - Use
includeKey=falseif you need to export a value field named"key". delimiteris optional and must be a valid CSV delimiter: exactly one character, not\r,\n,", or Unicode replacement rune\uFFFD. Default is comma.prefixis optional; empty or omitted means all keys.limitis optional; if omitted or<= 0, all matching entries are exported. Positive values are capped at1000000.
exportCSV() data-shape constraints:
exportCSV()exports tabular object data.- Each exported value must decode to a plain JSON object row.
- Internal contract: value deserialization must produce
map[string]any. - Only top-level scalar fields are supported.
- Missing requested fields are written as empty cells.
- Nested objects and arrays are rejected.
- Nested path expressions (for example,
user.id) are not supported. - Scalar/non-object stored values are rejected.
- Stores opened with
serialization: "string"are generally incompatible withexportCSV()because values decode as strings, not JSON object rows. - Supported scalar field types: string, boolean, integer/float,
json.Number, andnull. - CSV is a flat text format and does not preserve JSON value types across roundtrips.
- Numbers/booleans are exported as text cells and
importCSV()reads those cells as strings. - Use
exportJSONL()+importJSONL()for type-preserving roundtrips. - For scalar/string values, use
exportJSONL(). - No flattening, schema inference, transforms, or automatic nested JSON stringification are performed.
Rejected-shape examples:
await kv.set("bad:1", {
status: 200,
body: { id: 123 },
});
await kv.set("bad:2", "raw body");
await kv.exportCSV({
fileName: "./bad.csv",
prefix: "bad:",
columns: ["status", "body"],
});
// rejects:
// - bad:1 -> nested object column ("body")
// - bad:2 -> scalar/non-object row value
// use exportJSONL() for nested/scalar dataexportCSV() uses the same temp-file -> flush/sync -> rename -> parent-dir-sync replacement strategy as exportJSONL().
importJSONL() example:
const result = await kv.importJSONL({
fileName: "./exports/users.jsonl",
batchSize: 1000,
});
console.log(result.imported);
console.log(result.bytesRead);importJSONL() options:
fileNameis required and must be a non-empty string.limitis optional; if omitted or<= 0, all records are imported. Positive values are capped at1000000.batchSizeis optional; if omitted or<= 0, the default batch size is used. Positive values are capped at10000.importJSONL()enforces a per-line safety cap of 64 MiB; oversized records are rejected with a line-numbered parse error.
importJSONL() streams records and writes them in setMany() batches. Existing keys are overwritten.
bytesRead reports how many input bytes were consumed by the importer.
Each line must be a JSON object with:
key- required non-empty string.value- required JSON value, includingnull.
importJSONL() rejects blank lines and malformed JSON records. Errors include the source line number.
The import is batch-atomic, not file-atomic: if a later line is invalid, already committed batches remain imported, while the currently failed batch is not partially written. Failure messages include committed progress (records, bytes, and line context) so callers can locate the bad record and decide whether a retry is safe.
importCSV() streams rows and writes them in setMany() batches. Existing keys are overwritten.
keyColumn is required; with hasHeader: true it is a header name, with hasHeader: false it must be a zero-based column index encoded as string.
CSV rows are lenient by width: if a row has fewer columns than the header, missing fields are imported as empty strings; if a row has more columns than the header, extra fields are preserved as column_N.
Rows with missing/empty keys are rejected with row-numbered parse errors.
importCSV() options:
fileNameis required and must be a non-empty string.keyColumnis required (column name whenhasHeader: true, zero-based index string whenhasHeader: false).delimiteris optional and must be a valid CSV delimiter: exactly one character, not\r,\n,", or Unicode replacement rune\uFFFD.hasHeaderis optional and defaults totrue.limitis optional; if omitted or0, all rows are imported. Negative values are rejected. Positive values are capped at1000000.batchSizeis optional; if omitted or0, the default batch size is used. Negative values are rejected. Positive values are capped at10000.- CSV import/validation currently relies on Go
encoding/csvand does not enforce a per-record byte cap; avoid feeding untrusted arbitrarily large rows. For bounded per-record guards, prefer JSONL (64 MiBper-line limit).
validateCSV() example:
const csvCheck = await kv.validateCSV({
fileName: "./examples/fixtures/users.csv",
keyColumn: "id",
hasHeader: true,
});
if (!csvCheck.valid) {
throw new Error(`csv invalid at row ${csvCheck.firstError.row}: ${csvCheck.firstError.message}`);
}validateJSONL() example:
const jsonlCheck = await kv.validateJSONL({
fileName: "./examples/fixtures/users.jsonl",
});
if (!jsonlCheck.valid) {
throw new Error(`jsonl invalid at line ${jsonlCheck.firstError.line}: ${jsonlCheck.firstError.message}`);
}Validation semantics (validateCSV() / validateJSONL()):
-
Read-only: no writes to the KV store.
-
validateCSV()has two modes:- syntax mode (
validateCSV({ fileName, hasHeader, delimiter })) checks CSV readability/header shape; - import-shape mode (
validateCSV({ fileName, keyColumn, hasHeader, delimiter })) also validates key extraction compatibility withimportCSV().
- syntax mode (
-
Validation methods are KV-instance methods for API consistency and metrics collection.
-
They require an open KV handle (
StoreClosedErrorafterclose()), but they do not read or write KV store contents. -
They are read-only (no KV writes) but still go through normal operation lifecycle/metrics paths.
-
Stops at first malformed content record.
-
Malformed content resolves a structured invalid result (
valid: false,firstError). -
Invalid options, file open/permission/system I/O failures, and context cancellation reject.
-
bytesReadis a diagnostic byte counter for data read from the file, not a stable resume/cursor offset. -
validateCSV()uses the same delimiter contract asimportCSV()/exportCSV()(exactly one character, not\r,\n,", or\uFFFD). -
validateCSV()uses Goencoding/csvparsing compatible withimportCSV(). -
CSV row-width handling is lenient (same as
importCSV()): missing columns are treated as empty fields; extra columns are accepted. -
In import-shape mode (
keyColumnset), this means key checks run against normalized rows (""for missing cells,column_Nfor extras). -
validateCSV()limitdefaults to full-file validation when omitted or<= 0. -
validateJSONL()limitdefaults to full-file validation when omitted or<= 0. -
checkedAll=truemeans validation reached EOF. -
checkedAll=falsemeans validation was stopped bylimit; in this casevalid: trueconfirms only inspected prefix validity. -
Contract:
valid: truemeans "whole file valid" only whencheckedAll === true. -
With
limit > 0, only the inspected prefix is validated:- CSV: first
Ndata rows; - JSONL: first
Nrecords. In that mode,valid: truedoes not imply the whole file is valid.
- CSV: first
-
Empty CSV/JSONL files are considered valid preflight input (
rows: 0/records: 0). -
If your workflow requires non-empty data, assert
rows > 0orrecords > 0in script checks. -
close(): void- Synchronously closes this KV handle. Call once inteardown(). Afterclose(), this handle rejects async operations withStoreClosedErroron both backends. Operations that already started beforeclose()may still resolve or reject normally. Closing one handle does not affect other open handles until the shared store refcount reaches zero. It also does not allow lateropenKv()calls to switch backend, path, serialization, or key-tracking options. Do not callclose()fromdefault()iterations.
randomKey()complexity by backend:trackKeys: true:- disk backend: no prefix -> O(1); prefix -> O(log n) via key index.
- memory backend: no prefix includes a small O(shards) selection step; prefix performs per-shard range selection (O(shards * log n)) before final rank/select.
trackKeys: false(default): scan-based path with linear cost in the matching keyspace. Achieving tracked-path speeds means keys are mirrored in memory helper structures, so large datasets consume more RAM and index slices/maps do not shrink automatically. Budget for that footprint or rebuild indexes periodically.
- Random key workloads: Calling
randomKey()repeatedly withtrackKeys: false(especially on the disk backend) keeps a read transaction open while it counts and selects keys, which can stall the lone bbolt writer until the call finishes. Turn ontrackKeys(for O(1)/O(log n) sampling) or throttle/redesign these workloads to avoid head-of-line blocking. randomKeys()complexity by backend: WithtrackKeys: true, both backends use key indexes for small samples; memory first builds shard ranges (O(shards * log n)) and then selects sampled keys by rank, while disk may fall back to cursor scan for near-full unique samples. WithtrackKeys: false, it collects candidates viascanKeys()and samples in memory (linear in matching keys).- Memory
trackKeys: falsescan/list costs:scan(),scanKeys(),list(), andlistKeys()use untracked shard-map iteration. On large keyspaces, repeated pagination can become expensive; if these operations are hot, prefertrackKeys: true. - Disk backend and
trackKeys: bbolt is the persistent source of truth. WithtrackKeys: true, the disk backend maintains an exact derived in-memory key index rebuilt from bbolt on open/restore and updated after successful mutations. That index acceleratesrandomKey(),randomKeys(), andcount(), while cursor-style key reads (scanKeys()andlistKeys()) still read from bbolt. - Disk claim allocation: with
trackKeys: true, claim metadata is stored in process-local in-memory OST metadata (not bbolt).claimRandom(),claimKey(),claimRandomMany(),releaseClaim(),renewClaim(), andcompleteClaim({ deleteKey: false })stay on the in-memory path; only durable key deletes (popRandom(),popRandomMany(),completeClaim({ deleteKey: true })) require bboltUpdate(). WithtrackKeys: false, claim metadata remains in the bbolt claims-bucket fallback path, and batch random claim allocation may scan/materialize a large candidate set. For high-throughput disk random allocation, prefertrackKeys: true.
allocationStats() is diagnostic and can be scan-heavy on large prefixes. Use this benchmark matrix to profile your environment:
go test ./kv/store -run '^$' -bench 'AllocationStats' -benchtime=3s -count=3 -benchmemScenarios covered in the benchmark:
- memory
trackKeys=falsewith prefix coverage1%,10%,100% - memory
trackKeys=truewith prefix coverage1%,10%,100% - disk
trackKeys=falsewith prefix coverage1%,10%,100% - disk
trackKeys=truewith prefix coverage1%,10%,100% - memory keyspace-scaling matrix (
1k,10k,100ktotal keys with10%prefix matches)
Memory keyspace-scaling command (to quantify O(N)-style diagnostics pressure before optimizing):
go test ./kv/store -run '^$' -bench 'MemoryStore_AllocationStats_KeyspaceScaling' -benchtime=3s -count=3 -benchmemOn the current benchmark machine (linux/amd64, AMD Ryzen 9 9950X, short benchmark windows), disk random allocation favors trackKeys: true for claim operations:
| Operation | trackKeys=true |
trackKeys=false |
Note |
|---|---|---|---|
claimRandom |
~1.5-1.6 us/op |
~0.48-1.28 ms/op |
tracked path avoids bbolt claim writes |
claimRandomMany (count=100) |
~58-64 us/op |
fallback path substantially slower under load | near-linear scaling with count |
renewClaim (tracked) |
~170-180 ns/op |
N/A | in-memory claim metadata update |
popRandom |
~0.6-2.6 ms/op |
~0.6-2.6 ms/op |
durable delete path dominates |
Treat these as directional numbers for this machine/runtime profile; rerun benchmarks in your environment for release gating.
- Bound large reads and writes: For large keyspaces, prefer
scan()/scanKeys()with bounded positivelimitvalues instead oflist()/listKeys()or unlimitedlimit <= 0calls. Oversized positive limits are rejected to avoid unbounded allocations and long transactions. count()/count({ prefix })complexity:count()(same assize()):- memory backend: O(shards)
- disk backend with
trackKeys: true: O(1) - disk backend with
trackKeys: false: bbolt bucket stats (not guaranteed constant-time)
count({ prefix }):- memory +
trackKeys: true: O(shards * log n) - memory +
trackKeys: false: O(n) - disk +
trackKeys: true: O(log n) - disk +
trackKeys: false: O(k), wherekis number of keys under prefix
- memory +
- Prefix cardinality on large disk datasets: If
count({ prefix })is hot on disk, prefertrackKeys: true. Without tracking, the disk path walks a bbolt cursor through matching keys. - Both backends are optimized for concurrent workloads, but there's synchronization overhead between VUs
Runnable entry points: Start here. Complete examples are in examples/; production-style scenarios are in e2e/.
Observability-focused scripts:
- Example operation metrics in a worker queue:
examples/metrics-operations-worker-queue.js - Example
stats()/reportStats()health snapshots:examples/metrics-report-stats-health.js - Example batch random key sampling + hydration:
examples/random-keys.js - E2E lease-worker observability scenario:
e2e/subscription-renewal-lease-observability.js - E2E credential pool drain scenario:
e2e/credential-pool-drain-observability.js - Batch claim allocation example:
examples/claim-random-many.js - Batch claim lifecycle example:
examples/claim-batch-lifecycle.js - Explicit fixture reservation example:
examples/claim-keys.js - Batch pop allocation example:
examples/pop-random-many.js - CSV import example:
examples/import-csv.js - CSV export example:
examples/export-csv.js - Allocation diagnostics example:
examples/allocation-stats.js - Import preflight validation example:
examples/validate-import-files.js - E2E batch lifecycle scenario:
e2e/batch-claim-lifecycle.js - E2E explicit fixture reservation scenario:
e2e/claim-keys-explicit-fixtures.js - E2E allocation diagnostics scenario:
e2e/allocation-stats-pool-health.js - E2E CSV export scenario:
e2e/export-csv-response-capture.js
- Unique users: import a user pool (
importCSV()orsetMany()), allocate withclaimRandom()/claimRandomMany(), thencompleteClaim()/completeClaims()on success orreleaseClaim()/releaseClaims()on failure. - Credential pool: fetch exact credentials with
claimKey()/claimKeys(), extend long-running work withrenewClaim()/renewClaims(), and always release stale/failed attempts. - Response capture: write response envelopes with
kv.set("responses:<id>", payload)during the run, thenexportJSONL()orexportCSV()in teardown/summary. - Pool diagnostics: use global
stats()for store-wide health andallocationStats({ prefix })for prefix-scoped claimability snapshots. - Cross-run handoff: use disk backend +
backup()/restore()orexportJSONL()/importJSONL(); claim leases are process-local and are cleared on writable open/clear/restore/rebuild/close.
import { openKv } from "k6/x/kv";
const kv = openKv({ backend: "memory", trackKeys: true });
export async function producer() {
const id = (await kv.get("latest-id")) || 0;
await kv.set(`token-${id}`, "value");
await kv.set("latest-id", id + 1);
}
export async function consumer() {
const key = await kv.randomKey({ prefix: "token-" });
if (key) {
await kv.get(key);
await kv.delete(key);
}
}import { openKv } from "k6/x/kv";
const kv = openKv({ backend: "disk", trackKeys: true });
export default async function () {
await kv.set("user:1", { name: "Alice" });
await kv.set("user:2", { name: "Bob" });
const key = await kv.randomKey({ prefix: "user:" });
if (key) {
const user = await kv.get(key);
console.log(`Random user: ${JSON.stringify(user)}`);
}
}This project uses a Taskfile for cross-platform development (Windows, Linux, macOS, WSL).
# Install Task (once)
# macOS/Linux: brew install go-task/tap/go-task
# Windows: scoop install task
# Install dev tools
task install-tools
# Format and lint
task lint-fix
# Run tests
task test
# Build k6 with local extension
task build-k6The Taskfile includes internal helpers (ensure-bin, tool installers) plus user-facing tasks below.
| Command | What it does |
|---|---|
task |
Show common tasks (task -l). |
task install-githooks |
Configure local commit hooks (.githooks). |
task remove-githooks |
Disable local commit hooks for this repository. |
task version-check |
Predict next semantic version from commits. |
task pin-tool-versions |
Sync pinned tool versions in CI/local files. |
task install-tools |
Install all pinned dev tools (golangci-lint, xk6, govulncheck). |
task lint-fix |
Run golangci-lint with autofix. |
task lint |
Run golangci-lint checks. |
task vulncheck |
Run govulncheck using GOTOOLCHAIN=go<GO_VERSION> from taskfile.yaml (kept in sync with go.mod via pin-tool-versions). |
task xk6-lint-community |
Run xk6 lint --preset community . with pinned toolchain; auto-installs gosec into ./bin if missing. |
task test |
Run unit tests (go test -v ./...). |
task test-race |
Run unit tests with race detector. |
task test-typescript |
Run TypeScript declaration smoke tests (npm ci + npm test in typescript/). |
task release-check |
Run full release readiness checks (lint, xk6-lint-community, test-race, test-typescript, and E2E via scripts/release_check_e2e.sh / scripts/release_check_e2e.ps1). |
task build-k6 |
Build ./bin/k6 with the local xk6-kv extension. |
task test-e2e-all |
Run all e2e/*.js scenarios across backend/key-tracking matrix. |
task test-e2e-single E2E_SCENARIO=tenant-prefix-count-window |
Run one scenario across backend/key-tracking matrix. |
task clean |
Remove generated binaries and repository-local temporary artifacts. |
test-e2e-singlerequiresE2E_SCENARIO.test-e2e-allandtest-e2e-singlesupportVUSandITERATIONSoverrides.- E2E scenarios require the Taskfile-built binary (
task build-k6) because they exercise the localk6/x/kvextension. - Temporary E2E database, snapshot, CSV, and JSONL artifacts are removed by
scripts/run_e2e_scenario.sh/scripts/run_e2e_scenario.ps1;task cleanremoves all repository-local scratch files undertmp/. - Example:
task test-e2e-single E2E_SCENARIO=tenant-prefix-count-window VUS=20 ITERATIONS=200.
- gofumpt for formatting (stricter than gofmt)
- golangci-lint with 40+ enabled linters
- gci and goimports for import organization
- golines for line length (max 120 characters)
Run task lint-fix before committing.
When adding or changing a public JS method, update all of:
- Go method implementation and operation metric name.
typescript/xk6-kv.d.ts.README.mdAPI table/reference.- user-facing example or e2e scenario when behavior is externally visible.
typescript/api-smoke.ts.
This project uses automated semantic versioning based on commit messages:
Before public release, run task release-check.
fix: description-> Patch version (1.0.0 -> 1.0.1)feat: description-> Minor version (1.0.0 -> 1.1.0)major: description-> Major version (1.0.0 -> 2.0.0)
When commits are pushed to main:
- Commit messages are validated
- Tests and builds run across platforms
- Version is calculated and release is created automatically
- Pre-built k6 binaries are attached to the GitHub release
Enable semantic commit validation:
task install-githooksCheck next version:
task version-checkContributions are welcome! Please:
-
Fork the repository
-
Create a feature branch (
git checkout -b feature/AmazingFeature) -
Set up git hooks:
task install-githooks -
Make changes and commit using semantic format:
fix: your bug fix descriptionfeat: your new feature descriptionmajor: your breaking change description
-
Ensure code passes linting (
task lint) and tests (task test) -
Push and open a Pull Request
Note: Pull requests from forks skip semantic commit validation for easier external contributions.