Skip to content

Extend KV scripts with core contracts, multi-shard routing, and RF3 recovery - #1497

Draft
void-ptr974 wants to merge 4 commits into
oxia-db:mainfrom
void-ptr974:test/kvscript-routing
Draft

void-ptr974 wants to merge 4 commits into
oxia-db:mainfrom
void-ptr974:test/kvscript-routing

Conversation

@void-ptr974

@void-ptr974 void-ptr974 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Extend the KV scripts from #1491 into a reusable core KV contract suite across shard routing, independent clients, and RF3 recovery. Maintainers can add readable regression sequences and check the same behavior under different layouts, while dedicated Go tests cover conditional mutations that compete concurrently.

A fixture lists SDK commands followed by expected records or domain errors. The runner executes real Go SDK calls in order, reports file/line differences, and isolates each fixture's storage. Saved versions and records express CAS and metadata relationships without depending on numeric IDs or fixed timestamps. The README connects each checked contract to its fixtures and states the boundaries of those guarantees.

This PR adds 29 hand-written fixtures and the supporting assertions:

  • Records and reads: Put/Get metadata equality, creation-time preservation, modification counts, persistent-record fields, stale versions after recreation, empty/Unicode keys and arbitrary binary values, all five comparisons with omitted values, internal-key visibility, and rejected operations preserving full records.
  • Routing and isolation: an independent gRPC probe checks actual placement against advertised XXHASH3 ranges; scripts cover default/partition routes, empty/colliding/wrong partition keys, comparison candidates, range completeness/deletion scope, both sorting modes, and namespace isolation. Independent clients exchange completed writes, deletes, and CAS versions.
  • Conditional atomicity: synchronized two-client Put/Put, Put/Delete, and create-only competitions require exactly one success and a version-conflict loser; both clients check the winner's complete record and reject old versions after recreation.
  • RF3 recovery: real coordinator runtime/reconciler and three data servers use separate endpoints, WALs, and databases. Controlled stop/restart checks election terms, assignments, CAS, follower catch-up, deletion/recreation, and multishard queries. New acknowledgement cases stop the leader directly after successful Put/Delete, without an intervening read or all-replica barrier.

The matrix runs 250 script scenarios (112 RF1, 134 RF3, and four RF3 namespace cases), plus 144 conditional competitions, across one/four shards, both key sortings, and default/partition routes where applicable. wait-replicated separately checks database application through captured quorum commit offsets. Malformed commands, incorrect placement/record relations, and cluster barriers remain fatal with -rewrite; non-default clients cannot invoke probes or control commands that target the default namespace.

All cluster components run in one Go test process with in-memory coordinator metadata and graceful server stops. The same SDK clients remain open; a test resolver maintains ready discovery endpoints. These checks cover bounded competitions and controlled recovery. General concurrent-history validation, hard crashes, quorum loss, network partitions, static/stale bootstrap recovery, and Java SDK execution require separate work. Timeout/transport errors fail the suite and are not interpreted as proof that a mutation did not occur.

Validation on Linux amd64 with Go 1.27.0:

  • make build and full workspace tests with coverage, race detection, and test-result caching disabled: passed (134 packages).
  • All 250 script scenarios and 144 synchronized conditional competitions passed in that workspace run; KV package 426.0s.
  • Sixteen intentionally invalid scenarios were rejected with -rewrite, including all four non-default namespace probe/control combinations; fixture checksums stayed unchanged.
  • Full workspace lint (golangci-lint 2.13.2), license check, and checksum verification of all 51 committed test inputs: passed. Source remained clean before and after validation.

Signed-off-by: void-ptr974 <void-ptr974@protonmail.com>
Signed-off-by: void-ptr974 <void-ptr974@protonmail.com>
@void-ptr974 void-ptr974 changed the title Strengthen multi-shard KV script coverage Extend KV scripts with multi-shard routing and RF3 failover coverage Oct 5, 2026
Signed-off-by: void-ptr974 <void-ptr974@protonmail.com>
Signed-off-by: void-ptr974 <void-ptr974@protonmail.com>
@void-ptr974 void-ptr974 changed the title Extend KV scripts with multi-shard routing and RF3 failover coverage Extend KV scripts with core contracts, multi-shard routing, and RF3 recovery Oct 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant