bench(zero-cache): add rm vs load benchmark harness#6070
Draft
Karavil wants to merge 1 commit into
Draft
Conversation
|
@Karavil is attempting to deploy a commit to the Rocicorp Team on Vercel. A member of the Team first needs to authorize it. |
This was referenced May 26, 2026
7c2f898 to
c3d3d0b
Compare
Why: make RM to serving-replica perf work measurable and reviewable before changing production paths * add a v6 RM to serving replica benchmark with reconnect catchup * add config, scenario, and workload tests * expose package scripts for smoke and e2e runs
c3d3d0b to
6d98d20
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BLUF: this PR gives the rest of the stack a benchmark that matches the deployment shape we actually care about. Its first output is a reproducible baseline for the old serving apply path: 247.0 tx/s, 4,939.9 rows/s, 2.561 ms avg VS transaction apply, and 0 sync worker read errors with eight readers sharing the serving SQLite file.
The production shape to keep in mind is:
The benchmark harness in #6070 models this as
shared-replica-sync-workers: one VS-side stream consumer, one serving replica applier, eight reader workers on the same SQLite file, v6 websocket frames, per-message ACKs, and reconnect catchup during load. The point is to avoid measuring an easier but less useful shape where each sync worker gets treated like an independent SQLite writer.This benchmark is worth adding because the real bottleneck can move between the RM and the VS. A fast RM does not help if the VS serving replica cannot apply and publish fresh versions. A fast VS does not help if the RM storer or fanout path cannot keep the stream moving. The harness keeps those parts in one run so we can see which change moved the actual RM to VS path.
The code is intentionally contained under
bench/rm-vs-load.packages/zero-cache/bench/rm-vs-load/scenarios.tsdefines the scenario,packages/zero-cache/bench/rm-vs-load/runner.tswires together the RM, storer, forwarder, stream consumer, reconnect consumer, and readers, andpackages/zero-cache/bench/rm-vs-load/sync-workers.tsstarts the shared SQLite reader workers.Baseline output:
Validation:
Stack context: #6070 adds the benchmark, #6071 reduces RM storer cost, #6072 reduces VS apply cost, #6073 reduces thread-worker handoff cost, #6074 removes the serving-mode thread hop, #6075 improves RM fanout flow control, and #6076 removes subscription queue shifting.