Repository navigation
perf(metrics): one pass per histogram, and scrapes share the running read - #77
Merged
Merged
Conversation
… already running Each /metrics scrape joined every event, run and visit ever recorded to every bucket bound, and took over 90 s on the shared Postgres; with a scrape every 15 s on two replicas, six piled up and slowed every floor read. A histogram now buckets each row once with width_bucket, and a scrape that arrives while one is answered shares its answer.
|
Lore is reviewing this PR — 29093c7d-9345-4bcb-9fe9-f7add5cb5f20. Submit a review that requests changes to have Lore push a fix for it. Comment |
There was a problem hiding this comment.
Lore review — Approved
Computes histogram buckets in a single pass via width_bucket over negated reversed bounds and coalesces concurrent /metrics scrapes via sharedRead while preserving Prometheus le bucket semantics.
Must fix (0) · Consider (0) · Nits (0)
Reviewed by gemini-4-argon
Submit a review that requests changes to have Lore push a fix for it. Comment @lore review to re-run the review.
Posted by floor, visit d7dc726f-36c7-4a4b-aa19-b06992f6a3d9.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every page that reads the floor took 18-21 s.
pg_stat_activityon the shared lore-db showed fifteen copies of the claim-latency histogram running at once, the oldest for 93 s:/metricsjoins every event (and every run and visit) to every bucket bound on each scrape, Prometheus scrapes each of the two api replicas every 15 s, so the scrapes stacked and starved the floor's own reads.What Changed
histogram(packages/store/src/metrics-sql.ts): one pass, each row's bucket fromwidth_bucketover the negated bounds (keepingseconds <= le), cumulated infoldHistogramRows.sharedRead(apps/api/src/routes/shared-read.ts): a scrape that arrives while one is being answered waits for that answer; a failed read is not reused.Testing
New store tests run the histogram SQL on the test database (inclusive bounds, label grouping); fold and shared-read unit tests; api suite 409 and store metrics tests pass; lint, typecheck, format clean.
🤖 Generated with Claude Code