fix(inkless:consolidation): route ListOffsets(EARLIEST) to the control plane for consolidating topics [KC-332]#709
Merged
Conversation
viktorsomogyi
force-pushed
the
svv/ts-unification-listoffsets-earliest
branch
from
July 21, 2026 14:41
d85bf1e to
7310570
Compare
…l plane for consolidating topics [KC-332] PR 4 Problem A: for a switched consolidating topic the earliest offset was served from the broker-local classic UnifiedLog.logStartOffset, which is frozen at the switch on followers, so ListOffsets(EARLIEST) diverged across brokers depending on which replica the metadata transformer routed the client to. DisklessFetchOffsetRouter now routes EARLIEST for every consolidating partition (born-consolidated and switched) to the authoritative control-plane cross-tier earliest, giving one consistent answer on every broker. Non-consolidating switched partitions keep the classic leg while it still owns the pre-switch prefix. Co-authored-by: Cursor <cursoragent@cursor.com>
viktorsomogyi
force-pushed
the
svv/ts-unification-listoffsets-earliest
branch
from
July 21, 2026 14:53
7310570 to
bace6f7
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes inconsistent ListOffsets(EARLIEST) results for consolidating diskless topics by ensuring the request is always served via the control plane’s broker-agnostic cross-tier earliest rather than potentially stale broker-local classic log state (notably on followers after a switch).
Changes:
- Update
DisklessFetchOffsetRouterto routeEARLIEST_TIMESTAMPto the diskless/control-plane leg for all consolidating partitions (born-consolidated and switched), while preserving classic handling for non-consolidating switched partitions that still own a pre-switch prefix. - Update and extend unit tests to assert the new routing behavior and add a regression guard ensuring consistent earliest offsets across brokers with different local classic log starts.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| core/src/main/scala/kafka/server/DisklessFetchOffsetRouter.scala | Changes EARLIEST routing logic to always use control plane for consolidating partitions; keeps classic path for non-consolidating switched partitions while classic prefix exists. |
| core/src/test/scala/unit/kafka/server/DisklessFetchOffsetRouterTest.scala | Updates existing tests to match new semantics and adds a new regression test ensuring EARLIEST consistency across brokers. |
viktorsomogyi
marked this pull request as ready for review
July 21, 2026 15:04
giuseppelillo
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For a switched consolidating topic the earliest offset was served from the broker-local classic UnifiedLog.logStartOffset, which is frozen at the switch on followers, so ListOffsets(EARLIEST) diverged across brokers depending on which replica the metadata transformer routed the client to.
DisklessFetchOffsetRouter now routes EARLIEST for every consolidating partition (born-consolidated and switched) to the authoritative control-plane cross-tier earliest, giving one consistent answer on every broker. Non-consolidating switched partitions keep the classic leg while it still owns the pre-switch prefix.