Repository navigation
fix(inkless:switch): start the diskless leader epoch at a lowered seal [KC-837] - #884
Open
giuseppelillo wants to merge 1 commit into
Open
giuseppelillo wants to merge 1 commit into
giuseppelillo wants to merge 1 commit into
Conversation
…l [KC-837] `lower-seal` moves the committed seal from `S` to `S' < S`, but the diskless log still starts at `S`. When consolidation fetches at `S'`, the first batch starts at `S`, so the leader epoch cache starts the diskless leader epoch `E_d` at `S` and maps `S'` to a classic epoch. `hasConsolidatedDisklessSuffixFromSeal` reads the epoch at the seal, so `maybeTruncateSwitchedFollower` and `maybeReconcileSwitchedLeader` truncate the consolidated records to `S'`, and `divergenceFromCommittedHistory` recreates the replica on a new leader epoch. This hits every replica in phase 2. The leader that the seal was lowered to has `E_d` at `S'` from `makeLeader`, but the restart that the recovery needs bumps its leader epoch. The consolidated batch in `E_d` then replaces that entry with `E_d` at `S`. When the first batch that consolidation fetches at the seal starts past the seal, `ConsolidationFetcherThread` now assigns `E_d` at the seal before the append. The cache then maps every offset from the seal on to `E_d`, as it does when the first batch starts at the seal. Testing - `ConsolidationFetcherThreadTest.testAssignsDisklessLeaderEpochAtLoweredSeal` appends a batch at `10` to a real log that ends at the lowered seal `7` and has a later leader epoch at `7`. The log ends at `11`, and the cache has `E_d` at `7`. Without the change, the test fails with `E_d` at `10`. - `testDoesNotAssignDisklessLeaderEpochWhenFirstBatchStartsAtSeal` checks that a batch at the seal leaves the cache to the append. - `ReplicaManagerInklessTest.testSwitchedReplicaKeepsConsolidatedRecordsAfterLoweredSeal` applies a phase 2 delta with a new leader epoch to a replica with classic records `[0, 7)`, `E_d` at `7`, and consolidated records `[10, 13)`. The replica keeps its log as a leader and as a follower. - No cluster test takes a lowered seal through phase 2. Assisted-by: Claude Opus 5.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
lower-sealmoves the committed seal fromStoS' < S, but thediskless log still starts at
S. When consolidation fetches atS', thefirst batch starts at
S, so the leader epoch cache starts the disklessleader epoch
E_datSand mapsS'to a classic epoch.hasConsolidatedDisklessSuffixFromSealreads the epoch at the seal, somaybeTruncateSwitchedFollowerandmaybeReconcileSwitchedLeadertruncate the consolidated records to
S', anddivergenceFromCommittedHistoryrecreates the replica on a new leaderepoch.
This hits every replica in phase 2. The leader that the seal was lowered
to has
E_datS'frommakeLeader, but the restart that the recoveryneeds bumps its leader epoch. The consolidated batch in
E_dthenreplaces that entry with
E_datS.When the first batch that consolidation fetches at the seal starts past
the seal,
ConsolidationFetcherThreadnow assignsE_dat the sealbefore the append. The cache then maps every offset from the seal on to
E_d, as it does when the first batch starts at the seal.Testing
ConsolidationFetcherThreadTest.testAssignsDisklessLeaderEpochAtLoweredSealappends a batch at
10to a real log that ends at the lowered seal7and has a later leader epoch at
7. The log ends at11, and the cachehas
E_dat7. Without the change, the test fails withE_dat10.testDoesNotAssignDisklessLeaderEpochWhenFirstBatchStartsAtSealchecksthat a batch at the seal leaves the cache to the append.
ReplicaManagerInklessTest.testSwitchedReplicaKeepsConsolidatedRecordsAfterLoweredSealapplies a phase 2 delta with a new leader epoch to a replica with classic
records
[0, 7),E_dat7, and consolidated records[10, 13). Thereplica keeps its log as a leader and as a follower.
🤖 Generated with Claude Code